Overview
When a teacher, at any level of learning, establishes a grading rubric – a set of guidelines for assigning points to different aspects of that which is being measured, there are two essential considerations:
1. How well was that which was supposed to have been learned mastered?
2. What part of the greater learning activity (class) does the assignment represent?
If the course is divided into four self-contained units, each taking one quarter of the term and right answer instruments may be developed, there is really nothing to consider.
Logistics
The DCTE course runs 22-weeks, meets for the first week on campus and for 21-weeks online after that. The portfolio, according to the syllabus, is worth 50% of the grade. It represents nine weeks of collaboration and journaling plus one week of reflecting and writing – roughly 50% of the course time.
So far so good – except there should have been some part of the grade reserved for full group collaboration and self-evaluation of participation there as well. Although full group discussion is noted in the assignment, it cannot be measured at this point in the term. There is always the next iteration.
Learning Outcomes
Upon completion of the teaching/learning sessions, the need for multiple new behaviors should be established. The experience is not about mastery; it is about awareness of need for continuing investigation, experimentation, and reflection. Here are some of the issues:
As a teacher,
· Use a chart similar to Gagne’s Events of Instruction for each lesson presented.
· Begin with a process for engaging students; keep the momentum going.
· Participate in the creation and maintenance of real presences across distances.
· Use the online syllabus as a course contract.
As a learner,
· Work with the instructor to maintain an online community.
· Report all technical difficulties immediately.
· Communicate in a professionally responsible manner.
· Adhere to the rules of the course.
Web 2.0 Tools
Traditions evolve rapidly in the age of information. The traditional format of the experience was that the space for experimenting with online teaching and learning was the discussion forum of an electronic classroom. After several years of reporting on how to use the new tools,bits and pieces of literature began to appear about their value in improving learning.
The time had arrived to introduce a new wrinkle to the experience. Each teacher was charged with the addition of web 2.0 tools, namely a blog, wiki or podcast, in the execution of the lesson. No specific conditions of use were provided.
The Portfolios
From the initial week on campus, it was apparent that all 14 class participants were good citizens. Therefore, it was not surprising that 100% of the assignments were submitted in a timely manner. The required parts of the portfolio were:
1. Comprehensive lesson plan using Gagne’s Events of Instruction. Include a discussion of modifications made during implementation.
2. Copy of syllabus posted in topic. Include a brief discussion of modifications made during implementation.
3. One-two page discussion of web 2.0 tool used to include choice for selection, what took place and thoughts for future work with this tool.
4. One-two page evaluation of your performance as a teacher. Include mistakes, triumphs, and plans for the future. Include a statement on each of your students.
5. Two page evaluation of your experiences as learner. Write a paragraph for each session in which you participated. Name the teacher and the subject.
To a greater or lesser extent, everyone complied with all requirements.
An issue that experience has taught me is that peers tend to be generous in evaluating one another and to excuse less-than-acceptable participation. There are always reasons why something is not done or is done poorly.
People who teach adults were not required to undergo teacher education programs. Formal structures for lesson design and implementation are alien concepts. Some of the events charts were marvelous. Others merely echoed the requirements for what kind of experience belonged in each section. Still others were just wrong. Is that some do not believe in the value of structured planning or are lax habits too hard to overcome with one short exposure? This observation, too, occurs each term.
The experiences with the web 2.0 tools were time consuming, frustrating and marvelous. We worked with free tools only and, as experienced computer users, were often dissatisfied with the way things worked out. Many feared the intrusion of outsiders and tried to protect the integrity of their products by setting up requirements for passwords that confounded the process.
A major issue – perhaps one that might be commented upon – is what interaction goes in which tool. How does the teacher organize the online learning space so that students know what to post where and how the tools might be organized so that interaction is most meaningful?
As a computing technology in education discipline, we have a great deal to learn about enhancing teaching and learning with web 2.0 tools. What we have done this term is to get started along the learning curve.
Grading the Portfolios
Many years ago an adult student walked into my office, sat down, and announced that she was going to share a major concern – and she did! It seems that her work was highly superior to that of everyone else, her solutions were more efficient, her procedures were unique, etc. and the “A” she earned was definitely superior to that of other students in the class who had earned the grade. I did not know what she wanted from me then and still do not know. It is a fact of life that very little is equal or fair or truly representative.
Holding on to that notion, I decided to assign full credit for each portfolio submitted. Everyone, to some degree, performed each required part of the assignment, and everyone was more or less pleased with his peers. In personal communications, I will address specific issues. As to fairness, there remains 50% of the grade to be allocated to two formal papers; here is where the stars will shine and the course grades will reflect a somewhat normal curve.
15 comments:
Dr. Abramson:
When you comment about the student coming in and declaring that her "A" was better than that of others, I think that everyone has probably felt like that one way or another, especially when it comes to group work. Rarely is there a group in which "everyone" participates equally, although you may come across some in which come close. It is the experience in which you are in a group and work your .... off, and then the same mark is given to the whole group when the select few did all of the work.
I think that student just wanted some kind of recognition that she did a great job. The fact that everyone got the same grade probably did not matter that much, unless there was some sort of award or scholarship attached to grades. If there are no awards or scholarships attached - who ever asks to see your transcript? And, if someone does see your transcript, the chances are that they will never know that everyone got an A - they will just think that you were at the top of that curve!! And as for doctoral work - I would expect everyone to be on the higher range of that curve!
Brenda
Dr. Abramson asked, "How does the teacher organize the online learning space so that students know what to post where and how the tools might be organized so that interaction is most meaningful?"
I think this is something we struggle with for each new tool we encounter. I try not to see everything as a "nail" just because I’m enamored with my new "hammer." Yet without pushing the envelope, how do we move forward? How do we use a new tool in our environment to facilitate learning rather than make things more difficult for the student? In many cases, if the technology is not familiar and not part of the subject matter itself, it can frustrate the students and distract from the learning.
Classes like this in which we can experiment without risk are very valuable. I really like being able to try new things to see what works, but I don’t want to risk my students’ learning while I experiment with an approach that might not work. So I try to mitigate the risk by taking baby steps – trying something new here and there without changing too much from what I know works in my courses. I also try to review the literature to find the "best practices" or at least what has the potential to become a "best practice".
Where do we, as a community of educators who are interested in teaching with technology, share our successes and failures? I think Brenda’s idea of having a wiki or some other community space is a great idea for that type of sharing. (Sorry, Brenda, I meant to respond to your posting, but it slipped my mind.) Are there spaces like that out in the larger community?
I guess I have a lot more questions than answers tonight.
Jen
I have hit one of the bumps that some have pointed out as I try unsuccessfully to respond to Brenda.
There are procedures for evaluating group work so that no one gets a grade he has not earned. We can give children gold stars for exemplary work. Adults need to find internal rewards for excellence.
Reflection on Requiring Postings
The teaching and learning exercise was an eye-opener for me. As educators, we sometimes do not reflect on our own habits when talking about our students. Sometimes it is necessary to dig deeper into issues before we judge others. This course and the Online Learning Environment course gave me the opportunity to reflect on these issues a bit more than I did in the past.
When executing my lesson it was a bit disappointing when questions were posted for discussion and no one responded. At the time, I was thinking grimly “the students are not doing it just because it was not counted in the grade”. Then when I was called upon to be the student, I realized if you want participation then it has to be included to some extent in the grade. I was forced on some occasions to act when I really did not have the time but in the end I learned a lot from the interactions. Then an even more shocking realization came to me. I was doing the same thing in my DCTE course where the blog was concerned. I had my list of assignments (for grades) and was so focused on getting those done that I was not spending much time in the blog. I am sure I would have miraculously found some time if it was counted in the grade, as previous courses had required.
I concur with Dr. Abramson that sometimes it has to be included in the grade as well as there needs to be some room for personal choice that individuals make for their own growth. In my opinion, Dr. Abramson found a balance with some requirements (portfolio) and some open forums.
A need to dig deeper: The discussion forum postings within WebCT, where we have other interactions occurring, often produce a wealth of knowledge because we are constantly checking that site. The blog requires a separate effort in another place. The issue is to determine the appropriate balance between the two. It is an exciting new tool, together with the other Web 2.0 tools, and this is just the start of many innovative ways of including it in instruction. We are all in better teaching positions having experienced both sides. Our task is continue the reflection to ensure our future use does in fact enhance the educational experience for our students.
Dr. Abramson noted:
• Begin with a process for engaging students; keep the momentum going.
• Participate in the creation and maintenance of real presences across distances.
• Use the online syllabus as a course contract.
I began my short course on chronic disease with facts to identify the impact and incidence of chronic disease to encourage learning and participation, only to discover that both "students" had personal experiences that needed to be considered. I was hesitant to continue, and I now see that the focus of the material was changed to children. Personally, I appreciated the opportunity to reflect on the portfolio and the teaching/learning process to gain insight into the lessons learned.
The concept of presence identifies a tangible process. I have attended a presentation where the presenter encouraged use of sensory terms, such as "I feel," "I think you are saying...," and "I hear you...", as well as using sensory descriptors. These terms seem to enhance my presence in the online environment, so I intentionally use this technique. Another recommendation was to respond to specifics in student posts in a reflective manner, demonstrating that the teacher had read the materials.
Identifying the syllabus as a contract to the learner identifies the importance of the syllabus in the learning process. I had incorporated a self-evaluation in the short-course assessment tools which could be incorporated in the syllabus to identify learner expections.
Here is another blog lesson...the discussions can not flow based on the topic, but must be generalized based on the blog page. It is difficult to reply to a specific topic sequentially, such as the asychronous threaded discussions within WebCT.
"how would you structure the teaching/learning activity?"
I would have the short courses extend over most of the term to increase the numbers of student learners in each section. Consideration could be given to having a WebCT discussion on "Lessons Learned" from both the teacher and the learner perspectives.
Avril had commented on the necessity of going to another site to access the blog. It may be beneficial to have a link on the Course Homepage for the blog or other Web 2.0 technology to see if this would enhance interaction.
A question that I have had, is if the Web 2.0 formats improve social interaction, what is the expected or the optimal means of closing a post? It seems that I should leave some social salutation.
Best wishes,
Yvette
Assigning grades for online interaction content must be very a challenging and time-consuming chore (That's why I've never attempted it). In effect, every posting or response to a question is a separate essay, each of which would require a separate rubric if one desires to evaluate them consistently. No, thanks.
For adults who are forced to produce work that is viewable to the group, I don't believe "quality of writing" is the problem. We'll always write something that sounds good. I think the trick is to structure the content in such a way so that the students are *required* to read, review, or otherwise ingest a certain minimum amount of material in order to produce a subsequent piece of online interaction that makes sense. After all, we're not going to post something that doesn't make sense.
To that end, I like the idea of online students first doing some of the learning and online activity as their own work product, with instructor feedback. Later, when they're more familiar with the concepts, the students can collaborate on online projects such as research papers and multimedia products. Finally, after significant research and experience with the material, they could be in a position to evaluate each others' work. An important benefit of this approach is that it exercises various cognitive levels of Bloom.
But the strategy begs Dr. Abramson's question:
"How does the teacher organize the online learning space so that students know what to post where and how the tools might be organized so that interaction is most meaningful?"
I've found that the wiki, even the free version, is a Web 2.0 tool that is flexible enough to accommodate a multi-level teaching strategy like I just described. The instructor can create some areas for individual students to post their individual work and other areas for group collaboration. Any area of the wiki can be viewable, commentable(?), and/or editable by others in the class, depending on the instructional goals. In addition to allowing individuals to post content in individual text styles, the versioning power of this tool is great for tracking who contributed what, and when.
I've already started an educational/collaborative wiki for our church board, and I'm looking forward to using it in another class.
Rick
Hi Rick,
In the environment in which I teach -- undergraduate adults (they range from being in their first online class to the last class before they graduate) in 5-week super-concentrated classes for which I must provide weekly grade summaries -- I am required to grade online submissions. There are two papers due each week, and those do fit nicely in the framework of a rubric. But my students also must engage in discussion question threads: there are four questions each week; students are required to answer each of the questions and also post a minimum of ten comments over four out of seven class days. Many students go beyond the minimum; some of the posts are the "I agree" type, though most are suprisingly substantive, and some even include a show of initiative on the part of students via doing a bit of extra reading after a google search.
How to grade these discussions? First, each answer is worth one point, so if a student does a reasonable job of writing a substantive answer (a good paragraph or two), he/she earns full credit. If the answer is just a sentence or two, is superficial in content, is written in careless or slovenly fashion, or is just off the track in terms of the question itself (that does happen), I deduct portions of points. My "rubric" is no more detailed than that; I've read so many hundreds of answers to the questions (I use the same questions each time I teach the class, though most of the questions have been refined by experience to make them more clear, or more apt to elicit thoughtful responses), that I can peg the points pretty consistently. I always get on in there with students, poking and pushing them to elaborate or clarify what they've said; sometimes I have to outright correct them, but I do it by making the correction and then asking a question that will coax a response.
Eventually getting to Trudy's point about adults needing to find some value in addition to, or alternate to, a grade, when I write the weekly summaries, I always note which of their DQ answers were especially well done (that's in addition to their receiving full credit), and say exactly what was missing from or wrong with answers that did not receive full credit.
Here is the backup. Students also receive full credit for fulfilling that participation requirement (10 over 4 days), and I add comments like "nice follow up throughout the DQ threads!" as another bit of extrinsic motivation which I hope they internalize (research shows that motivation stems from intinsic, not extrinsic, rewards).
All those words add up to: give full credit unless there's reason not to; add praise for a job well done to add value to the credit.
Disclaimer: The process I use is not an exact science. But you guessed that!
Marilyn O.
Marilyn, you seem to have an effective system for evaluating discussion board type of interaction. I can see that discussion tools like WebCT would facilitate your approach by tracking who said what to whom and when. The instructor can search the whole discussion board for messages posted by a particular student, and them compile them for evaluation. The "I agree" postings would jump out quickly.
However, this makes me think of another question: Does the quality of the online interaction depend on the communication tool? It seems to me that people are more likely to post the "I agree" or off-track messages when they know that it's going to be buried in some thread, rather than have their messages exposed side by side with other comments in a blog or a wiki.
For a given set of discussion questions, which online tool would promote higher quality interaction: a threaded discussion, a blog, or a wiki?
Rick
Hi Rick,
Each Web 2.0 tool has a particular strength, I think, and no two are alike.
The students in my online classes (University of Phoenix) have a specific participation requirement and work in a very compressed time frame, so discussion actually does tend to be quite lively, and a DQ in one week may still be drawing comments a week or two later, even though a new set or sets of DQs have been posted in the meantime. Our classes are set up for discussion threads. For about 10 years, we used Outlook Express newsgroups that allow for threading, with the entire 5-week class evolving as a single long thread -- making for very easy reading and responding. We've gone now to a somewhat similar Web-based instrument designed by UOP that I don't like as well as OE (the interface is clunkier; access is slower), although it is still a discussion thread/forum model.
I make it a point to ask open-ended, application questions rather than ones that ask for recitation of elements of the week's reading assignment, and that usually draws willing involvement, sometimes quite beyond the minimum requirement. Students often say they like/love the DQs because they are asked to think (what a novel idea!) and because they are asked their opinion of some issue, and then have the opportunity to match their responses against those of classmates. Sometimes they dig in and add further to their original arguments; sometimes they admit to changing their minds after reading others' responses. For example, a current humanities survey class (prehistory to the end of the Middle Ages) is in the last week, and we've been through the major cultures of the ancient world. One of the DQs this week asks what the argument might be for preserving old buildings rather than tearing them down to make way for newer structures, using as an example the destruction of the architect Louis Sullivan's beautiful Stock Exchange building in Chicago (parts of it were saved and installed at the Art Institute of Chicago; an office building was built on the property). The question is intended to push on issues like cultural identity, cultural and social values vs. issues like rights of ownership, authority for making decisions, and so forth. So the questions students are asked to address are part of the issue of instrument effectiveness.
Getting to your question (I eventually do get around to the point!): Wikis are designed for collaboration, because any reader can edit, remove, or add content. I expect you've seen wikis whose owners specifically ask readers to help them build content (an established education blogger sets up a wiki and asks readers to add information he can use to build a conference presentation, for example). So that's not a discussion forum so much as it is a content-developing tool.
Blogs can be set up as very effective discussion tools (for instance, see a political blog like Daily Kos; click on Comments for just about any post and you may see discussions going several hundred posts long. Some of the posts are minimal, even silly; some are a paragraph long. But some are quite lengthy; posters often address each other by name, which "knits" the conversation together; the software the blog uses allows for threaded conversations within the main conversation. However, if a blog does not have comments enabled, of course no interaction can occur so the question becomes moot. And so, I think the real strength of blogs is the platform they provide for extended writing, writing in constructed, shaped essay form that encourages responses in kind.
A well-designed discussion forum, especially one that enables the reader to scroll down through the entire library of posts to the newsgroup as well as click-in, click-out access to individual posts (like our OE newsgroups before UOP shifted to Web delivery)is, I think, the most effective tool specifically for discussion, where input tends to be shorter and the pace of participation faster. In this context, however, a contributing element is the time frame for participation, if one has been set.
Marilyn O.
Marilyn-
With this in mind, why isn't there a Web 2.0 tool similar to Outlook Express for use in education? Does a recommendation need to be presented to some of the developers? What Web 2.0 tool is the closest to what you were using at UOP?
Thanks,
Yvette
I apologize for being so late to respond to the original post here. I really thought I had already, but I don't see my post here.
In any event, I must agree with Marilyn's comment that a discussion forum may be the best tool for a discussion.
"A well-designed discussion forum, especially one that enables the reader to scroll down through the entire library of posts to the newsgroup as well as click-in, click-out access to individual posts (like our OE newsgroups before UOP shifted to Web delivery)is, I think, the most effective tool specifically for discussion, where input tends to be shorter and the pace of participation faster. "
I am currently running an online support environment in wordpress for my students out on clinicals. I really wish I had chosen Blackboard (with all its limitations) for the format instead of a blog format. The students are very enthusiastic but chaotic!
Interestingly, there was no requirement for posting on the student blog. The students have been cut off from each other for 3 months and were eager to re-connect. In addition, they are all looking for jobs and applying for their occupational therapy liscense. I have responded to that interest by posting job openings and directions for getting background checks done, etc. So I suppose in addition to having participation linked to a grade, meeting an intense need may also prompt participation.
I have also been impressed that they have been posting problems with clients and asking for feedback from their peers. This demonstrates the clinical reasoning that I had originally hoped to inspire, but how would I measure this? The amount of postings on a topic is not really what I am after. I want to measure the quality of the reasoning, perhaps according to Bloom's taxonomy. I believe I would need to identify key words or statements that demonstrate a certain level of thinking and then search for those terms in the postings. As already mentioned on this blog, that would be very time consuming.
Has anyone used a tool like N vivo to do a qualitative analysis? I would be interested in your recommendations.
Sorry for the late response. While reflecting on the teaching and learning online lesson, I found it to be difficult to get students to actively participate in discussions. I revised my requirements for my class after experiencing how late submissions were. One can not respond to a non-existing entry. It was like knocking on the door of an empty house, pointless. Would requiring responses create a disscussion that would further learning or rather just confuse the students? My personal feelings on such matters is that is someone has something to say they will say it. Forcing a discussion may just lead to students giving nods of agreement.
Hi Yvette,
Re your question about an Outlook Express-like tool in Web 2.0:
The UOP is closing out OE as its platform for online classes, and is substituting a platform devised by its own tech people. I've used it for a dozen classes, now (UOP classes are only 5 weeks), and I don't like it -- it's more like WebCT, though not as clunky. To tell you the truth, the system that GSCIS used when I started the program (in 2003) was about the best I know of in terms of ease of use and options for filtering posts in various ways, plus HTML capability. But that was set aside in favor of WebCT, which admittedly is one-stop shopping (as OE is, to some extent) for forums, email, and so forth.
There are various programs out there that feature uncluttered forums plus various desirable features, not that I can think of any at the moment. However, if you can subscribe to a really active listserv of a state's tech directors, you can pick up all sorts of good information. The state of Maine has an excellent such listserv; I subscribed when I was working on my dissertation. The discussion in that listserv is carried on by a good many very knowledgeable people; you don't have to be a Maine educator or belong to the Maine ed. association to be a subscriber. The acronym is ACTEM; I'll see if I can find my subscription info. Meantime, you can probably google for it.
One more thing comes to mind: Bill Fitzgerald is a force behind Drupal, a multi-faceted teaching and learning tool that combines blogging, wikis, and by now no doubt other features. If his resource is not what you're looking for, send him a note asking if he knows of and can recommend an instrument. He'd probably respond in a helpful way!
Marilyn O.
The student according to Grant and Dweek ( 2003) was probably motivated by proformance-approach goals. These type of students often desire to perform better than their peers. These students tend to express their superior ability and strive to have their ability validated.
Grant, H. & Dweek, C.S.(2003) Clarifying achievement goals and their impact. Journal of Personality and Social Psychology, 85, 541-553.
Post a Comment