Resources

Regardless of how one may feel about online learning (now, during COVID-19, thrust upon us, the willing and unwilling), admittedly it is now a vital and critical academic and professional skill. Helping students become proficient in online learning has arguably become as important as mastering academic content in whatever discipline one teaches. One way to help students become more proficient at online learning is to actively assess their performance in online discussion forums. Most instructors at least provide a list of minimum expectations, something like: Post at least two entries for every forum; avoid non-substantive posts (“I agree”); post by a deadline for a session; cite references, respond to questions from the professor, etc. Some instructors place limits on word count. Some insist on complete sentences and proper grammar. In addition to assessing engagement with the course content (academic concepts and course texts, for example), and checking for adherence to minimum expectations as noted above, instructors can help students become more proficient online learners by assessing metacognition student performance, those transferable skills and competencies that will serve students well as they become lifelong online learners. Metacognitive assessment helps students become critically aware of themselves as thinkers and learners. Robert E. MacDonald refers to these as part of the “informal observations” [i] that instructors engage in as part of the evaluation of student learning. Here are examples of metacognition student performance in online discussion forums that you can look for, assess, and for which you can provide feedback to students: The consistency in the amount and quality of their posts. The quality and kinds of questions students ask during online discussions. The cooperative peer learning skills students demonstrate in discussion forums. The manner in which they receive directions and challenges from the instructor. The way students respond to questions from the professor and other students. Their ability to follow through on assignments and activities to completion. Their level of initiative in asking for help, seeking information, offering critique, and questioning assumptions. Their ability to uncover their own bias and prejudices. Their ability to recognize their misunderstanding and demonstrate corrective thinking. Their ability to come up with novel and original examples. The quality of their written skill in expressing and explaining ideas. Their ability to manage their time and participate in online discussion forums, as well as complete assignments, promptly. It is no longer enough to help our students master academic content related to our particular scholarship. Part of the work of teaching in this technological age is helping our students become better learners, and that includes becoming more adept at learning in online and virtual environments. Notes [i] Robert E. MacDonald,  A Handbook for Beginning Teachers: Facing the Challenge of Teaching in Today’s Schools (New York, NY: Pearson, 1999).

In remote teaching we all wrestle with how to effectively translate our typical assessments of student learning, and possibly, how to create new assessments.  This requires (re)determining what we most want our students to achieve and thinking creatively about how they can best demonstrate their learning in an online environment.  While there are tips to discourage cheating online, many instructors are concerned about academic integrity in online high-stakes, closed-book exams. Because students are more likely to learn from (and less likely to cheat on) assessments that provide authentic experiences or reflection on learning, it is valuable to think broadly about how best to assess our students learning in creative ways online. The principles of inclusivity and sustainability can guide our development of online assessments from the beginning to the end of the course. Sustainable assessments are those that help students develop skills necessary to direct and monitor their future learning.  Shifting students’ expectations from all feedback originating from the instructor to using self and peer assessment will help them reflect on their learning processes, and identify gaps in their skills and understanding. Making this shift will allow students to begin to assess and monitor their own learning, making it sustainable a skill that students can use after leaving your course. Inclusive assessments are those designed to measure students equitably, and in ways that are sensitive to cultural, economic and social differences. Transparency and clear communication regarding expectations for successful completion of work is key so that students understand its purpose, the necessary tasks to successfully complete the work, and upon what specific criteria they will be evaluated. This approach levels the playing field for diverse students while facilitating learning for all students. To apply these principles in your online course: Begin by seeking input from your students At the beginning of a course, surveys can be used to discover students’ incoming knowledge and skills as well as something about them as a person.  Rather than focusing on knowledge ‘deficits’ that the curriculum must fill, this kind of preassessment will allow you to discover students’ interests, lived experiences and motivations.  Equitable assessments should be accessible and responsive to students whose abilities, access to computational tools and reliable high-speed internet, access to quiet work spaces, and extent of flexibility in setting their study schedules may vary. Therefore, it will be important for you to know about your students for your course planning. Ask students to share what they are most excited about in the class, unique skills they are proud of, past educational experiences, and their own perceptions of their current knowledge and skills.  Use students’ input to help them build connections between the course content and their interests. To help them take ownership in creating spaces that welcome all members of the class, invite students to contribute to ground rules for class interactions. Where possible, provide opportunities for students to make choices about an assessment topic or format that leverages their skills and interests to help them feel more empowered and engaged in the course.  Use frequent low-stakes assessment to guide students’ learning throughout the course As your course progresses, assessments are a learning tool that can develop the students’ sense of belonging in the course community, as well as a shared responsibility for and awareness of their learning. There are many ways to accomplish this. Here are a few: Include a variety of low-stakes activities and assignments early and frequently such as short quizzes, reflective writing prompts, group projects, and synchronous or asynchronous discussions. This allows students to calibrate your expectations, get feedback they can incorporate, and understand their individual progress. Use online discussions to build opportunities for interaction that develop students’ sense of belonging, and motivate them to learn from each other in their responses and question of each other.  In both asynchronous discussions and synchronous sessions, guide opportunities for effective peer feedback by modeling it yourself, highlighting examples of productive exchanges, prompting them to ask each other guiding questions and asking them to use rubrics / clear criteria to guide and assess their responses. Before students hand in work, ask them to self-evaluate according to the grading criteria and to identify areas where they would most benefit from your feedback. Culminate the course with authentic applications of course knowledge and skills with integrative assessments Students can demonstrate their achievement of the course goals by applying disciplinary tools to real-world situations, analyzing authentic data or exploring solutions to so called “wicked problems.”  These are problems that have changing parameters, are resistant to solutions, involve incomplete data, or are difficult to recognize (Hanstedt 2018).   Creative projects with formative feedback will support students in developing the sustainable assessment skills necessary for lifelong learning. Consider To synthesize key ideas or reflect on what they’ve taken away, have students write research reports or papers. These can provide opportunities to practice disciplinary language and styles of communication.   To creatively demonstrate learning, have students create artifacts such as maps, figures, photo essays, journals, videos, blogs, podcasts, or portfolios. Structure checkpoints and opportunities for formative feedback to support students in successfully completing these projects.  To learn to self-assess the quality of their work, have students use use rubrics, peer-feedback, and/or compare it to exemplary examples.  Whatever forms of assessment you choose, clear communication is critical to their success.  Consider starting each unit with a brief overview of how all the course components fit together and alerting them to upcoming deadlines.  Make sure to inform students how each assessment will be useful for their learning, make expectations as transparent as possible, and be clear about where students can find answers to questions as they arise.  Learn More The Art & Science of Successful Online Discussions (Faculty Focus) Five Discussion Ground Rules for the Online Classroom (Colorado State University Online Blog) Professors Share Ideas for Building Online Community (Inside Higher Ed) Alternative to Exams for Remote Teaching (Teaching@Tufts) Creating Epic Finales or Limping Across the Finish Line (Teaching@Tufts) Curricula for Wicked Problems (Wicked Problems Project) Transparency in Learning and Teaching (TILT Higher Ed) Inclusive Assessment: Equal or Equitable? (Teaching@Tufts) Selected References Dewsbury, Bryan, and Cynthia J. Brame. “Inclusive Teaching.” CBE—Life Sciences Education 18, no. 2 (April 26, 2019): fe2. https://doi.org/10.1187/cbe.19-01-0021. Gikandi, J. W., D. Morrow, and N. E. Davis. “Online Formative Assessment in Higher Education: A Review of the Literature.” Computers & Education 57, no. 4 (December 1, 2011): 2333–51. https://doi.org/10.1016/j.compedu.2011.06.004. Hanstedt, P. (2018). Creating wicked students: Designing courses for a complex world. Stylus Publishing, LLC.   Kelly, D., J. S. Baxter, and A. Anderson. “Engaging First-Year Students through Online Collaborative Assessments.” Journal of Computer Assisted Learning 26, no. 6 (2010): 535–48. https://doi.org/10.1111/j.1365-2729.2010.00361.x.

Evaluations of faculty, both peer and student, can be a valuable part of the teaching landscape. Without doubt, faculty peers see strengths and weaknesses an individual instructor might not otherwise notice. Similarly, asking students about their experiences yields important insight into how learning happens--if one poses the right questions. Well-designed instruments interpreted with a critical eye by employers should be standard. For online instruction, however, particular downsides emerge that often prove dicey for faculty whose merit pay, tenure and promotion considerations, and, most especially, contract status depends, even if only in part, on the results. Online faculty must proactively design processes that handle the challenges of the online environment appropriately. While I have by no means mastered making evaluation of my online courses helpful, I do have observations about what I have tried and what I am planning to do to negotiate this required process more thoughtfully. When a peer evaluator lacks experience with online learning, or does not have familiarity with best practices for creating, mounting, and running an online course, the utility of the evaluation can come into question. Too often an idealized imagination of the traditional classroom remains the model by which a course is judged, rather than looking at the online experience for what it offers. If a university employs a given standard, like Quality Matters, for online course development, having peer evaluators look through that prism can be useful (https://www.qualitymatters.org/qa-resources/rubric-standards/higher-ed-rubric). The elements outlined provide clear criteria for assessment. Additionally, affording a faculty member the opportunity for peer evaluation from other online instructors or learning technology designers, even if they are not subject matter experts, can also be valuable. My university is now doing “teaching squares” for online instructors where we get together and work through one another’s courses. Seeing how other online faculty structure learning, and hearing feedback about my course format and assignments generated new ideas and helped me see where I needed to improve. Indeed, I would welcome a combination for peer evaluation that blended the insights of other online instructors with respect to the medium and what departmental colleagues would note about content. In the end, these options all stress one thing: that the purpose of peer evaluations ought to be to improve student outcomes by focusing on pedagogical practice. Regarding student evaluations, online instructors already know what studies demonstrate: online courses tend to yield lower ratings for faculty than face-to-face courses. This result likely stems, in part, from the lack of personal contact built up in weekly face-to-face meetings. Relationship mitigate against harshness in evaluation. But this also serves as a reminder that faculty presence, however it is accomplished in online classrooms, is important. While faculty members cannot completely alleviate the electronic remove, students still need to see faculty members in videos and engaged in discussion. They continue to want personal interactions, whether it be through email, text or video messaging, or phone calls. Structuring that contact into the course is vital. Returning work and inquiries promptly also proves important. The urgency of our online lives drives expectations here, so faculty must set up precise standards for communication and stick to them. Students might not be able to get an answer to a question posted at 2:00 a.m., for example, but it is not unreasonable to expect an answer to come within the next 12 hours in most cases. Many students evaluate faculty based on expectations about online courses and they often anticipate online options will be easier than face-to-face classes. Institutional messaging around online options can plant that idea. “Learn anytime, anywhere” our promotions might say, as if students can successfully navigate a class while strolling the aisles of the grocery store or pausing briefly between serving customers on the job.  Faculty, therefore, must establish detailed instructions for assignments and specific rubrics for grading that can help learners understand what they need to accomplish. Still, disappointments in this arena can lead to frustrations expressed via end of term evaluations. More helpful options might include evaluations geared to each assignment, perhaps even completed when the assignment is turned in. What did a student learn from completing that task? How did the assignment relate to the goals of the course? If faculty include these elements in their course design, it can alleviate problems along the way and serve to remind students at the close of a term about what they achieved. Likewise, making students partners in their own learning asks for a higher level of reflective awareness. Foregrounding why a student enrolled in a course, what the student wants to gain by completing it, and how it might relate to overall academic goals can be a great starting exercise because it makes students clarify expectations. Instructors get a snapshot of why students show up and the opportunity to engage in discussion of where faculty expectations may correspond to those of students, where they may differ, and why. Modifications, even if only mental, can happen through this exercise and students can come to see faculty as partners and guides in learning as opposed to obstacles. In the end, we must acknowledge that evaluations will remain a fact of faculty life. How to make them less punitive–not focused on faculty shortcomings or “consumer satisfaction” measures–and more formative, aimed to achieving educational purpose, should always be a goal for faculty and administrators. Adapting the evaluation form to the delivery method of a course is a part of that process.