assessment

Arguably, due to suspicion, skepticism, and the entrenchment of classroom teaching practices, the most scrutinized, assessed, and evaluated pedagogy in theological schools in the last two decades has been online learning (“distance education”). Fortunately, the skepticism that fueled a demand for more rigorous evaluation of student learning in the online environment has yielded hard evidence that online learning is as effective, and sometimes more effective, than traditional classroom teaching.

The goal of this work by Catalano, an associate professor of teaching and learning at Hofstra University, is “to assist researchers of distance education, both novice and expert, in finding well-developed and value measures to suit their research questions” (2). The focus of the book is “on properly validated measures, thereby relieving the researcher from having to develop his or her own instrument” (2).    

Catelano provides a concise guide to seventy selected assessment instruments (surveys, scales, and methods) that have been used to evaluate the effectiveness of online learning programs. The instruments can be applied to evaluate key metrics and pedagogical, as well as programmatic, elements such as: student engagement, student and faculty satisfaction, information retention, self-efficacy, student and teacher readiness, the online learning environment, cooperative learning, competencies for online learning, student attitudes, student retention and attrition, critical thinking, and achievement. Several instruments focus on specific pedagogical approaches such as constructivism and andragogy.

Five chapters provide nine categories of evaluations. Specifically, Chapter 1 provides eleven instruments on student engagement and satisfaction. Chapter 2 contains sixteen instruments on student readiness to learn and self-efficacy. Chapter 3 offers a selection of seventeen instruments focusing on the online teaching and learning environment. Chapter 4 provides twelve instruments on student learning behaviors. Chapter 5 contains five instruments to assess student achievement, retention, and attrition. Obviously, there are enough offerings in these chapters to meet just about any evaluation need an assessor may have. Each chapter includes a brief overview of each instrument covered. Each instrument entry includes a summary of the tool, a description of how the measure is used, and a description of its development and validity.        

Some of the instruments described are complete tools (for example, a fully developed satisfaction questionnaire, a complete survey on student evaluation of online web-based instruction, and one on the online learning environment. Other instruments are described with information about their sources should an assessor wish to use them.         

In the Introduction Catalano provides helpful background on the criteria for inclusion of the instruments selected for the compendium. The author describes the search process used in identifying the instruments, including applying the criteria of ensuring reliability and validity.

For those needing to continue to assess online learning through disciplined and rigorous evaluations, whether as part of formative curricular assessment or to satisfy accreditation requirements, Amy Catalano’s brief compendium of assessment instruments, scales, and approaches is a handy and accessible tool. Indeed, it should be on every evaluator’s desk.

A conservatory of music in my hometown annually brings to campus a famous singer who leads a master class for its voice students. This is a ticketed event open to the public and regularly draws a large audience. It’s simply fascinating to watch the singer teach. One by one, students come on stage, they perform pieces they have practiced for the occasion, and she offers her critique. Occasionally she offers a mini-lecture on some aspect of singing but mostly she makes students work certain sections of their pieces over and over, all the while offering correction, advice, and support. The audience hears how their music – which sounded pretty good, to begin with – improves with her coaching.

While a master class is not the same thing as a studio, the two pedagogies share certain features. In my experience, teachers of religion and theology rarely make use of studio pedagogies; this book made me realize that we should.

Studio pedagogy is typically defined by the following elements: lengthy design sessions conducted in large spaces where materials are readily available and works-in-progress can be publicly and permanently displayed. Instructors roam the space, stopping at individual desks to offer feedback that gets intentionally overheard by nearby students. Lectures and discussions are rare; studio pedagogy relies instead on coaching, modeling, correcting, responding, affirming or questioning choices, and occasionally offering on-demand content instruction. It combines authentic learning theory, constructivism, socialization into a profession, and the theories behind flipped classrooms and communities of practice. It shares features of other student-centered pedagogical approaches such as problem-based learning and service learning, although it focuses more on the process of students taking iterative steps toward a final, deliverable product of their own choice and making.

This book is an edited volume presenting fifteen narratives by design instructors describing the studio courses they teach in fields like architecture, interior design, and instructional design. Contributors describe the joys, challenges, concerns, and vulnerabilities they have experienced through this sort of teaching. Overhearing their honest confessions and reactions is one pleasure of reading this book, and it gives the reader a taste of what being in a studio is like. This volume is also designed like a studio in that its editors explicitly eschew analysis and summary, preferring instead to “curate” the narratives and let readers draw their own conclusions.

Indeed, religion and theology teachers might have to work hard to relate this book to their contexts. It will be most directly applicable to those in field education and those teaching certain kinds of performance or design – preaching, worship, ritual, or religious architecture. Yet its implications are valuable to all who are intrigued by non-native pedagogies. As I read, I kept asking myself, “Why do we keep our critique of student work private?” As one contributor points out, it is often when budding academics begin to share our work publicly that we take it more seriously, find it more gratifying, and believe it has value. Why shouldn’t our students experience the same?

Empowering Learners with Mobile Open-Access Learning Initiatives is a well-designed book giving an overview and awareness to mobile activities as they can be provided in an educational setting. The anthology was compiled by Michael Mills and Donna Wake, both from the University of Central Arkansas. Most of the studies are North American, but there is ample diversity of circumstances in the populations studied and techniques showcased. The book is separated into four parts: practice, curriculum, assessment, and theory.

This book will likely become a historic piece of educational observation on today’s environment, but just as importantly, it is future-looking. So how is the future looking? The authors are clearly optimistic about the future of higher education. The evidence shows the effectiveness of mobile technologies to provide a more equal and motivated voice in society.

Considering the Wabash Center for Teaching in Learning in Theology and Religion’s audience, this book would be most effective for those in curriculum development and assessment. It is easy to read, but scientifically formatted. Each chapter constitutes a separate study contributing to the overall discussion, and new vocabulary is introduced and defined at the conclusion of each chapter. The publisher, IGI Global, is an established publisher of Information Science and this text could be useful even as a textbook for courses in Information Science and Technology.

The book could have been enhanced through a greater diversity of authorship and a wider distribution of geographical locations. Mobile technology is world-reaching, but much of this book’s arguments were grounded in a Western cultural understanding of the world. It would have been helpful for that to have been disclosed in the preface as a both a limitation of this volume and a signal for further study about student learning in online open-access models of education. There was some effort by the authors to attend to issues related to physical and learning disabilities and learning needs of underprivileged communities. However, the only examples the authors provide outside of the United States were Kenya and Portugal.

The real value of the book is its comprehensive structure of presentation and approach to mobile technology as a discipline. It does not make light of the common lay person’s experience with mobile technology. Rather, there is a sense of power behind today’s and the future’s possibilities for reducing social barriers in education. Empowering Learners with Mobile Open-Access Learning Initiatives would be an excellent contribution to a higher education library, and that is said without hesitancy even when the examples of technology in the book could be somewhat fleeting given the rapid changes in technology and online learning.

Few words strike fear in the hearts of post-secondary teachers more than “assessment.” Most faculty in undergraduate and graduate-level education are better-versed in their own specializations than they are in the more administrative aspects of their schools. Assessment often feels like one of those top-down assignments that faculty must add to their workload in order to get ready for the next accreditation visit.

Darla K. Deardorff provides relief from this fear by removing the mystery from the process of assessment. She places it within the healthy context of enabling teachers to see if their work is actually accomplishing what they intend for it to accomplish, and, on a broader scale, if the overall mission of the institution is being achieved. Deardorff takes time to define all of the technical terms so that even those who feel like novices in this domain understand the issues, making it an easy read. The second half of the book is comprised entirely of appendices, full of succinct guidelines on how to create and implement an assessment plan along with examples of tools and processes used at various schools.

The book begins in an engaging manner by using several myths regarding assessment as a foil to make a case for the importance of doing assessment well. The second chapter then looks at thirty frequently asked questions about assessment. Once the reader finishes the first two chapters, he or she is prepared to find out more about the distinctive aspects of international education and learn what goes into creating an effective assessment program – from start, through implementation, and finally to evaluation and revision.

Most of this book would serve as a practical guide for anyone involved in educational assessment, but Deardorff relates the book most specifically to those engaged in international education, by which she means “efforts that address the integration of international, intercultural, or global dimensions into education” (29). This includes schools with study abroad programs, those with a strong international student or faculty presence on campus, or even those that are actively engaged in preparing students to work in other cultures or simply be better global citizens. This adds a level of complexity to the assessment process that traditional models of assessment have not addressed.

While many books on assessment are geared more toward institutional assessment in comparison with other institutions or benchmarks, Deardorff is focused on student learning outcomes and how one knows whether or not they are being achieved. Special attention is given to whether or not methods of measuring these outcomes are both valid and accurate. There is no one-size-fits-all approach to this kind of assessment, but the processes outlined in the book should help any institution create an assessment plan that is both feasible and useful. While it is vital that faculty be heavily involved in this process, a good assessment plan will involve multiple stakeholders, will be integrated into the ongoing program of the school, will make use of both indirect and direct methods of feedback, and will make use of well-planned rubrics.