Arguably, due to suspicion, skepticism, and the entrenchment of classroom teaching practices, the most scrutinized, assessed, and evaluated pedagogy in theological schools in the last two decades has been online learning (“distance education”). Fortunately, the skepticism that fueled a demand for more rigorous evaluation of student learning in the online environment has yielded hard evidence that online learning is as effective, and sometimes more effective, than traditional classroom teaching.

The goal of this work by Catalano, an associate professor of teaching and learning at Hofstra University, is “to assist researchers of distance education, both novice and expert, in finding well-developed and value measures to suit their research questions” (2). The focus of the book is “on properly validated measures, thereby relieving the researcher from having to develop his or her own instrument” (2).    

Catelano provides a concise guide to seventy selected assessment instruments (surveys, scales, and methods) that have been used to evaluate the effectiveness of online learning programs. The instruments can be applied to evaluate key metrics and pedagogical, as well as programmatic, elements such as: student engagement, student and faculty satisfaction, information retention, self-efficacy, student and teacher readiness, the online learning environment, cooperative learning, competencies for online learning, student attitudes, student retention and attrition, critical thinking, and achievement. Several instruments focus on specific pedagogical approaches such as constructivism and andragogy.

Five chapters provide nine categories of evaluations. Specifically, Chapter 1 provides eleven instruments on student engagement and satisfaction. Chapter 2 contains sixteen instruments on student readiness to learn and self-efficacy. Chapter 3 offers a selection of seventeen instruments focusing on the online teaching and learning environment. Chapter 4 provides twelve instruments on student learning behaviors. Chapter 5 contains five instruments to assess student achievement, retention, and attrition. Obviously, there are enough offerings in these chapters to meet just about any evaluation need an assessor may have. Each chapter includes a brief overview of each instrument covered. Each instrument entry includes a summary of the tool, a description of how the measure is used, and a description of its development and validity.        

Some of the instruments described are complete tools (for example, a fully developed satisfaction questionnaire, a complete survey on student evaluation of online web-based instruction, and one on the online learning environment. Other instruments are described with information about their sources should an assessor wish to use them.         

In the Introduction Catalano provides helpful background on the criteria for inclusion of the instruments selected for the compendium. The author describes the search process used in identifying the instruments, including applying the criteria of ensuring reliability and validity.

For those needing to continue to assess online learning through disciplined and rigorous evaluations, whether as part of formative curricular assessment or to satisfy accreditation requirements, Amy Catalano’s brief compendium of assessment instruments, scales, and approaches is a handy and accessible tool. Indeed, it should be on every evaluator’s desk.

Do students learn as well online as they do in the face-to-face classroom? Miller argues not only that they can, if the instruction is well-designed, but also that online tools and resources are uniquely able to deliver instructional experiences that take advantage of the innate properties of human learning. “[W]hat technology allows us to do,” she claims, “is amplify and expand the repertoire of techniques that effective teachers use to elicit the attention, effort, and engagement that are the basis for learning” (xii).

Miller realizes that many faculty do not share her positive view of technology in instruction, however. She opens her book by targeting that resistance with well-reasoned and supported counterarguments: in chapter 1, she counters the fear that teaching with technology is a faddish, passing phenomenon with a recounting of its potential lasting value to both administrators and instructors; in chapter 2, she presents evidence for the effectiveness of online learning and against typical worries about its effects on students; and in chapter 3, she debunks several major myths about learners and computing that have made instructors skeptical or misdirected their efforts.

Having assuaged common concerns about instructional technology, Miller moves on in chapters 4 through 8 to discuss current psychological theory on the functioning of attention, memory, reasoning, the effect of multimedia elements on learning and motivation, paired in each case with practical strategies to help online instruction optimize these elements. Her evidence is compelling even as it is presented at an accessible level for an audience with little to no background in psychology. Finally, in chapter 9, she presents possibly the most valuable portion of the entire work: a demonstration of all of her principles in practice, in the form of a sample design process and syllabus annotated to show their use of cognitive best practices.

The value of Miller’s overall guidance is only slightly undermined by its fuzzy definition of learning for these purposes. Even as she describes how technology can impact learning, learning itself seems to be equated variously with retention, course completion, grades, and reasoning at different points in the text. In all fairness, each of these measures may well suffice at times for the instructor of a large introductory psychology class, which is the primary use case Miller discusses in this book; and to her credit, she does acknowledge the limitations of each of these as measures of student learning, and also explicitly notes the need for learning experiences that foster deep critical thought, and that engage students at the highest levels of Bloom’s Taxonomy. Nonetheless, a more intentional definition of what is meant by learning would do a great deal to strengthen her argument that online resources can support it.

Despite this limitation, however, this book is a useful, readable guide to an area of instruction whose study is still in its early stages, unusually and gratefully practical even while firmly grounded in theory. It provides a valuable reframing of, and supplement to, the core principles of instructional design, and I expect it to inform my online and even face-to-face instructional practice for some time to come. I would recommend it for any academic library and for the personal collection of any instructor teaching, or considering teaching, in an online or blended environment.