Resources

Every once in a while, integration becomes the golden fleece in curriculum design, teaching, and assessment. Deans can feel pressured to identify the way the curriculum, and the Faculty, integrates subjects and learning in the curriculum and its course of study. They may feel frustrated when called upon to find ways to demonstrate, through assessment, that integration is taking place: what it looks like, to what degree, for what outcome. This points to a fundamental challenge: in what ways and to what extent are the things learned in seminary transferable (applicable) to ministry contexts? Novice deans often wrestle with some basic questions. What are we trying to integrate--subjects, fields of study, concepts, skills----all of them? Who is supposed to do the integration--faculty? Students? How do we assess integration? What evidence do we look for? How do you grade it? What does it look like, anyway?! Attempts to engage the faculty in how to integrate the curriculum become frustrating and ultimately futile. If you are frustrated trying to integrate, maybe you're focusing on the wrong thing. Transference of learning. A more helpful framework to focus on may be transference of learning. Transference of learning is a powerful indicator of higher order learning. It denotes the ability of a student to take something learned in one context or a particular field of study, and apply it in a different context or field of study. Transference of learning provides evidence of higher order learning: synthesis, imagination, application, innovation, and creativity. Focusing on transference of learning keeps the focus where it belongs: the student's attainment and mastery of learning. One common attempt among faculty is to focus on integrating content learning---concepts from distinct and diverse fields of study. While that is possible, in reality, it rarely happens by intent or design, and more often than not by serendipity, if at all. Busy faculty members rarely spend enough time in conversation among themselves about their own scholarship, fields of expertise or even discuss their own teaching and courses to creatively design integration in a course of study. Focusing on transference of learning can be a more effective framework for faculty discussion about teaching and learning and its outcomes. Ask a professor "How do you strive to integrate what you teach in your course with what students are learning in a course in a different field of study?" and you'll likely succeed in merely stumping the teacher. Ask, however, "What are those things in your course you want to see your students use or apply in other courses as they continue their courses of study?" and you'll likely get a clear and confident response---and a hint about what to look for. While the concept of transfer of learning is easier to grasp than the vague "integration," it must nevertheless be applied with informed rigor in order to be effective as a framework for teaching and learning in the curriculum. Dale H. Schunk provides a list of types and characteristics of transfers of learning from the literature of educational research. The summary below can serve as a guide to develop program level goals, craft course learning outcomes to align with those goals, and point to evidence for assessment. Types and Characteristics of Transfer of Learning Overlap of learning between situations and contexts: the original and transfer contexts are similar ("near") What is learned in one context enhances learning in a different setting ("positive") Knowledge of a previous topic is essential to acquire new knowledge ("vertical") Knowledge of a previous concept is not essential but helpful to learn a new concept ("horizontal") Explicit new knowledge transfers to new task ("literal") Use some aspect of general knowledge to think or learn about a novel problem ("figural") Transfer of learning involves abstraction requiring conscious formulations of connections between contexts ("high road") Abstracting situations from one learning context to a potential transfer context ("forward reaching") Abstracting in the transfer context features of a previous situation where new skills and knowledge were learned ("backward reaching"). Engaging the Teaching Faculty in conversation about transfer of learning can re-shape its thinking about course methodology and learning outcomes. Using transfer of learning as a framework for assessment may make evaluation of "integration" a less stressful and more effective way to measure student learning. Starting Questions What evidence do you have that students use what they learn in one cognate field of study as applied in another? What evidence do you have that students are able to apply what they learn in academic courses in their ministry contexts? What evidence do you have that students are able to apply what they learn in core courses in their field-based studies and supervised ministry experiences? What transference of learning do you wish to see in what students are learning in basic Bible courses to homiletic courses? What transference of learning do you wish to see between acquired academic skills and ministry context competencies? SOURCE: Schunk, D. (2004). Learning theories: An educational perspective (4th ed.). Upper Saddle River, NJ, USA: Pearson, p. 220.  

Theological school deans are not just theological leaders for their institution, they must be EDUCATIONAL leaders. That is, they must implement sound educational practices related to curriculum, instruction, supervision, assessment, and administration. There is a variety of ways to assess the effectiveness of the curriculum, and there are several levels of assessment (program-level, course-level, student testing, student projects, etc.). While faculty members can focus on course-level and individual student learning assessment, academic deans need to focus on program-level assessment in order to evaluate the effectiveness of the school's curricular course of study. Here are ten basic curriculum assessment tools every academic dean needs, nine are covered in previous posts:  Outcomes alignment worksheet Syllabus assessment worksheet Curriculum maps Program-level rubrics Alumni survey Grade Distribution report Program retention and completion rate worksheets Entering student profile Graduating class profile Student course evaluations In this final entry of the series we review: 10. Student course evaluations that are worth the trouble Student course evaluations, a form of indirect assessment, can be a meaningful component of a school's formative assessment of its curriculum. Unfortunately, most course evaluation tools do not provide sufficiently meaningful data to be helpful. One evidence may be how difficult it is for most schools to collect meaningful data from the evaluations. Another is how students tend to see them as a chore, resulting in cursory responses and a low rate of return. Further, in most cases, the data collected rarely is analyzed at depth or used to prompt pedagogical actions for improvement in teaching and learning. Two approaches can help make course evaluations worth the trouble for students and for deans. First, a better-designed course evaluation tool, and second, a procedure that helps ensure a higher response rate. 1. A well-designed student course evaluation tool An effective student course evaluation tool will provide data and feedback that is meaningful. That is, the information from student feedback should address issues of pedagogy that are relevant, measurable, and actionable. For example, the mythical Central Generic Theological Seminary gathers the following clusters of information on its student course evaluations: (1) Student profile information (2) Feedback on program-level goals (3) Feedback on instruction and pedagogy (4) Feedback on instructor effectiveness (5) Feedback on the relevance of the course to the practice of ministry (6) Feedback on program and learning integration. In order to make these clusters meaningful, the instrument focuses on instructional effectiveness and curricular program goals, not on what students "like" or "enjoyed." Additionally, the clusters of items are co-factored to yield meaningful interpretation. See the attached "Anatomy of a Student Course Evaluation Tool" which shows how the instrument is structured by clusters and for co-factor analysis.  Download Anatomy of student evaluation Once the student evaluations are collected (CGTS uses its learning management system (LMS) for its course evaluations) the dean prepares an aggregate report for the Faculty. The aggregate report includes a comparison of selected items over the course of several semesters. The comparison focuses on areas targeted for improvement based on the student evaluations. Here are some examples: In Example 1 the dean compares student responses to degree program goal 1.A across two semesters. Additionally, the report compares two related items by gender.  In Example 2 the dean highlights three instructional items related to coursework (knowledge, principles, skills) and compares the responses over three semesters. This report item shows improvement in the most recent semester indicating that the interventions faculty members applied in their courses to address these issues are having a positive effect.  In Example 3 the dean compares a cluster of items related to instruction over three semesters. This feedback becomes important for the Faculty of CGTS in helping it realize the need to be more overt in applying pedagogical strategies that help students be more aware of the course learning objectives and to create learning experiences that yield a higher response from students about achieving the course outcomes. As a result of this feedback, the dean led the Faculty in applying teaching and learning practices to increase the effectiveness of these items.  You can download a copy of the student course assessment questionnaire here  2. A rigorous procedure for assessment The second strategy employed by CGTS is the implementation of policies and procedures that help ensure a high rate of return on student course evaluations. The school's policy makes completion of the student course evaluation part of the course completion requirements. Students who do not complete the course evaluation do not receive a grade for the course. Additionally, the school has put in place the procedures to help ensure this indirect assessment data set is part of the formative assessment plan. A Student Course Evaluation Sample Here is a sample of student course evaluation questions with questions that cover the various clusters for co-factor analysis.  Download Student Course Evaluation sample

Cláudio Carvalhaes Associate Professor McCormick Theological Seminary All of the schools under the Association of Theological Schools are required to have a very clear and strong sense of educational effectiveness. One of the ways to gauge such effectiveness is by evaluating to what extent a course meets its stated ‘learning

Theological school deans are not just theological leaders for their institution, they must be EDUCATIONAL leaders. That is, they must implement sound educational practices related to curriculum, instruction, supervision, assessment, and administration. While faculty members can focus on course-level and individual student learning assessment, academic deans need to implement program-level assessment in order to evaluate the effectiveness of the school's curricular course of study. There is a variety of ways to assess the effectiveness of the curriculum, and there are several levels of assessment (program-level, course-level, student testing, student projects, etc.).

Molly Bassett Associate Professor of Religious Studies Georgia State University In my last post, I talked about the final class of my new course “Between Animals and Gods” and promised to say more about the informal student evaluations I give at the end of courses. Like students everywhere, mine take

Molly Bassett Associate Professor of Religious Studies Georgia State University At the end of the term, I like to do three things: wrap up the class, ask students to take an informal course evaluation, and eat cookies. (These are the best ginger spice cookies on the planet. You’re welcome.) In

Molly Bassett Associate Professor of Religious Studies Georgia State University On Monday, I told the students that for the first time since I started teaching I was blown away by the entire class’s projects. Their podcasts are fantastic, and you can listen to them here. I’m tempted to keep gushing...

Nancy Lynne Westfield Associate Professor of Religious Education Drew Theological School Though this particular meeting of the Academic Standing Committee was five or six years ago, my memory of a request as filed by a student yet lingers. Bonnie, not her real name, was petitioning for a grade change from..

Theological schools and seminaries have been relative latecomers to rigorous practices of educational assessment. There are varied and plausible reasons for that which "make sense." However, in the current age of higher accountability to accrediting agencies, stakeholders, and educational consumers,...

Eric D. Barreto It’s that time of the semester when fine weather and the end of an academic term meld into one lovely concoction of hopeful anticipation. After a long winter, I am relishing the open windows and the warm sunlight. After a long academic year, the promise of summer...