Course Design: Micro Quizzes & Exams

Audio Transcript

00:08–00:37Andrew ColettiHello and welcome to this episode of The T in Teaching. This episode is focused on micro quizzes versus exams. In this episode, our host, Brendan Goggin interviewed legal studies professor Jeffrey Boles. Jeffrey Boles joined Temple University as an assistant professor of risk actuarial science and legal studies in 2011 and became an associate professor in 2017. He earned a Ph.D. in psychology from the University of California, Berkeley, in 2006 and earned various research fellowships through his time at Temple. 

00:37–00:41Andrew ColettiThank you for listening and please enjoy. 

00:44–00:51Brendan GogginWelcome back to the T in Teaching. I'm Brendan Goggin and our guest today is Jeff Boles. Jeff, welcome. Thanks for being here today. 

00:51–00:52Jeffrey BolesThanks for having me. 

00:52–01:11Brendan GogginSo in this episode we are going to discuss the benefits of micro quizzes compared to cumulative finals and considerations when designing assessments that work best for students. Well, let's get right into it. I guess the first question I have for you, Jeff, is what are the different ways you assess students in your course? Mostly exams, quizzes or both? 

01:11–01:46Jeffrey BolesWell, I teach undergraduate and graduate courses in the disciplines of business law and ethics and for graduate courses and upper-level undergraduate seminars. I see student learning primarily through their written work product, which may be case analyzes, executive reports, discussion board responses, as well as in class presentations and experiential learning projects. These may be individual or group based assignments for any course I teach, I focus my assessments primarily on writing assignments as well as class participation. 

01:46–02:20Jeffrey BolesI'm a strong proponent of the writing to learn approach. As research shows, this approach promotes active learning. The writing assignments help students think through key concepts or ideas presented in my courses. So I don't use exams or quizzes in the graduate courses or upper level undergraduate seminar courses I teach. In terms of assurance of learning, exams and quizzes are not necessarily the most effective assessment tools to ensure that the learning goals are being met primarily if the exams are structured through multiple choice and true false. 

02:20–02:35Jeffrey BolesThat stated, I'll use exams and quizzes to a limited extent in introductory undergraduate courses though, and in those courses exams and quizzes will never constitute more than about 30% of the student's final grade for the course. 

02:35–02:40Brendan GogginSure. And in that light, how is it helpful for students to vary the assessments they complete? 

02:41–03:12Jeffrey BolesIt's extremely helpful and important, I believe instructors should always use more than one assessment type to provide students with different opportunities to demonstrate their knowledge. Research shows that variability in students cultural, linguistic and disciplinary backgrounds affect their performance on and motivation for different assessment types. So for instance, what may be an easy task for native English speaker could contain barriers for a non-native English speaker. 

03:12–03:18Jeffrey BolesThat's why it's so important to provide varied forms of assessment as it helps level the playing field for all students. 

03:18–03:23Brendan GogginThat's a great answer. And what kind of feedback have students given you about this approach? 

03:23–03:45Jeffrey BolesWell, I've taught thousands of students over the years from large lecture hall to tiny seminar engagement. Students have always been pretty open and will gladly share their views about what helps them learn. My students have been particularly vocal to stress that they do not prefer what I call the traditional testing approach to midterms and a final exam a midterm. 

03:45–04:13Jeffrey BolesThey test on three or more chapters or units worth of material. And students have reported that the amount of material they have to prepare for a single exam is excessive and doesn't help their learning. And students tend not to retain the content shortly after the exam. So if exams are to be used from a student perspective, as I've learned, students prefer an incremental approach where they're tested after each chapter or unit of material. 

04:14–04:38Jeffrey BolesYears ago, I was staff to teach a traditional lecture style course that followed the two midterms and a final exam approach with about 12 textbook chapters covered in the course. After about a semester, I revise the course assessments by breaking down the two midterms and final into 12 quizzes, one for each chapter and added other assessment types in the mix. 

04:39–05:13Jeffrey BolesThe student feedback was extremely positive and it was a more effective way to employ the exams and quizzes approach. I want to point out, Brendan, that research shows that for many courses of varying format and size across disciplines, reasonable alternatives to traditional testing exist. And in fact oftentimes the alternatives may be even more effective in promoting student learning and be more of authentic means of students demonstrating what they've learned at higher levels of Bloom's taxonomy synthesis analysis and Evaluation. 

05:14–05:34Jeffrey BolesHere's what I would relate to all instructors listening. Unless there is a solid pedagogical reason for a comprehensive, high stakes test like a midterm, you might consider a series of shorter tests throughout the semester. You can always add one or two questions relating to previous units in the course to make it more cumulative. 

05:34–05:39Brendan GogginWell, that's very salient. As far as your prep time is concerned. What does the workload look like for you? 

05:40–06:04Jeffrey BolesI find the workload to be very manageable. I'm the chair of the Department of Risk Actuarial Science and Legal Studies, and I lead the department by example, including through spending an appropriate amount of time updating and revising my courses. The Fox School has a rule of thumb. The faculty should be devoting about 10 hours a week to each three credit course. 

06:04–06:19Jeffrey BolesThey teach cumulative of classroom hours and prep time. We are very fortunate to be entrusted to educate our students and they deserve our full commitment to ensure that their education is top notch. 

06:19–06:29Brendan GogginIt's a great answer. And as far as the benefits you've seen from using micro quizzes compared to the large stake exams that so many people are used to. What are you seeing in that regard? 

06:29–06:57Jeffrey BolesWell, notwithstanding my earlier critiques of traditional testing, I'm a fan of micro causes. A micro quiz might be five or so questions in a low stake, low stress context that can be used effectively in a number of ways. I now follow in my classes a flipped classroom approach for the courses I teach, where students learn by doing specific engaging in group projects and class. 

06:57–07:19Jeffrey BolesIn order to be successful at these group activities. The students need to have read the assigned material for that week. I'll assign a micro quiz at the start of the week that tests on the assigned material in order to motivate students to complete the reading before class. As they need to understand these concepts before they roll up their sleeves and work on the group activities. 

07:20–07:30Jeffrey BolesIf an instructor is unsure whether students are grasping a particularly thorny concept, a micro quiz can be an effective tool in that situation as well. 

07:30–07:39Brendan GogginThat makes total sense and you touched upon it, but when is the right time for micro quizzing? What is the best approach or method of presenting MICROLEARNING? 

07:39–08:10Jeffrey BolesYes, I feel like it certainly is the right time. And from a student's perspective, I think it's key that they are notified at the start of the semester that the faculty member will be engaging in quizzes. They always want to know. And a common classroom motto of mine for the students and myself is forewarned is forearmed. So if they know they're going to be quizzes in the class, that helps set their expectations. 

08:11–08:43Jeffrey BolesAnd even with pop quizzes, I have a colleague at another university who tells her students that for any class period, there is going to be a 45% chance that the students will get a quiz on the material that was to be covered in that class. She does not determine that chance herself. She goes on a computer application, she types in 45%, and the program will determine whether or not there will be a quiz based on the percentage. 

08:44–09:11Jeffrey BolesAnd she reports the students are quite active in staying on top of the material, given the fact that there might be a quiz. So that's one way to go about it. I think at the end of the day, though, quizzes are a tool that is in the toolbox for any faculty member to help with assessment and ultimately determine are students learning what you're setting out in terms of the course, learning goals. 

09:12–09:35Brendan GogginAnd to touch upon that as an instructional designer here, as a faculty that's teaching online, [they] are fortunate, sometimes, I like to think, to have an instructional designer to work with them. And we're coming from our department in ODL where we're seeing and I'm having faculty ask me a lot to help them design more micro quizzing and get away from the big cumulative effects of large midterms, maybe two more times in a final. 

09:35–09:49Brendan GogginSo this will… I think it'll resonate with the faculty at large. What are the benefits of an end term cumulative final? You know, what helps determine when to assign larger committee exams compared to multiple shorter quizzes? 

09:49–10:26Jeffrey BolesWell, I think a cumulative final exam could be beneficial if it's designed in a way where students can synthesize the different concepts they've learned throughout the term and make connections across the material. So in my experience, I've seen little benefit of assigning large cumulative exams over multiple shorter quizzes. If, again, this final cumulative exam is simply multiple choice and true false questions, there's very little sophisticated analytical thinking that can be demonstrated through such testing. 

10:26–10:52Jeffrey BolesBut if the cumulative final allows for more writing, let's say essay, I think that is a far better platform for students to demonstrate what they've learned in a cumulative way. Because again, I think the point of the cumulative approach is to allow students to be able to draw connections across the different weeks, across different units, to show more higher-level thinking. 

10:52–11:03Jeffrey BolesAnd if you wanted to do an exam to do that, you have to vary your exam type questions to allow for more synthesis analysis and the like. 

11:04–11:13Brendan GogginJeff what piece of advice would you give to faculty and professors that are thinking from migrating from larger cumulative exams to micro quizzes? 

11:13–11:50Jeffrey BolesI would relate to the faculty to take your time with it because it's going to be a big shift in terms of the assessment model and it's not something that you would want to rush out. So I would give certainly a set number of weeks before the start of the semester so that you can take the time to thoughtfully draft the micro quiz questions, certainly set it up in canvas and sort through the technicalities, make sure that everything is structured appropriately and it's ready to go and then update your syllabus accordingly. 

11:50–12:16Jeffrey BolesSo because once the semester starts, you're pretty much locked in, of course, to what you have laid out in the syllabus. And the students are going to set their expectations accordingly, I think. So I would say be kind to yourself in the sense of giving yourself plenty of time, let's say about 6 to 8 weeks over the summer in order to make that transition, because it's not something that I think can be rushed. 

12:17–12:17Brendan GogginVery good point. 

12:17–12:45Jeffrey BolesThe second piece of advice that I would give to faculty is to be mindful that there are plenty of resources and teaching experts here at Temple to help. First, we are very fortunate at the Fox School to have a number of talented course designers. Brendan, you and your team. I found it to be a joy to work with instructional designers in connection with setting up my courses and implementing them. 

12:45–13:30Jeffrey BolesYou are the experts in terms of educational technology, and I learned so much from the designers in terms of new features of canvas. How are online learning platform can be used as a tool to help me do my job and help students understand what's expected of them. And so if we're thinking about modifying assessment types, if we're thinking about converting from large exams to quizzes, don't be afraid to reach out to online digital learning group and the course designers because you have a wealth of knowledge and it's always such a joy to work with your team personally and I would say, second, there are at Temple a number of other resources available. 

13:31–14:18Jeffrey BolesWe are also very fortunate at Temple to have the Center for Advancement of Teaching (CAT) that is a wonderful group of scholars who put forth a number of workshops, seminars, programs, activities for all faculty throughout the university, some phenomenal experts in that group and I and it's a joy to work with them as well. I try to go to at least two or three workshops from CAT a semester because they keep me on my toes and it's important for me in terms of my professional development to make sure that I continue to learn how to do better at my job as a teacher. 

14:18–14:30Jeffrey BolesBecause ultimately, at the end of the day, it's a big responsibility so I'm going to do everything that I can, but I'm also going to lean into the resources here and I encourage all faculty to do so as well. 

14:30–14:37Brendan GogginI'd like to thank you so much for taking the time for joining us here at our studio. And we appreciate it. 

14:37–14:40Jeffrey BolesMy pleasure. My pleasure. Great talking with you, Brendan. 

14:40–14:40Brendan GogginLikewise, Jeff.