Course Design: Instructions, Rubrics and Evaluations
This episode is focused on best practices for professors in regard to providing instructions, rubrics and feedback to students.
Show Transcript
[MUSIC PLAYING]
ANDREW COLETTI: Hello and welcome to this episode
of The T in Teaching.
This episode is focused on best practices for professors
in regard to providing instructions, rubrics,
and feedback to students.
In this episode, our host Sarah McCarthy
interviewed Laura Aboyan and Patty Terry.
Dr. Laura Aboyan currently serves
as the Director of Curriculum Management and Assessment
and Accreditation as well as an Adjunct Instructor
in the Department of Management, as well
as the Department of Marketing.
She joined the Fox School in 2015
after previously working in assessment and accreditation
at the University of South Carolina.
She earned her doctorate in higher education from Temple
in 2021.
Patty Terry currently serves Temple University
as the Associate Director of Curriculum Management
and Assessment for the Fox School of Business.
She previously worked at the University of Pennsylvania
Health System as the Undergraduate Medication
Evaluation Coordinator.
Thank you for listening, and please enjoy.
[MUSIC PLAYING]
SARAH MCCARTHY: Hello, and welcome back
to the next episode in The T in Teaching.
I'm Sarah McCarthy, and we have two guests
from Curriculum Management Assessment and Accreditation.
I will let you both introduce yourselves.
LAURA ABOYAN: Thanks, Sarah.
I'm Laura Aboyan.
I'm the Director of the Curriculum Management
Assessment and Accreditation team.
PATTY TERRY: I'm Patricia Terry, and I'm
the Associate Director of the Curriculum
Management and Assessment team.
SARAH MCCARTHY: Thank you both so much for joining us today.
Could you just tell me a little bit more about your department?
LAURA ABOYAN: So Curriculum Management Assessment
and Accreditation is what we do.
And really, it's just a fancy way
of saying quality assurance for all of our academic programs
at the Fox School.
SARAH MCCARTHY: Awesome.
Well, thank you both for coming today,
excited to give our faculty some overview on rubrics, feedback,
any advice you guys can offer.
But we will dive right into the first question,
and either of you could start.
What advice or feedback for useful instructions,
creating rubrics, setting clear expectations--
where would you have faculty start with that?
LAURA ABOYAN: That's a great question,
and I think you mentioned it in the question.
Use a rubric.
If you're doing some kind of project-based assignment,
a rubric is going to really help you focus in on that.
And the key to designing that really useful rubric for both
you and your students to avoid some of those questions
is to first identify the objectives of the assignment.
What is it that you want your students' completed work
to show you?
You can tie these to your learning objectives.
Mostly, you want to try and keep it to about 5 or 6 categories
otherwise it gets a little unwieldy.
So these should represent the most critical knowledge
and skills that are required for the assignment.
Remember that each trait you use, it should be measurable,
and it should be unique.
If you find yourself struggling to exactly define
what it looks like at various levels of mastery,
it might not be a critical component of the assignment.
Or alternatively, if you find yourself looking
at two traits that are similar but represent
different levels of application, you
want to go with the one that's a little higher level.
The lower-level skill or knowledge
is generally an inherent part of that higher-level skill.
PATTY TERRY: And I would add to that making
sure that you give students a clear idea and clear guidance.
And I know one of the things that we'll touch on probably
shortly is that it's important to have
just clear, concise language in your directions,
but also to remember international and English
as second language students to make sure that you're not using
kind of American colloquialism and other kind of slang
that they may not understand or you
may have a general reference to, but they
wouldn't have-- so just making sure that you are making
it plain and simple for the students
to understand kind of where you want
them to go with the assignment.
LAURA ABOYAN: Yeah and just one more thing
that I would add on to that, like Patty said, you want
your language to be clear and concrete and really
student-friendly.
Sometimes, as faculty, we get locked into the theoretical
or the more academic speak.
But you'll be able to avoid questions from your students
if you choose those simple, concrete, less academic,
and less theoretical words.
It also helps if you can use parallel language
across your rubric scale so that--
and across any rubrics you're using
because that will set the expectation for your students.
They'll become familiar with your style,
with your expectation.
And it will minimize some of those questions.
You're never going to eliminate the questions entirely.
So these are just some general tips that can help you.
PATTY TERRY: [LAUGHS]
LAURA ABOYAN: But really, the key
is to make sure that your rubric is
going to reflect what is in your assignment instructions.
They're companion documents.
Think about them that way.
So if you can build it in to your class,
go over the rubric in class.
Give the students the opportunity
to ask those questions while you're there.
It'll go a long way to eliminating
maybe a large volume of emails asking the same thing over
and over again.
And you can also make a short video
going over your assignment instructions and rubrics
so that your students can refer back to it as they
complete their assignment.
SARAH MCCARTHY: That's a great advice, especially the video.
I like that a lot.
We have Zoom.
We have all these tools at our disposal.
We should absolutely be using them.
So when it comes to students, like what they need and want,
do you guys have any feedback on you know
clarifying within the rubrics?
Or I love how you said the parallel,
and it's the companion document to the assignment description.
Do you guys have any advice for--
like from the student side, what they really like
to see in the instructions, the rubrics, or even
feedback on their assignments after completing it?
PATTY TERRY: So I would say--
I'm currently a graduate student,
and I know it's super helpful for me
when I receive timely feedback.
SARAH MCCARTHY: Yes.
PATTY TERRY: So that's one thing that's a little bit important
is to make sure that I'm not waiting weeks
or towards the end of the semester to get the feedback,
because it's frustrating when you get that feedback
and you realize that there are things that you either missed,
or you could have used assistance and done it
in real time and possibly had a better grade or a better grasp
on the concepts that happened in the course.
So if there's one thing I could say about--
on the student side of what is helpful,
it's getting feedback that's timely.
LAURA ABOYAN: I would be inclined to agree.
That's what my students have told me.
And also, I found that the more timely the feedback,
the less likely they are to make the same mistakes
in subsequent assignments.
My students have also told me they
prefer more personalized feedback.
So the more personalized, the better, is what I would say.
And make sure that it's constructive
so that your students have the opportunity to learn from it.
This is where a rubric can be super helpful
because it gives you a starting point for your feedback.
You already know what you're looking for,
and you're able to build on that and point out
specific things and specific areas
where your students can maybe improve.
They know what's expected because they've
looked at the rubric, and then that rubric points to places
where they could do better or maybe even
places where they've excelled.
It's important not just to do the constructive feedback,
but to also point out things that they've done well,
even something as simple as like including grammar on a rubric.
I teach a writing course, so I tend
to fall back on the communication examples
if that doesn't fit what you're doing,
just try and apply it to what works for you.
But if your student has a lot of run-on sentences,
for example, you can rate them accordingly,
however your grammar tree is structured on your rubric.
And then you can leave a note on their actual work
in 1 or 2 places that say, hey, maybe you should combine these,
or maybe you should break this up into smaller portions,
showing them how to fix it.
You don't necessarily have to do it for every instance,
but give them that example they can follow and tell them
to keep looking through it.
And then, if you can, if you find,
as you're going through things, that there's
common things that your students are struggling with,
bring them into the classroom.
Review them generally.
Provide some kind of activity where
they can work on improving that particular skill.
Do you have anything you wanted to add?
PATTY TERRY: I'll piggyback off what
Laura said about the feedback is really
only helpful if you have guidance to figure out
where you made mistakes.
I know, for me, it's very frustrating to get back
that you got a 4 out of 5, and you have no idea where
you missed or what the problems were,
so you can't course correct.
So I think it's important to make sure
that, when you're giving that feedback,
you don't necessarily have to go line by line,
but giving the student an overall idea
if they've kind of met the objectives of the assignment
or they haven't and where they need
to bolster up or get assistance if they're really struggling.
LAURA ABOYAN: Yeah and I think the important thing
to keep in mind, too, is that this doesn't just
hold true for rubrics.
The same thing is true of exams, right?
If the students don't know what they got wrong on an exam,
and they don't know what the correct answer is,
how are they ever going to be able to master that concept?
How are they going to be able to build on it
later in your course or even later
in their own academic journey?
So make sure you're telling them where they need to improve
and so they can understand why.
SARAH MCCARTHY: Absolutely.
Yeah, and I love that you brought up
bringing it into the classroom.
If one student's making the mistake multiple times,
other students probably are.
Bring that in so everyone can kind of
benefit from that lesson.
That's great.
PATTY TERRY: And the student realizes
they're not the only person that's
struggling with that concept or that struggling with that concept or that [INAUDIBLE]..
SARAH MCCARTHY: Yeah, they don't feel singled out
or like it's just me.
That's-- yeah, great points.
I love that you brought up exams.
That segues perfectly into my next question.
When evaluating students, are they really grasping
these concepts, are they ready to graduate or move on
from this class, are exams always the way to go?
LAURA ABOYAN: I think it largely depends on the subject
matter and the level of the course
and also the size of the course.
Determining what assessment is going to be the best one
takes a little work, right?
And you really have to start thinking through what you want
your students to be able to do.
If your goal is to have them apply their knowledge,
are you necessarily going to be able to gauge that
from performance on an exam?
Or would some kind of project or presentation-based assignment
really showcase their learning better,
whether that's an individual project or even
a small group or paired presentation or project.
Sometimes not using exams as the default
also kind of helps motivate the students,
especially if you can design a project
where they're able to choose a topic that they're
interested in.
And we've got a couple of really great examples
of this happening in our undergraduate program.
I'll talk about one.
Then you want to talk about the other, Patty?
PATTY TERRY: Sure.
LAURA ABOYAN: For example, our business communication class,
which, full disclosure, I'm currently teaching-- so again,
came straight to mind--
their big culminating assignment is
to create a business proposal.
They go through the entire proposal process, development
process, from choosing a topic that's important to them,
convincing their audience there is a problem,
and offering a solution.
And there have been a couple of times
that we can point to where proposals
from this particular course have actually gone on
to be expanded and implemented.
Our center for Ethics, Diversity, and Workplace
Culture, for example, hosted an event about a year, year
and a half ago called Voices of Pride
that came out of a student proposal
in an honors business communication section.
Last year, the National Association
of Black Accountants hosted a conference at Temple.
And that came out of a proposal in one of my business
communication sections.
The students are more inclined to put in the effort
because it's something that they feel connected to
and that is really important to them.
PATTY TERRY: And one of our new courses in our redesigned
curriculum is BA 1103, which is a new course that
was designed with Jeff Boles, who is the chair of the Legal
Studies Department.
And their assessment, they use a podcast assignment
where students are to create a podcast discussing
an ethical issue and kind of come up with their own script
and record it.
And students have been--
the feedback, anecdotally, has been really, really positive.
Students like that is kind of a different assignment.
It's not your typical kind of fill out this paper exam.
But they're still demonstrating what they've
learned in that modality.
So it's been really positive to see
that there are different ways to assess learning outside
of just giving them a multiple-choice exam
or high-stakes exams, which we know
from the science tells us doesn't really
work in demonstrating learning.
It causes more anxiety for students to do those things.
So the importance of making sure that, when
we're talking about assessment or evaluating student learning,
is we're giving them an opportunity
to demonstrate certain things.
And a lot of times, exams may not
give them the opportunity if it's the only way
that we're looking at it.
And so I would suggest that, especially
for more quantitative courses that
tend to lean on exams to also think about not making them
as high stakes and then also offering
other ways for students to demonstrate learning,
so other homework assignments or written assignments or in-class
assignments so that students don't feel the pressure
to just have to do well on one exam
to do well or show what they've learned in the course.
LAURA ABOYAN: Yeah, I would be inclined to agree with that.
We know we're never going to be able to scrap exams entirely.
They are important, depending on discipline,
depending on class size, depending on level of course,
subject matter.
We know that.
But the same way you would design a rubric,
you want to be thoughtful and deliberate in how
you're designing your exam.
Think about if you're asking questions
that are really reflective of the way you've
presented concepts in class or in homework.
Or are you focusing on really the most important takeaways
from your course?
Or are you just asking questions for the sake
of asking questions.
Like Patty was saying, consider the length of your exam.
How many questions do you really,
really need to ensure that the students have
mastered the material.
If you're asking multiple questions about the same topic,
why?
Is it because that topic is directly
part of the course learning objectives?
Because it's the most one of the most important things
they're going to take away from your course?
Or is it something a little more tangential
that maybe doesn't need such a heavy emphasis on it
on an assessment?
Much like with your rubrics, you want the exams
to be focused on the most important parts
of your courses.
And the questions should be reflective of the topics
that the students have had the chance to investigate and apply
in other ways throughout the semester.
SARAH MCCARTHY: This is wonderful.
So I'm just going to-- my final question--
as Jeff Boles did with the 1103 new course design,
if faculty want to come and redesign their exams
or see other modes to assess their students-- like you said
that podcast assignment, different.
The students were engaged in it.
That's what we want to do.
Can faculty just reach out to you guys, stop by?
LAURA ABOYAN: Absolutely.
We love the opportunity to collaborate--
SARAH MCCARTHY: [LAUGHS]
LAURA ABOYAN: --if we can and help wherever we're able.
Either one of us can help across any program,
although we each have our own areas that we tend to focus on
just for ease of workload.
Patty is your go-to for all things undergraduates.
Whether that's a core course or a course in the major,
she is here to help with that.
We kind of split our specialized master's programs.
But then anything with the MBA or the doctoral programs,
I can certainly take on.
PATTY TERRY: Our responsibilities here
is to really consult and work with the faculty
to deliver a wonderful, solid educational program.
So where the faculty are the content experts,
were the educational experts.
We're the experts in the delivery.
So what we want to do is find ways
to help faculty deliver in a way that
is palatable for the students, and it
helps them actually learn the concepts
that they're trying to relay.
So the important thing, I think, to remember
is that we're here to work with the faculty,
not tell them how to teach, but work with them on making sure
that they're doing the teaching in the most effective manner.
SARAH MCCARTHY: Absolutely.
LAURA ABOYAN: So they can just send either one of us an email
or stop by our offices.
We're both here a few days a week in person,
and we're located in the dean's office.
So anybody, please come on by.
We love company.
SARAH MCCARTHY: Absolutely.
Well, thank you both for coming today.
This is-- I'm sure the faculty-- this is very helpful
as we get into the semester, we start
thinking about our courses, anything we
might need to change or update.
These are wonderful suggestions and feedback
for the faculty to consider.
So thank you both so much for joining us today.
PATTY TERRY: Thank you.
LAURA ABOYAN: Thanks for having us.
PATTY TERRY: Thank you for having us.
[MUSIC PLAYING]
