AI and ML in Education
In this episode we revisit a topic from Season 1: Artificial Intelligence and Machine Learning. This time, we want to see how its being taught and the challenges it poses.
Show Transcript
[MUSIC PLAYING]
DREW: Hello, and welcome to this episode of The T in Teaching.
This episode is focused on a topic
we covered in season one, artificial intelligence
and machine learning.
This time, we wanted to discuss specifically
how it is being taught and used in education.
For this episode, I hosted three experts and applicators
of AI and ML--
Todd Schifeling, Konstantin Bauman, and Jaehwuen Jung.
Dr. Todd Schifeling joined Temple University in 2017
as an Assistant Professor of Strategic Management
in the Fox School of Business.
Prior to joining Temple, Dr. Schifeling
earned his PhD at the University of Michigan in sociology.
Dr. Schifeling currently teaches a graduate-level course
called Getting Your Hands Dirty: The Craft of Data Management
and Analysis, which helps graduate students use
AI and ML to answer various questions
in their field of research.
Konstantin Bauman joined Temple University back in 2018
after doing postdoctoral research
at New York University.
Bauman's research interests are primarily
in areas of technical information systems,
with focus on fields of quantitative modeling, data
science, and specifically developing novel machine
learning methods for predicting customer preferences.
Jaehwuen Jung is a PhD student at the Fox School of Business,
studying information systems.
Her primary research focuses on economics
of artificial intelligence and market design.
Her and Konstantin Bauman recently published a study
on the effectiveness of AI chat bots
in educational assessments.
Thank you for listening, and please enjoy.
[MUSIC PLAYING]
Hello, and welcome back to this episode of The T in Teaching.
In this episode we get to visit a topic
we talked about in the first season, which
is AI and machine learning.
In that episode, we talked a lot about what
AI and machine learning are, what
the function and the future of that technology will be.
But this episode, we want to revisit it but instead
talk more about how it's being taught and the challenges
that it poses to the teachers and students alike.
With me, I have three guests who are experts in AI and machine
learning.
In fact, Konstantin, one of our guests,
hosts and runs a workshop every month
for the Fox faculty on AI machine learning,
which I was lucky enough to be able to attend.
In it, one of the leading researchers, Sudipta Basu,
said we're opening up a new toolkit with AI machine
learning, which I thought was a really interesting concept
because inside of a toolkit are a lot of different things.
So let's start with this idea that AI and machine learning
is this really big toolkit and has a bunch of different things
that we're going to be working with.
Todd, can you tell me a little bit?
Because you start and you teach this class
that's kind of an introductory-level course
on machine learning.
Tell me about what you're teaching,
and kind of how you go about it, and what
do the students like about that course.
What are the challenges with it?
TODD SCHEIFLING: Thanks, Drew.
I feel so lucky to be able to teach this course because I get
to work with students from across the entire Fox
School of Business.
Last semester, we had students from accounting, finance,
marketing, MIS, supply chain management, sports tourism,
hospitality management-- every area of the business school.
And so they come from all these different perspectives,
backgrounds, and interests, and the research
questions that they're working on.
And I agree with that toolkit metaphor
because what I try to do in the class
is provide students with lots of different tools
that could be developed in lots of different ways,
recombined in different ways even.
And the whole point is that these are PhD students.
So they're working on creating new research, new findings,
new insights about the world.
And one of the great ways to do that
is to come up with new methods.
New methods enable you to answer new questions that
weren't previously answerable or weren't answerable
at the same level of insight.
And so that's what students are really excited about with AIML
is they can construct new variables, measure things
in ways that hadn't been measured very well before,
and that way answer new questions.
So this last semester, students were
working on all sorts of different things related
to their interests with AIML.
They were studying public companies and their disclosures
about climate change and how that
matches what they're actually doing about climate change.
Using text mining methods, they were
studying how COVID-19 changed how renters think
about apartments and what they're
interested in with apartments.
They're studying the rise of video assistant referee VAR
systems in soccer and how fans react to that
and understand the fairness of those systems
and who's responsible for them.
So so many different methods enable
students to ask and answer new questions,
measure things in new ways, and create new insights,
and ultimately have very successful careers from that.
DREW: I mean, it really seems like regardless of what field
you're in there is an application for AI and ML.
And your class lets these students
from all these different types of backgrounds kind
of find that.
Now, you've told me you've taught this for three or four
classes, right?
So you're seeing how it's being used.
Has that really made you change the way you teach it
as you see students kind of take these methods,
and evolve them, and apply them to different areas?
And if so, how have you seen that change in your class?
TODD SCHEIFLING: So it's a really exciting time
to work in this area and teach in this area
because every year it becomes easier and easier.
DREW: Mhm.
TODD SCHEIFLING: The threshold to start
doing these types of analyzes declines
every year as this tremendous community of researchers,
developers, create code and tools that are typically
very open and a tremendous emphasis on collaboration
and providing access, building on each other's work.
So this is really a community in which
you're encouraged to not reinvent
the wheel but take what other people have already created
and run with it.
And so every year, it becomes more and more
feasible for students to collect large amounts of data,
construct new variables, incorporate
increasingly ambitious or sophisticated techniques
in their research.
And I see that in the semester as well
that I provide some tools that we teach in the class.
But then students also bring in a lot of other tools
that they know about and they discover,
and find easy on-ramps, and bring those to the class
as well.
So just to give you one example, the really simple things
that people were doing to understand
what is the meaning of a sentence
is this dictionary-based approach.
So you have a list of words, and you count how many times
those words were in the corpus, or the paragraph,
or the article.
And now you know something about the meaning of that text.
And that's something that we still teach.
Students should understand that, but now there's
all of these deep learning algorithms that
are able to much more accurately detect the sentiment of meaning
of sentences.
And those have become tremendously accessible
so that students can just pull that down, and incorporate,
that into their code, and have much more accurate sentiment
classification and analysis.
DREW: Wow.
That actually sounds really fascinating from an educator's
perspective because you're seeing everything
that you care about that you're teaching change
and be adapted and operationalized
by your students in front of you.
That's got to be really interesting, at least
from the educator perspective.
So let's talk about the student perspective.
Jaehwuen, you're a PhD student, and you're
working a lot with AI and ML.
And as Todd just said, it's changing a lot.
What is that like?
Because I don't know if that's an experience
that any other field really sees that often.
So can you talk a little bit about that?
JAEHWUEN JUNG: I mean, as a student,
I use ChatGPT very often.
But also I explore on other tools like Cloud AI
and also Gemini from Google.
And I think from research perspective
it's really allowed me to explore topics
more efficiently.
For example, if I'm interested in the technicality
behind the GenAI or the large language model,
I can just ask ChatGPT.
Explain me the technicality of the large language model
as if you are explaining to a high school student.
That is so easy to understand, so I can personalize
the information up to my needs.
It's very efficient.
And also from learning perspective,
I generate a lot of examples to understand the concept.
So I really want student to use ChatGPT very effectively
for their learning.
Yeah.
Super, super good tools indeed.
DREW: Yeah.
I mean, it seems like you're making really good uses of it.
And, obviously, again, it's only been a few years
that you're making this much use of it.
So I'm sure that that's going to expand as time goes on.
But you talk a little bit about research
and making it easier to start your research.
And you Konstantin recently did some research
on just how does the use of AI and machine learning
affect education.
Do you guys want to talk a little bit about that?
Because I think that's an interesting concept.
Yours was specifically in one field, right?
It was programming, right?
But obviously, the ramifications go much larger.
It can apply to any field.
So why don't you guys talk a little bit about that?
KONSTANTIN BAUMAN: So the study that we are running
and what we exactly trying to understand
is how the existence of large language models
can affect or support students in their learning,
not the quality of the thing that they are producing.
They write in a search.
ChatGPT can support them, help them to write it
instead of them or something but on their improved ability
to do something themselves, on their improved knowledge base.
So that's what we are studying.
We are trying to understand can we
find the right way to use large language models to really
support students in learning, not just simplifying their life
to pass all the tests because tests are kind of outdated.
They were all developed before the existence of ChatGPT.
Nowadays, as educators, we have to adapt.
We haven't done that so far.
So we evaluate knowledge of students
based on regular tests.
We need to understand how ChatGPT changes student ability
to learn things.
So that's what we are trying to learn
to understand in our study.
DREW: That gets the same point that Todd was talking about,
that it's all changing.
And it's changing very quickly, but we also don't necessarily
have the answers yet.
And I think that was kind of the result of your study,
if I'm not wrong, was that the results were
kind of inconclusive and that there needed to be replication
and some alteration.
But you know I have to be that guy.
I'm going to push a little bit further.
What do you think from what you're seeing right now?
Where are the opportunities for it to be used better
or to be how maybe assessments may get
changed for the emergence of AI and machine learning?
How do you see education changing with this technology
being used?
KONSTANTIN BAUMAN: It's changing from multiple perspectives.
One thing-- we need to prepare our students
for the future work.
They are future employees, and as future employees
they will use this technology in their workplace.
So they have to know how to implement it.
Let's say they need to write 100 emails during the day.
It's impossible to do it right now,
but with ChatGPT you run the prompt.
You get the initial draft.
You write each email within like half a minute.
DREW: Mhm.
KONSTANTIN BAUMAN: And then it simplifies your work.
It makes you more effective, so our students
need to learn how to use this technology to do a better job.
Recently, I just read the paper about the copilot
implemented in GitHub.
So they tested how it affects the programmers.
They run a large experiment, and what the AI actually
do in there-- like, programmers type the initial line
of the code, and either Copilot generates
the next line of the code, and they can accept,
or they can change the prompt to tell what
they are planning to implement.
And Copilot would produce another code.
So it's kind of helping them, and they're
very positive results from there.
Like, they write more effectively.
They write higher quality of the code,
and all the benefits are coming.
DREW: And I like you talked about how
it links to the career path, the field,
and how it's actually being curated and observed
by employers.
One of the things that was brought up
towards the end of that workshop meeting
by some of the people who work in PhD admissions
was that they were seeing students
who were putting on their cover letters, their resumes,
that they knew how to use it.
But they were actually finding they didn't totally
understand it.
So it was like they were familiar with the toolbox.
But when they actually had to open up that toolbox
and use it, they were picking the wrong tools.
So let's talk a little bit about that.
Where is the disconnect for a lot of students right now,
or where are you seeing it the most?
And how do we kind of adjust that
so that they are coming in with the right understanding
of the tools?
KONSTANTIN BAUMAN: So it's a very good question.
[CHUCKLES] So it's really hard to know
the exact answer for any topic.
DREW: Mhm.
KONSTANTIN BAUMAN: But what we see right now
is that large language models produce some output.
It's reasonable.
It's meaningful, so it can serve as a great support for a person
to solve certain tasks, especially simple tasks
like writing a simple code which will take a list
and reverse it.
That ChatGPT will do easily.
Once you get to more complex tasks,
ChatGPT can produce some output, but that
would be a good first draft.
So you should consider the output of ChatGPT as a support,
as a first draft of what we are doing, not the final answer.
So in order to be able to fix the output of ChatGPT
and produce the actual answer, you
need to understand what's going on.
So the existence of ChatGPT should not
replace the need for students to know things.
So they still need to learn the basic things,
like how to do the programming, how to write the loops.
So in order to be able to check what is the output
and what it's doing, yeah, it's hard to say
how much they need to understand it to be able to use it.
But we need to find out this balance.
JAEHWUEN JUNG: Yeah.
To add on to that, because I was on the--
I was waiting for admission offer last year.
So I understand where that criticism
comes from because as a candidate
I am pretty sure to demonstrate that I
know everything about Python.
I'm ready to do the statistical testing.
But then I think once I got in the PhD program,
I think what really PhD program is about deep engagement
of how you use that method to solve
a unique creative problems.
And then I think the disappointment
from the maybe faculty comes from that-- oh, you know,
you knew that you knew.
You told us that you know the Python,
but you haven't really engaged with that technology
to solve the problem.
DREW: Yeah.
JAEHWUEN JUNG: So I think that's where
the all the criticism and disappointment came from, just
as my thought.
DREW: Yeah.
Thank you for your perspective on it.
Obviously, as you just went through that,
that's pretty invaluable.
And Todd, I see you over there kind of laughing a little bit.
So you're the one actually teaching the course,
and you're around a lot of people
also teaching these courses as well.
Obviously, you talked about how you teach your course.
But are other professors, whether at this university,
or at other universities, or just the topic
itself in how you teach AI and machine learning--
is it matching what we actually need
to be doing for the students and actually drilling
down and using it appropriately and not
replacing the skills entirely, or is it just something that's
lagging a little bit behind?
TODD SCHEIFLING: Yeah.
I think the answers to these questions,
like Konstantin was saying, depends so much
on the specific context.
So it's very different perhaps teaching undergraduates and PhD
students.
And the big challenges in PhD students
are aligning all of the different pieces
of the project.
It's very easy to get so excited with the tools,
and we can go so far with the tools.
And now with ChatGPT, really we can really
push progress and get tremendous amounts of data constructed
very quickly, for example.
But ultimately to create successful and meaningful
impactful research you have to have
this really strong alignment between we've
got a great question, and we've got great methods
to answer that question.
And so where students kind of fall off
or some of the pitfalls or traps they can run into
is focusing just on the tools, on the methods.
We're putting together large data sets
and constructing new variables.
But those data don't actually answer the questions
they're trying to answer, or the question
isn't actually interesting.
So one example that I heard about from another university
was this really sophisticated analysis of real estate
listings and doing image recognition
and processing the images in these in these ads.
And ultimately, the finding was that if you put pictures
of plants in your real estate listing,
you'll be able to have some lift in interest from buyers.
But within that it's kind of interesting
in an everyday sense.
And certainly if you're thinking about listing a property,
it's interesting to you.
But from an academic perspective from within the discipline
of knowledge that's being created,
generalizable knowledge, not about specific practical topics
like listing your home but a wider, more generalizable body
of knowledge, that candidate wasn't successful on the job
market because they weren't contributing
to that larger, generalizable pursuit of knowledge.
So it's about creating the fit between the methods
and the research question that is really important at that PhD
level anyways.
DREW: Great.
Thank you, and I think you kind of got-- and you've
given a few really good examples of how people are using it
in fields that I think are a little bit less than obvious.
I think when we talk about AI and machine learning,
we often kind of think and exclude it
to coding or like data finance.
Right?
And I think, as you've pointed out,
there's some really cool ways that people are
using AI and machine learning.
Can we talk and share examples of ways
that people have used it answering questions that maybe
are typical for that field but doing it
in a way that utilizes new methods that
are just outside of the norm that people maybe
don't know about?
KONSTANTIN BAUMAN: There are so many different applications
of ChatGPT and other large language models now.
The basic thing is to get the knowledge.
You can take, let's say, your internal documents
in your company, feed it to large language models,
and then any employee instead of searching through the data
can chat with the large language models and get the answers.
So--
DREW: Todd, you presented a list of different students
who have used it in a really wide range of fields.
I mean, to me, the one that always stood out
were students in STHM.
That just doesn't seem like a field that's naturally
as applicable to AI and machine learning,
yet there were dozens of articles
that you presented that your students had written.
I think that's fascinating.
Do you have any examples you want to share?
TODD SCHEIFLING: Yeah.
Again, what's really exciting about working in this area
and teaching this class is getting
to hear about all the different interesting projects
and directions that students are taking things.
Basically, the larger background to the class
is so much of business activity is
being digitized in digital texts, or videos, or images.
And all of that is now data as oil.
All of that is now possible to analyze and extract insights
from.
And so if you're talking about STHM,
some of the things that they're interested in
are the fans around sports, and how they understand
what is happening in those competitions,
and how they organize to support teams
or engage in competitions and leagues of their own.
But another big area that they're interested in
is the rise of sports celebrities
and sports branding, both with professional athletes
and now with college athletes having
increasing access to those same markets
to be able to market their own brand.
So another area that they're super interested in
is understanding how athletes build their brand
and how fans respond to that effort those efforts.
And as well, of course, it's tourism and hospitality
management.
So there's a vast-- you can think about all
of the websites that are dedicated to building
up, evaluating, and rating tourism
destinations, hospitality businesses,
and things like that.
So there's a lot of research there as well.
I think some of the things that people are pushing towards now
is emphasizing video data and multimodal data.
So for example, people in finance and accounting
are really interested in these conference calls, where
managers meet with analysts to discuss the performance
of their company.
These are publicly-traded companies in the last quarter,
and previously people focused on text mining of the transcripts
of those conference calls.
But now they can also bring in the multimodal data
about the facial expressions and the audio tones associated
in those data and try to tease out signals about, for example,
how sincere are the managers.
Is there something they're hiding from analysts?
And can we try to explain that behavior and the consequences
of that behavior?
So basically, in every field, in every subject
of interest not just to management researchers
but within management and the business school-- every field,
there's tremendous amount of new opportunities
for developing these methods.
DREW: That sounds like quite a range.
Jaehwuen, you're obviously on the ground
floor with all the other PhD students researching.
Is anybody, including yourself-- any research that's
standing out to you as really new and kind of doing things
in a different way?
JAEHWUEN JUNG: Yeah.
I wanted to mention some [INAUDIBLE]
how e-commerce utilized generative AI.
So for example, Amazon already adopted the generative AI
to summarize the reviews because there are so
vast amount of reviews.
But also Shopify in the back end helps
sellers to generate product photos and the product
description with GenAI to increase the sales.
Now, also there are some interesting research
on also using chat bot but have the salesman personality, very
outgoing, sociable.
So if that impact your sales as well-- so
I think also how AI can assist the medical diagnosis
and the B2B negotiation and stuff.
So there are a lot of research, and real-world examples
are going on.
DREW: Yeah.
You guys presented a bunch of different examples
of how people are using in completely different but also
similar ways.
And I think one of the things I found
interesting you mentioned, e-commerce,
how that's going to be completely structured
around data mining, AI, and machine learning.
So that seems really interesting.
As we kind of wrap up, I was going
to ask where do you guys see AI and machine learning
going in education.
But I think you've already given a pretty good perspective
on that.
So let's talk a little bit more just for the individual
because now, obviously, as you've all pointed out,
you can use it in any field.
You just need to find the right question
and the right methodology for it.
So for those that might be interested in whatever field
they're in, where might be a good place
to get started learning about AI and machine
learning besides, obviously, Konstantin's
wonderful workshop that he runs every single month?
Where else can people look to get
a little bit more information on AI and machine learning.
TODD SCHEIFLING: Well, within Fox or Temple,
they should definitely attend the workshop.
And my class is offered every fall semester
to PhD students from across Temple University.
And, again, the great thing about this community
is there's so much collaboration and sharing of resources.
So there's so many totally free materials
available online, and tutorials, and so many different on-ramps.
I think what I find in my class is
that with education the big challenge is motivation.
How do you motivate--
before you can learn anything, you
have to be motivated to engage.
DREW: Mhm.
TODD SCHEIFLING: And so there is a value
to a workshop or a class structure
to help students build that motivation to continue
to engage and build with it.
And then part of that, the most important thing,
is to find questions that you're interested in that you think
these methods can help you with so that you're not just
learning the methods on their own.
But you're learning them towards this end
of answering a question.
And that's super important too, I think, to stick with it
and to make progress.
And the more you do, the more your horizons open,
the more new questions you can envision,
and the more methods you become familiar with.
And really if you take that approach
of let me find challenges that help me
that are aligned with building these skills,
that can really expand from there and grow over time.
DREW: Great.
KONSTANTIN BAUMAN: I was going to say almost the same thing.
Like, if you want to learn the technology,
you need to start working with the technology.
And I taught that to my students in the analytics class
even before ChatGPT was introduced.
Just start playing with that.
And the following thoughts comment
like it's very important to find some sort
of a project, something which would really
motivate you to do the work.
As an example, I had a student in the previous semester,
undergraduate student.
He was very interested in one game.
Unfortunately, I don't remember the name of the game,
and the story behind it with lots
of characters, and the interactions, and so on.
And he developed the--
he fine-tuned the ChatGPT model to be able to accurately answer
the questions about the story behind each of the characters.
And he is a big fan of the game.
And he was really into that and really worked in the project
and developed a tool which was helpful for all the fan base.
DREW: I got to do that for all my favorite video
games now too.
So--
KONSTANTIN BAUMAN: OK.
We can discuss it in the next podcast.
[CHUCKLES]
DREW: How about you, Jaehwuen?
JAEHWUEN JUNG: Oh, well, I think two professors already
mentioned about good resources.
So I would just suggest interact with ChatGPT a lot
and also with Gemini because it's really fun just
to generate this photo for me.
So use it as an assistant and have fun with it.
And I think that could be a good starting point.
DREW: Yeah.
Well, thank you guys all for everything you've said.
Just a recap real quick.
It just sounds like regardless of what field
you're in, if you have a good question,
there's a methodology out there using
AI and ML to get the answers that you're looking for.
And then it really seems like the best way
to get involved is just to get involved,
start working with it, and, as you said,
maybe even work with a couple different chat bot
prompts and everything like that to really get involved.
[MUSIC PLAYING]
I want to thank you all for being on this episode of The T
in Teaching and discussing AI and machine learning.
JAEHWUEN JUNG: Yeah.
KONSTANTIN BAUMAN: Thank you.
[MUSIC PLAYING]
