(applause)
-: Well, good morning, everybody.
Pleased to be here with you,
and we're gonna take a few minutes and talk a bit
about assessment issues, evaluation of performance,
from the instructor's perspective,
whether you are assisting with the class,
taking full responsibility for a class,
just being handed a sheaf of papers by the professor
and told, "Help grade these puppies."
Or, thinking about it even from the students' side.
So, we're gonna talk a little bit about
the whole issues and what are entailed.
So, let's start with a quiz.
This will be simple,
'cause we've just been studying for it, right?
So, here we go, Question Number One.
What is the answer?
(audience murmuring)
Come on, now, what's the answer?
(audience shouting at once)
All right, as I read the question,
I would put down 140.
But the instructor that made this up
would have marked that wrong.
What do you suppose the instructor was looking for?
How many bananas, 560.
And what do you suppose
the instructor's defense would have been?
(audience murmuring)
You know what I meant.
(laughs)
Or maybe you didn't.
Let's try this one.
Pretty straight-forward, right?
(audience murmuring)
So what's the answer?
(laughs)
I don't know!
I don't what Jasmine had to spend.
I could tell you what would be the total cost, without
without tax, I could figure that out.
Maybe Jasmine doesn't have a penny to her name,
and she's borrowed a friend's debit card.
(audience laughter)
Let's try a few fill-in questions.
Those are easy-breezy.
How about this one?
(audience murmuring)
All right, someone give me an answer.
(speaking off microphone)
Reflect, good question, good try, but no.
Voiceover: Reverse?
-: Reverse, good try, but no.
Voiceover: Distort.
-: Distort, good try, but no.
Well, of course, the answer as we discussed in class,
was invert.
Right?
Voiceover: Isn't that (mumbles off mic)?
-: Isn't that reverse, no actually, reverse (laughs).
Okay, let's try this one.
(audience laughter)
So what's the answer?
Voiceover: Anything.
-: Anything, good try, but no, no credit for you.
There we go, we need context clues, don't we?
Well, the answer, of course, is astronomy.
What else could it have been, right?
Let's try this one.
(audience laughter)
Voiceover: Capitalist?
-: Capitalist, good try, but no, no credit, sorry.
Voiceover: Mixed?
Professor: Mixed, no, sorry.
Voiceover: Recessing. Professor: Recessing.
(laughter)
Professor: Resuscitating, no.
Voiceover: Great?
-: Great, okay, good try, no.
Voiceover: Market?
Voiceover: Unstable.
Professor: Unstable, good try, but no.
Of course, if you had been paying attention in class,
you would have known this, right?
Market-based, right?
Market-based, now, you're thinking to yourself,
"Boy, these are silly questions."
Every one of these examples
came from actual class tests.
Let's try this one.
(audience murmuring)
Oh, come come, now, don't be shy.
Voiceover: From Mexico.
-: From Mexico, I like that answer.
That wasn't the one that the instructor was looking for.
(audience laughter)
Yes, sir.
(speaking off microphone)
I like that, too, but that's not what
the instructor was looking for.
Mexican, I.
(audience laughter)
(applause)
It is difficult to argue with that logic.
Let's try another, shall we?
(audience laughter)
(audience murmuring)
All right, what would you say about the career of Hitler?
This is from a history class.
I'll give you that clue.
Failed artist.
Voiceover: Truth.
Voiceover: Also, (mumbles off mic) opinion.
-: I like that.
(audience murmuring)
If you are faced with a task like this,
put yourself in the shoes of someone in that class.
What do you do when you get a task like this?
Voiceover: Try your best.
-: You start writing as much as you can think about
related to Hitler and hope that it sticks, right?
(audience laughter)
All right, let's try a simple matching exercise.
You've got three premises and three alternatives.
Let's see if we can sort them out.
(audience murmuring)
Now, maybe you didn't prepare for this one.
But let's see if we can get it, nonetheless.
Au, what's the answer?
Audience: C.
Professor: Great, and you didn't even study.
Eight?
Audience: A.
Professor: Very good, and finally, Dr. Sabin?
Audience: B.
-: How did you do so well on that?
(audience murmuring)
Okay, so the big question is,
what went wrong with every one of these items?
Voiceover: They were very easy.
-: The fact of the matter is, nobody vetted them
to assure that they were competent,
that they were posing the task intended,
that they weren't ridiculous, like all Mexicans are blank.
Okay.
None of you,
probably few of you
have had any kind of background in developing tests,
any formal background in developing tests,
and yet, you see the silliness before you.
Okay, it doesn't take a lot of effort
to see when things go wrong.
The trick is to be able to foresee that,
and avoid it happening in the first place.
Okay, so if we're gonna talk about all the issues
that are instructor concerns, these would be them,
in my opinion, okay.
Figuring out what to assess in a class,
because you can't cover everything
through assignments or tests, you have to be selective.
You have to pick and choose what you think
are the key, the important, the vital,
the skills that they have to have to succeed
in the next class, or later in the course, whatever,
those are judgments that have to be made.
How to assess it.
When my daughter, when my older daughter
was taking driver's ed,
they had one car that they would take out
and the rest of the kids would be in study hall.
And so, three kids and the instructor
would be out at any one time,
and she was out on one of these excursions,
and you probably remember how that goes.
They swap driving responsibilities
during the cruise and then come back
and then get another batch, and so forth.
Well, the instructor, a coach, thought this would be
a teachable moment for how to change a spare tire
on this one day on this one loop that they were making.
So, he tells the driver, "Young man,
"pull the car over, and let's see you change the tire."
And the student is game for anything.
So he says, "Yes, sir."
Pulls the car over,
finds the jack, equipment, successfully,
and fortunately, this is a car
that actually has a spare tire in it.
Not all of them do, these days.
Puts the jack in place, cranks it up,
and is ready to proceed with other parts of the process,
when one of the other students says, "Coach Blank,
"I see something dripping."
Sure enough, the kid had put the jack on the gas line,
broken the gas line, and gasoline was coming out.
What went wrong with that, what could be more authentic
than telling someone to change the tire,
to see if they know how to change a tire.
That is as high fidelity a measure
as you could possibly have, but what went wrong?
(audience murmuring)
Voiceover: Didn't teach 'em how, first.
-: Well, maybe they should have taught 'em how first.
Yeah, I like that.
But the fact of the matter is,
the instructor wasn't paying attention,
step by step, to what was going on
and correcting it if it was wrong.
Just letting the kid go with the flow.
Okay, and you think, "Oh gee, nobody does that."
Oh, yeah.
Okay, test construction, performance assessment,
in other words, what kinds of work products or samples
would be helpful for making decisions
about student proficiency, grading and marking.
How many of you believe in the grading on the curve?
(audience murmuring)
Let me change the question.
How many of you have ever been in a class
where the instructor has graded on the curve?
Holy cats, look at that.
What's going on when that happens?
Voiceover: Most people are doing bad.
-: Say again?
Voiceover: Most people are doing bad.
-: Oh, that's a very good observation.
A lot of instructors say to themselves,
"Well, I'm not gonna do that."
Until they get the first test results back,
and they find that, as a group,
the class has kind of crashed and burned.
So they say, "Okay, 70 is the new 100."
(laughing)
And on it goes.
The problem with that system?
Voiceover: It's not actually an assessment
of the capabilities (mumbles off mic).
-: Very good.
Depending on whether you, as an individual,
happen in a very able group,
your standards of performance are gonna be much higher
if you're lucky enough to be in a bunch of beanheads,
then your standards are gonna be much lower,
and the problem is the standards are gonna fluctuate
from class to class.
So obviously, when you go to bed at night,
you pray for dumber classmates, right?
(audience laughter)
All right, feedback to students about performance.
What do you think the literature says,
the research says, about feedback?
Good thing or a bad thing?
Audience: Good thing.
-: Very good, it is a good thing.
How quickly should students get feedback
about their performance?
(audience speaking at once)
How soon is, as soon as possible?
Voiceover: When they get their grades back.
-: How about letting them know
two minutes after they've turned in the test
or completed the test, if it's online?
Voiceover: It's not possible to get online
because the computer graded it right after it.
-: There's a good question.
The fact of the matter is, instant or near-instant
feedback turns out not to be nearly as effective
as feedback that's delayed for say, 24 hours.
Can you guess why?
Voiceover: We haven't had time to actually look at
the same paper in depth.
-: That's a good point, but the other side
of that equation is the student hasn't had time
to process what they've been through.
If you simply tell 'em, "Boy, your score is blah,"
they're gonna stop the processing right then and there.
And they'll move on to the next thing.
So, in point of fact, you know, prompt feedback
is a good thing, and by prompt, I mean in the next
day or two, but as opposed to the next semester.
(audience laughter)
But instant feedback, not so much.
Cheating, appeals, and other issues, those are gonna happen.
So if you have responsibilities associated with the course,
the recommendation is, hold on to student papers
for at least a semester.
There are gonna be folks that recommend other things.
The great appeal process at MSU has a timeline
associated with it, so you don't have to hold things
seven years later, and so forth,
but do hold on to some things.
Okay, so what's important to assess?
There's some, there's a, I guess, a fair trade off
between everything in the book, which is probably overkill,
and then little or nothing, under the assumption
that they'll remember little or nothing afterwards.
But to me, what are they gonna use later in the class,
what are they gonna use later in subsequent classes?
All right, let me reminisce a bit.
My freshman orientation at college,
I was going to a school that was on the coast of Florida.
Every student had to take a swimming proficiency exam.
Didn't matter what your major was,
didn't matter whether you were able-bodied or disabled,
every student had to take a swimming proficiency exam.
Why would they take important time
out of the universe to do that?
(audience murmuring)
Voiceover: Location.
-: Sir.
Voiceover: Location.
-: Location, location, location,
just like the real estate agent.
And they knew what college kids are all about.
Okay, they know that some of them
might get close to the water
in less than full possession of their faculties,
and they wanted to know that,
you know, you could get to safety, if necessary.
In point of fact, if you didn't pass that exam,
you were signed up for swimming your first semester.
No ifs, no ands, no buts.
It was that important to them.
So, clearly, one of the things you wanna be thinking about,
or any instructor should be thinking about,
is what are the important take-aways for this course?
Because if you're gonna be spending time on it,
in class, if you're gonna be spending time on it,
looking at student work,
if you're gonna be spending time on it
assessing student skills, then you know,
you're giving up something else, so prioritize.
Okay, objectives are a big way, a simple way,
to help students figure out what's important to you.
Just like old tests are a good way to let students know
what are important to you.
Or study guides are a good way to let students know
what's important to you.
In fact, the day before a test,
or the class meeting before the test,
what's the most common question, do you suppose?
Voiceover: Tell me the questions--
-: Is this gonna be on the test?
Why do you suppose students ask that?
(audience speaking off microphone)
They want to know what to study, for goodness sake, yes.
Of course it makes sense from that point of view.
Okay, so should it be kept a deep, dark secret?
Oh, it's gonna come from the chapters.
Or not (laughs).
(audience laughter)
Voiceover: That's pretty good.
-: Nothing puts students at ease more
than that kind of coaching, okay.
So, think, just think about, how do I communicate to them
what the heck I think is important for them?
Important enough to guide their study efforts,
important enough to assess them on,
and important enough to make decisions about them.
Okay, so there's all kinds of ways to make that happen.
How to assess, we talked a little bit
in the tire changing about task authenticity.
The more authentic the task,
the less people can bitch and moan, and say,
"Oh, that wasn't realistic, that wasn't real world."
Okay, you can make tasks as real world as you like,
but what you give up when you do that?
There's several things that, unfortunately,
you have to give up when you do that.
Every other student in that car, on that day,
learned something from that mistake.
So, if the coach was to take the other students
and say, "Okay, now you change the tire."
Are they starting from the same baseline
that the first student did?
No, so in that sense, it's not fair to everybody
to do it that way.
Just like, if I go around the class,
and ask everyone, "What's the capital of Mississippi?"
-: Jackson.
-: Jackson, beautiful, what's the capital of Mississippi?
-: Jackson.
-: Perfect, isn't that great, everybody has learned
the capital, okay, you know what's wrong with that system.
Okay, so figuring out how to sort it out
in a way that makes it feasible,
and something you can handle.
Yeah, you have to work on it.
Okay, at the bottom, there's a lot of information
about, well, let me rewind that tape.
Have you ever been in a class
where the instructor asked,
or gave you the task,
"Tell me what grade you think earned, and why?"
Have you ever gotten one of those?
So, what do you say?
Voiceover: A.
(laughing)
Shoot high!
I learned so much in this course,
I never knew how important this material was.
Oh, my gracious, sakes alive, I am virtually
self-fulfilled, thanks to having completed this class.
The fact of the matter is, college-age students,
and I'm not saying anything that isn't supported
by the literature here, more or less,
don't have a lot of self-appraisal skills
that are highly accurate, that,
the fancy word we use for that is, calibrated.
They're not well-calibrated to figuring out
what they know and what they don't know.
In point of fact, the people who know the least
tend to overestimate what they know,
and the people who know the most,
tend to underestimate what they know.
So you're getting penalized at both,
you know, biased at both ends of the continuum.
So that's probably not a good strategy.
All right, open book test.
What to you think, a good thing or a bad thing?
(audience speaking at once)
Oh, who said, "I hate 'em?"
How come?
Voiceover: The questions are usually more detailed,
and they take longer, and I'm gonna study regardless,
and I don't like, all these people that didn't study
get to use the book and feels like they get to cheat.
Professor: Oh, okay, all right,
so there's an equity issue associated with that.
Any other opinions, yes ma'am?
Voiceover: I thought my score when I took
an open book test, students tend to perform less on,
I don't think they prepared, and plus,
they spend too much time looking for (mumbles off mic).
-: (chuckles) The last part is absolutely true.
If we could broaden that scope to just saying,
"Are you gonna allow students access to resources?"
A lot of times, the answer is, no.
And for some instructors, the answer is, "Well, yes,
"because in real life you would have access to resources."
When I give tests in my class, they are open book,
open note, open anything, except another human being.
Okay, and to me, that includes the interweb.
So, they can't do that.
Why, because to my way of thinking,
that's they're gonna do it in real life,
the other thing, however, is before a test,
I coach 'em, or I try my best to coach 'em
to how to organize their materials and their notes
so they won't be fumbling through 500 pages,
looking for something, and if they fail to do that,
well, it's on them, isn't it.
Okay, and yes, I give questions that don't lend themselves
to looking up the sentence in the book that says,
"The career of Hitler was," (laughs) "mixed."
(audience laughter)
All right, take home test.
All the people who have studied this
are convinced based on evidence that students
don't learn as much of the material,
nor do they retain it as well as in class tests,
or tests that you take as part of the class.
Even if it's online, maybe under a timeframe, same effect,
not, take home is not nearly as effective
as a learning tool.
Why do you suppose that is?
Voiceover: Distractions.
-: Distractions, sure, what else?
Voiceover: They take it and just (mumbles off mic)
and they don't actually learn it, they just.
(mumbles off mic)
-: Exactly.
Voiceover: They give it to a friend to take.
-: (laughs) Even more plausible.
The fact of the matter is, yes,
what do they focus on, they focus on the specific tasks
in the take home test, they work just as long
as they think it takes to address those tasks,
and no others, and that's it, close the book, I'm done.
Okay, so it's not surprising that the breadth
of what they learn isn't as good,
and it's not surprising that they don't take it away
and retain it as well.
So, something to think about.
Oh, collaborative test.
Letting students take tests together.
What do you think the outcome of that is?
(audience speaking of microphone)
All right, Question Number One,
do you think students like it?
Audience: Yes.
(audience talking at once)
-: Very good, it actually turns out to be a mixed bag.
On average, yes, students will say,
"I love this format, thank you, instructor."
However, some students are going to be
unhappy with that format, why?
(audience talking at once)
Exactly, it's the same thing that happens oftentimes
with group work, some people ending up doing
the heavy lifting, and the others are just
along for the ride.
And some folks think that's an unfair system,
and in fact, it is.
What do you think happens in terms of the overall
amount of learning or retention?
(audience murmuring)
For the class as a whole, not as good, not as good.
Okay.
So if you're looking for good ratings,
at the end of the semester, give 'em.
If you're looking for better retention,
and having learned more, don't use collaborative tests.
Okay, frequency of testing, more often or less often?
Voiceover: More.
-: More, absolutely.
All the surveys of students say they prefer that,
and all the outcomes say
that's actually more effective in the long run.
Why do you suppose that is?
Voiceover: Retaining the information,
if you take it over a longer period of time,
then you're gonna have a harder time
remembering what happened two months ago, three months ago.
Professor: Exactly, you focus their attention on
more digestible chunks of material, that's one.
What do you think is another reason why?
(mumbling off microphone)
Voiceover: We didn't have a chance to catch back up--
-: Exactly, more opportunities to overcome a bad start.
What was the comment up here?
Voiceover: (mumbling) Across the course,
that way, you don't have to worry about just one test.
-: Absolutely, absolutely.
It's not eggs in one basket time,
like in the case of just a mid-term and a final,
and, "You're on your own, my friend."
So the students prefer it, it keeps them,
you have a better chance of keeping them
abreast with the material with frequent testing
rather than with sparse testing.
So think about that.
Okay.
See the arrow on the slide?
This is it, I've digested books and books into three rules.
What's a good test?
Number One, it measures the intended behavior,
and not something else.
Not something else.
Two, it clearly communicates, like bananas in bunches,
the task to the student.
And Three, it yields a reliable indication of performance.
That is, you can use that score as an indicator
and be confident that that's gonna tell you something
about what the students have learned.
Okay, those are it, the three rules for a good test.
Okay.
Now, how you get there, oh my goodness.
There's a lot of stuff that goes behind that.
Okay, these are what professionals do.
Okay, and when professionals write test items,
if 40% of the items they develop
make it through all of these steps,
they think they've done remarkably well, 40%.
So for the rest of us, we have to,
we have to be willing to
know that some of the things we've written
are not pearls.
Like, "There have been great advances in the field of."
Okay.
All right, different kinds of test formats.
Doesn't matter whether you're talking about paper, pencil
or what have you, but two basic families.
One, the answer is there, staring you in the face,
and all you have to do is find it.
That's called Selected response.
Those are true-false, multiple choice,
matching, and the like.
The other family, you have to generate the response.
It isn't there, staring you in the face.
Those would include things like essays,
projects, short answer, fill-ins, and so forth.
Okay, they all have their virtues,
and they all have their drawbacks, in point of fact,
you can make just about any format work,
in just about any situation, but it turns out
that some formats are a little better suited
for measuring certain kinds of outcomes.
Okay, so in a nutshell,
what are some of the general rules
that we should be thinking about
in terms of making up of tests?
Number One, top of the list,
have someone else review your work
before you unleash it on humanity, okay?
(audience laughter)
That'll help with things like, "All Mexicans are blank."
Okay.
Be prepared to revise items.
They're not all gonna be pearls.
Include clear directions.
It doesn't matter whether you tell 'em, give 'em examples,
post a two-minute video on how your tests
are gonna be conducted, or write it all down.
But communicate what is it you want 'em to do
and how is it you want 'em to do it.
Stay away from straight quotations,
unless that's what you're trying to measure.
Their ability to remember quotations.
Stay away from trick questions.
I'm gonna show you one or two in a minute.
And you'll see why you should stay away from 'em.
The measurement itself should reflect the time
and importance of the skills you're trying to assess.
Okay, just 'cause it's easier to ask questions
about Jackson being the capital of Mississippi,
doesn't necessarily mean that's the way the test should go.
If what you want the students to understand is,
how is Mississippi different from other states?
Okay, knowing the capital isn't going to tell you
a heck of a lot in that regard.
And don't do no double negatives, okay.
All right, let's take a quiz.
You've never studied this before.
I've given you the answer with the asterisk,
so here's what I want you to do.
Tell me why that's the correct answer,
even though we haven't studied this.
(audience chuckling and murmuring)
Oops, let's go back.
Exactly, there's a clue there.
If you don't know jack about the question,
you start a whole new set of strategies
and one of those strategies is,
is there anything in the stem of the item
that matches anything in the alternatives?
And there is, the first one has the word, cluss.
Turns out that's the right answer, oh!
How about the second one?
(speaking off microphone)
Oh, boy, you jumped right on that.
Notice that the stem ends with the article, an.
An, in English, that tells you that,
if people know what they're doing,
the next word will start with a vowel, not a consonant.
And guess what, only one of those options
starts with a vowel, hence the answer.
Okay, and you say to yourself,
"Oh gee, you're just making up tricky examples.
"No one would ever figure that out."
Yeah, fourth graders can do this.
So, I'm confident even college students can.
Okay, all right, let's try a couple of others.
How about the first one?
Mitushito was,
we have to go back to World War II time for this.
Why do you suppose
this item needs help?
Voiceover: There's only one.
There's only one people to answer.
Professor: Okay, if you look at the options,
unfortunately, only A matches with
what we believe Mitushito sounds like,
it sounds like a Japanese surname, does it not?
It doesn't sound like a Russian pilot,
it doesn't, or a Russian journalist
or an English pilot, or a U.S. delegate,
I mean, it, you know, maybe it could be.
But in this case, it wasn't.
So you, what happens is, the item is artificially easy
because of that clue.
How about the second one?
Voiceover: The word twice corresponds with poly.
-: Very good, very good.
If you look at the second, the first sentence,
"May result in twice the number of chromosomes,"
and if you look at the answers, "Poly," many.
There's a cue, okay.
So, again, inadvertently, the answer's been given away.
All right, let's try some debatable questions
and tell me what you think about those.
First one, who's the most important governor
of Mississippi?
Now what could possibly be wrong with that question?
(speaking off microphone)
It's an opinion.
Now, if what you're measuring,
if what you're intending to measure
is can they take a stand and defend that stand
with factual information, okay, great.
But that's not what this says.
It just says, who was the most important governor
of Mississippi?
So, someone says, eh, Allain.
It doesn't say, according to who,
or based on what, okay.
So that fails there.
How about the third one, who's the better poet?
Keats or Wordsworth?
(audience murmuring)
(laughing)
It depends on who the instructor likes, right?
No, again, what that does is, as it stands,
is just an opinion question,
and if someone just puts down,
"I think the better poet was Keats," end of discussion.
Or, enough said.
Okay, but that's, the instructor's probably looking
for something better than that,
but that's not what they're asking for.
All right, how about the first one?
"List the results of World War II.
(audience laughter)
"Use both sides of the paper if necessary."
So when you see that comment, what do you know?
You better write both sides of the paper.
(audience laughter)
Okay.
They're giving you a cue there,
you better pay attention to it.
So, what are the results of World War II?
Yeah, a lot of things, holy cats, where do you start,
and where do you stop?
So it becomes a matter of trying to guess,
you know, develop your psychic ability
to divine what the instructor's actually after.
And hope that you have sprayed enough of that
onto the, both sides of the paper
to get credit for it.
Okay, all right, let's look at a trick question or two.
First one, John L. Tukey developed a display
known as the box and whisker plot.
True or false?
There's one vote, true.
I will tell you that Professor Tukey,
first of Bell Labs and later in Princeton University,
contributed a great many things
to the world of data analysis,
including the box and whisker plot.
But the answer to this question is, false.
Does anyone know why?
(speaking off microphone)
Because his middle initial was not L.
(audience laughter)
Now, imagine the love in the classroom
when you hand that back to the students.
Well, of course it's false, his middle initial wasn't L.
Weren't you paying attention?
(speaking off microphone)
(laughs)
All right, let's...
Okay, let's take the second one.
About President Lincoln, sir?
Voiceover: False.
-: Okay, one vote false, any others?
Voiceover: False.
-: False.
Voiceover: False.
-: False.
Voiceover: False.
-: False.
As it turns out, the answer might be, "It depends."
Voiceover: States in rebellion.
-: States in rebellion, very good.
Okay, first, there's two bits of information here
that the student is gonna be forced to judge.
Number One, was he in fact the 16th president?
Yes, so far, so good.
Second, did he abolish slavery in the states?
And the answer is, some of them.
Some of them, but not all of them.
So, is that true, or is that false?
Could the instructor have just been a little careless
in writing that question?
Well, yeah.
So, which one gets you credit?
Okay.
That's a problem, and that makes it artificially difficult.
Okay, let's try this one.
Notice I haven't given you the answer.
(audience murmuring)
What's the answer?
Audience: E.
Professor: E, very good, none of the above.
All right, let's say that
2/3 of the students in a class pick E.
Do they know the capital of South Dakota?
(audience murmuring)
Maybe, maybe not.
Voiceover: They know what it's not.
-: They know it's not these four.
But they may or may not know it's Pierre.
Okay, that's a problem with this kind of item.
None of the above, or no 'taz, as I call 'em.
Okay, all right, evaluating answers.
You're gonna need to stop and think about
before you get started, what is it I'm looking for,
what am I gonna give credit for,
do I give partial credit,
and then try to stick to that set of guidelines
the whole way through.
I'll tell you what tends to happen.
In a big stack, your standards as you start at the top
of the stack are gonna change by the time
you get to the bottom of the stack.
And you're gonna, and unfortunately,
that's not fair to the students, okay.
Think about that from the students' perspective.
So, you really need to, ahead of time,
have what you think is a model answer.
What are the features you're looking for,
what are you gonna give credit for and...
Take breaks, rub your eyes, stretch,
get up, sit back down, okay.
All right, so here's some examples,
and let's talk about whether they deserve credit or not.
Here's one from a physics class.
Okay, how do you measure the height of a tall building
using a barometer?
(audience murmuring)
(audience laughing)
Okay, here's what the professor's looking for.
Measure the air pressure at the base of the building,
measure the air pressure at the top of the building.
There's a simple conversion formula
to figure the altitude from that.
That's what the professor's looking for.
Now, does he or she give credit for answers like these?
(audience laughter)
It would work, or how about this one?
(audience laughter)
It would work.
Okay, or how about this one?
(audience laughter)
It would work, too.
I love this one.
Use proportions, measure the shadows, use proportions.
It works, unless there's a cloud over the sky.
Okay, and last but not least.
(audience laughter)
I will give you this fine barometer.
(audience laughter)
Okay, now do you give credit for those,
because any of those could work, okay?
But that's the difference between posing the task
in a way that communicates what you're after,
versus just throwing it out there,
and letting the students' imagination take flight.
Okay, let's try a couple of others.
(audience laughter)
Where was the Declaration signed, at the bot,
well, where else would you sign it, of course!
Okay.
(audience laughter)
Hard water, ice, it doesn't get much harder than that.
Let's expand.
(audience laughter)
Let's expand the function, and sure enough, he did.
Okay, or.
(audience laughter)
You know, does it continue to go after it comes to rest?
And the answer is no, 'cause he drew an elephant,
and he says there's an elephant in the way.
(audience laughter)
Find x.
Well, give him credit for map skills, there it is.
(audience laughter)
Okay, with this one, what would fix that?
Voiceover: Value of x.
-: What is the value of x?
Okay, or solve for the value of x, not find x.
For goodness sake.
Okay, all right, or here's some examples,
Darwin was the author of organ of the species.
Not origin, do you give credit for that?
(audience murmuring)
The dodo is a bird that's almost decent by now.
The next to the last is my personal favorite.
If you smell an odorless gas.
(audience laughter)
It's probably carbon monoxide.
I mean, they've got some of the stuff there, right?
It's odorless, sure enough.
But you're never gonna, anyway.
All right, there's other kinds of ways of evaluating
besides tests, besides questions in class,
besides take home projects.
And you should give thought to those as options.
You should also think about your grading philosophy.
If you're gonna be giving marks that are gonna
walk through the students' transcripts forever and a day,
it's a big deal, so what goes into those?
What doesn't go into those, okay?
Think about those.
What about borderline cases?
You know, remember what we said about grading on the curve.
If you ask experts what a grade should do,
they'll tell you, it should do one thing.
It should communicate achievement
or proficiency over the subject,
over the skills of interest.
Nothing else, it's not enthusiasm,
it's not motivation.
It's not big smile on your face.
Personally, as an instructor,
I love those attributes in students, okay, but anyway.
Think about that.
How do you combine scores into grades?
And so forth, cheating, it's gonna happen.
It's gonna happen at about the same rate
for online classes as for face to face classes.
So just be aware of it.
It's gonna happen, MSU has a policy,
all the recommendations of folks who study this
start with, "Let the students know
"what your expectations are."
That's what they wanna know.
What, to this instructor, is plagiarism?
If I borrow from a paper I wrote last semester,
is that plagiarism?
To a lot of instructors, the answer is, "Yes."
To other instructors, the answer is, "No, heck use it."
So, think about what's gonna be puzzling your students
at the beginning of the semester,
those are probably the things that should be addressed
at the beginning of the semester.
Okay, why do people rationalize cheating?
How can they look at the mirror when they do it?
Well, here are the top reasons that they give.
It's easy to do, I was too pressed,
a friend needed help, so of course I helped him or her.
I have to pass, okay.
Everybody else does it, that's the Niagara Falls argument.
No one else cares if I cheat, uh oh.
That's expectations.
Okay, somebody stole my work, well that can happen.
The course is too hard, or the teacher's just unfair.
Unfortunately, that can happen, too.
But the worst one, to me, is the last one.
Think about it, if we can't even sell our course
as being worthwhile, maybe we need to rethink it.
So, yeah, we should all be advocates for
what we're all about.
So what do you do to reduce cheating?
Well, if you stop and think about those rationalizations,
then some of the proactive steps become fairly obvious.
Okay, try to build good relationships.
From the get-go, make it clear what to you
is inappropriate behavior versus appropriate behavior.
Okay, students'll feel happier, knowing the difference,
you'll be happier, knowing the difference.
Okay, and so forth.
By the way, all of these slides, Ms. McMullan,
I think is gonna post them in a way
that if you wanna refer to them later,
you are certainly welcome to do so.
Appeals will happen.
You just got through, I think,
a presentation about a syllabus.
The simplest way to defend yourself
is follow the guidelines you put in your syllabus.
Don't change 'em up midway in the course.
Don't change 'em for one student because
it turned out to be, you know, a situation
you didn't anticipate, stick to the syllabus guidelines.
Okay, document your discussions, know MSU's policies
about appeals, about the honor code, and so forth,
that will really help.
Don't forget about these kinds of things in a syllabus.
What are you gonna do about late assignments,
what are you gonna do about test dates,
if there really is a legitimate excuse
for a student to have to miss test.
Some instructors handle that with drop grades.
Others will give make up opportunities.
Okay, some'll just say, "Sorry,
"I said be there or be square and you weren't."
But you need to stop and think about the impact
that would have.
Here are some examples, if you want to look 'em up online,
about a syllabus, and decide for yourself,
did they do a good job, or oh my goodness,
would I want to stay the heck away from that class?
Based on the syllabus?
You're certainly welcome to do so.
Here is an example.
I'll let you ponder that at your leisure.
If you're gonna count attendance,
make it a carrot, not a stick.
That's my suggestion.
By the way, MSU has, in the last three years,
really honed in on attendance as a big darn deal.
Especially among underclassmen, freshmen and sophomores.
Why do you think that is?
Voiceover: 'Cause they're new to the game.
-: Say it again?
Voiceover: They're new to the game.
-: They are absolutely new to the game, number one.
And number two, it turns out that attendance becomes
a good predictor of performance in the class.
It's an inverse relationship.
Higher attendance, lower grades.
(laughs)
Higher absences, lower grades.
Fewer absences, higher grades, okay.
It's an inverse relationship.
So think about that as important.
Oh, this was a one that really, really,
I found interesting.
An entire class, you'd fail the entire class
if you didn't show up for the final exam.
Didn't matter how you performed.
Just stop and think about that.
Okay, reporting.
Grades are protected by FERPA.
You're not supposed to divulge 'em
or disclose 'em to other people, you knew that.
Okay, and you wouldn't want your grades,
if you were taking a class, you wouldn't want
your grades posted in a way that others could see.
So, you know, don't do it for others.
The mycourses system is perfectly fine for that,
and it works very well.
There's lots of resources you can turn to for advice,
for examples, some good, some bad, and so forth.
The one thing I would urge you to do is,
if you're using like test banks for a textbook,
pay real close attention to those items
because lots of times, those are written by people
that had nothing to do with the class
or the text, and they were hired guns
and told, "Create a test bank."
And so, as a result, items may be in the test bank
that don't even pertain to the text.
And that's a tough one to explain away,
when the students challenge you.
Okay, and then finally, remember this prayer.
(audience murmuring)
"If I should die before I wake,
"that's one less test I'll have to take."
Anyway, some resources that you can look at,
if you're interested, resources on test construction,
you can look at if you're interested.
Some course evaluation comments that are kind of funny.
The text makes a satisfying thud
when I drop it on the floor.
Some cheating references.
And at the end of it all,
I hope you have a terrific semester, thanks.
(applause)