WEBVTT
1
00:00:07.126 --> 00:00:09.376
(applause)
2
00:00:10.504 --> 00:00:12.744
Well, good morning, everybody.
3
00:00:12.744 --> 00:00:14.254
Pleased to be here with you,
4
00:00:14.254 --> 00:00:18.048
and we're gonna take a few minutes and talk a bit
5
00:00:18.048 --> 00:00:22.215
about assessment issues, evaluation of performance,
6
00:00:23.472 --> 00:00:25.777
from the instructor's perspective,
7
00:00:25.777 --> 00:00:29.050
whether you are assisting with the class,
8
00:00:29.050 --> 00:00:31.610
taking full responsibility for a class,
9
00:00:31.610 --> 00:00:34.730
just being handed a sheaf of papers by the professor
10
00:00:34.730 --> 00:00:37.429
and told, "Help grade these puppies."
11
00:00:37.429 --> 00:00:41.058
Or, thinking about it even from the students' side.
12
00:00:41.058 --> 00:00:43.616
So, we're gonna talk a little bit about
13
00:00:43.616 --> 00:00:46.897
the whole issues and what are entailed.
14
00:00:46.897 --> 00:00:49.230
So, let's start with a quiz.
15
00:00:50.680 --> 00:00:52.298
This will be simple,
16
00:00:52.298 --> 00:00:54.800
'cause we've just been studying for it, right?
17
00:00:54.800 --> 00:00:57.800
So, here we go, Question Number One.
18
00:01:00.925 --> 00:01:02.508
What is the answer?
19
00:01:05.354 --> 00:01:07.198
(audience murmuring)
20
00:01:07.198 --> 00:01:09.797
Come on, now, what's the answer?
21
00:01:09.797 --> 00:01:14.086
(audience shouting at once)
22
00:01:14.086 --> 00:01:16.856
All right, as I read the question,
23
00:01:16.856 --> 00:01:18.606
I would put down 140.
24
00:01:20.267 --> 00:01:23.326
But the instructor that made this up
25
00:01:23.326 --> 00:01:25.743
would have marked that wrong.
26
00:01:27.246 --> 00:01:31.206
What do you suppose the instructor was looking for?
27
00:01:31.206 --> 00:01:33.039
How many bananas, 560.
28
00:01:36.547 --> 00:01:38.464
And what do you suppose
29
00:01:39.478 --> 00:01:42.853
the instructor's defense would have been?
30
00:01:42.853 --> 00:01:44.605
(audience murmuring)
31
00:01:44.605 --> 00:01:46.078
You know what I meant.
32
00:01:46.078 --> 00:01:47.518
(laughs)
33
00:01:47.518 --> 00:01:48.717
Or maybe you didn't.
34
00:01:48.717 --> 00:01:50.300
Let's try this one.
35
00:02:04.886 --> 00:02:06.752
Pretty straight-forward, right?
36
00:02:06.752 --> 00:02:09.806
(audience murmuring)
37
00:02:09.806 --> 00:02:12.136
So what's the answer?
38
00:02:12.136 --> 00:02:13.936
(laughs)
39
00:02:13.936 --> 00:02:15.019
I don't know!
40
00:02:16.787 --> 00:02:19.620
I don't what Jasmine had to spend.
41
00:02:23.107 --> 00:02:27.274
I could tell you what would be the total cost, without
42
00:02:28.558 --> 00:02:31.641
without tax, I could figure that out.
43
00:02:32.696 --> 00:02:34.846
Maybe Jasmine doesn't have a penny to her name,
44
00:02:34.846 --> 00:02:37.604
and she's borrowed a friend's debit card.
45
00:02:37.604 --> 00:02:40.604
(audience laughter)
46
00:02:42.518 --> 00:02:44.906
Let's try a few fill-in questions.
47
00:02:44.906 --> 00:02:46.587
Those are easy-breezy.
48
00:02:46.587 --> 00:02:48.170
How about this one?
49
00:02:55.479 --> 00:02:58.562
(audience murmuring)
50
00:03:04.158 --> 00:03:07.070
All right, someone give me an answer.
51
00:03:07.070 --> 00:03:08.131
(speaking off microphone)
52
00:03:08.131 --> 00:03:11.040
Reflect, good question, good try, but no.
53
00:03:11.040 --> 00:03:11.931
Reverse?
54
00:03:11.931 --> 00:03:13.397
Reverse, good try, but no.
55
00:03:13.397 --> 00:03:14.505
Distort.
56
00:03:14.505 --> 00:03:16.838
Distort, good try, but no.
57
00:03:23.510 --> 00:03:26.600
Well, of course, the answer as we discussed in class,
58
00:03:26.600 --> 00:03:27.517
was invert.
59
00:03:29.710 --> 00:03:30.543
Right?
60
00:03:32.526 --> 00:03:34.110
Isn't that (mumbles off mic)?
61
00:03:34.110 --> 00:03:38.277
Isn't that reverse, no actually, reverse (laughs).
62
00:03:39.130 --> 00:03:41.213
Okay, let's try this one.
63
00:03:42.691 --> 00:03:45.691
(audience laughter)
64
00:03:51.920 --> 00:03:53.670
So what's the answer?
65
00:03:53.670 --> 00:03:55.490
Anything.
66
00:03:55.490 --> 00:03:59.490
Anything, good try, but no, no credit for you.
67
00:04:02.177 --> 00:04:05.159
There we go, we need context clues, don't we?
68
00:04:05.159 --> 00:04:08.659
Well, the answer, of course, is astronomy.
69
00:04:10.330 --> 00:04:13.320
What else could it have been, right?
70
00:04:13.320 --> 00:04:15.211
Let's try this one.
71
00:04:15.211 --> 00:04:18.211
(audience laughter)
72
00:04:24.505 --> 00:04:26.030
Capitalist?
73
00:04:26.030 --> 00:04:28.930
Capitalist, good try, but no, no credit, sorry.
74
00:04:28.930 --> 00:04:30.222
Mixed?
75
00:04:30.222 --> 00:04:31.966
Mixed, no, sorry.
76
00:04:31.966 --> 00:04:34.507
Recessing.
Recessing.
77
00:04:34.507 --> 00:04:37.451
(laughter)
78
00:04:37.451 --> 00:04:40.118
Resuscitating, no.
79
00:04:44.479 --> 00:04:45.768
Great?
80
00:04:45.768 --> 00:04:48.101
Great, okay, good try, no.
81
00:04:50.176 --> 00:04:52.094
Market?
82
00:04:52.094 --> 00:04:53.485
Unstable.
83
00:04:53.485 --> 00:04:56.575
Unstable, good try, but no.
84
00:04:56.575 --> 00:04:58.791
Of course, if you had been paying attention in class,
85
00:04:58.791 --> 00:05:00.604
you would have known this, right?
86
00:05:00.604 --> 00:05:02.271
Market-based, right?
87
00:05:03.379 --> 00:05:06.501
Market-based, now, you're thinking to yourself,
88
00:05:06.501 --> 00:05:09.621
"Boy, these are silly questions."
89
00:05:09.621 --> 00:05:11.381
Every one of these examples
90
00:05:11.381 --> 00:05:14.021
came from actual class tests.
91
00:05:14.021 --> 00:05:15.680
Let's try this one.
92
00:05:15.680 --> 00:05:18.763
(audience murmuring)
93
00:05:32.810 --> 00:05:36.372
Oh, come come, now, don't be shy.
94
00:05:36.372 --> 00:05:38.122
From Mexico.
95
00:05:38.122 --> 00:05:40.042
From Mexico, I like that answer.
96
00:05:40.042 --> 00:05:42.823
That wasn't the one that the instructor was looking for.
97
00:05:42.823 --> 00:05:45.823
(audience laughter)
98
00:05:47.878 --> 00:05:49.341
Yes, sir.
99
00:05:49.341 --> 00:05:51.280
(speaking off microphone)
100
00:05:51.280 --> 00:05:53.060
I like that, too, but that's not what
101
00:05:53.060 --> 00:05:55.170
the instructor was looking for.
102
00:05:55.170 --> 00:05:56.442
Mexican, I.
103
00:05:56.442 --> 00:05:59.651
(audience laughter)
104
00:05:59.651 --> 00:06:01.901
(applause)
105
00:06:02.791 --> 00:06:06.208
It is difficult to argue with that logic.
106
00:06:08.061 --> 00:06:10.394
Let's try another, shall we?
107
00:06:11.977 --> 00:06:15.102
(audience laughter)
108
00:06:15.102 --> 00:06:18.185
(audience murmuring)
109
00:06:19.740 --> 00:06:22.373
All right, what would you say about the career of Hitler?
110
00:06:22.373 --> 00:06:24.303
This is from a history class.
111
00:06:24.303 --> 00:06:26.303
I'll give you that clue.
112
00:06:30.103 --> 00:06:31.270
Failed artist.
113
00:06:32.478 --> 00:06:33.778
Truth.
114
00:06:33.778 --> 00:06:35.971
Also, (mumbles off mic) opinion.
115
00:06:35.971 --> 00:06:37.628
I like that.
116
00:06:37.628 --> 00:06:40.711
(audience murmuring)
117
00:06:41.554 --> 00:06:45.045
If you are faced with a task like this,
118
00:06:45.045 --> 00:06:48.303
put yourself in the shoes of someone in that class.
119
00:06:48.303 --> 00:06:52.053
What do you do when you get a task like this?
120
00:06:53.544 --> 00:06:56.565
Try your best.
121
00:06:56.565 --> 00:06:59.701
You start writing as much as you can think about
122
00:06:59.701 --> 00:07:03.141
related to Hitler and hope that it sticks, right?
123
00:07:03.141 --> 00:07:06.245
(audience laughter)
124
00:07:06.245 --> 00:07:10.263
All right, let's try a simple matching exercise.
125
00:07:10.263 --> 00:07:13.943
You've got three premises and three alternatives.
126
00:07:13.943 --> 00:07:16.776
Let's see if we can sort them out.
127
00:07:19.393 --> 00:07:22.476
(audience murmuring)
128
00:07:27.740 --> 00:07:31.034
Now, maybe you didn't prepare for this one.
129
00:07:31.034 --> 00:07:34.701
But let's see if we can get it, nonetheless.
130
00:07:35.565 --> 00:07:37.834
Au, what's the answer?
131
00:07:37.834 --> 00:07:38.743
C.
132
00:07:38.743 --> 00:07:41.162
Great, and you didn't even study.
133
00:07:41.162 --> 00:07:42.794
Eight?
134
00:07:42.794 --> 00:07:43.681
A.
135
00:07:43.681 --> 00:07:46.322
Very good, and finally, Dr. Sabin?
136
00:07:46.322 --> 00:07:48.034
B.
137
00:07:48.034 --> 00:07:50.784
How did you do so well on that?
138
00:07:53.909 --> 00:07:56.992
(audience murmuring)
139
00:08:00.714 --> 00:08:03.482
Okay, so the big question is,
140
00:08:03.482 --> 00:08:07.315
what went wrong with every one of these items?
141
00:08:08.459 --> 00:08:11.292
They were very easy.
142
00:08:12.602 --> 00:08:16.519
The fact of the matter is, nobody vetted them
143
00:08:17.983 --> 00:08:21.442
to assure that they were competent,
144
00:08:21.442 --> 00:08:24.874
that they were posing the task intended,
145
00:08:24.874 --> 00:08:29.081
that they weren't ridiculous, like all Mexicans are blank.
146
00:08:29.081 --> 00:08:29.914
Okay.
147
00:08:32.014 --> 00:08:33.339
None of you,
148
00:08:33.339 --> 00:08:34.910
probably few of you
149
00:08:34.910 --> 00:08:38.170
have had any kind of background in developing tests,
150
00:08:38.170 --> 00:08:39.992
any formal background in developing tests,
151
00:08:39.992 --> 00:08:43.960
and yet, you see the silliness before you.
152
00:08:43.960 --> 00:08:47.529
Okay, it doesn't take a lot of effort
153
00:08:47.529 --> 00:08:49.862
to see when things go wrong.
154
00:08:51.059 --> 00:08:54.392
The trick is to be able to foresee that,
155
00:08:56.167 --> 00:08:58.779
and avoid it happening in the first place.
156
00:08:58.779 --> 00:09:02.947
Okay, so if we're gonna talk about all the issues
157
00:09:02.947 --> 00:09:07.835
that are instructor concerns, these would be them,
158
00:09:07.835 --> 00:09:10.307
in my opinion, okay.
159
00:09:10.307 --> 00:09:13.539
Figuring out what to assess in a class,
160
00:09:13.539 --> 00:09:16.350
because you can't cover everything
161
00:09:16.350 --> 00:09:20.059
through assignments or tests, you have to be selective.
162
00:09:20.059 --> 00:09:22.291
You have to pick and choose what you think
163
00:09:22.291 --> 00:09:25.241
are the key, the important, the vital,
164
00:09:25.241 --> 00:09:27.291
the skills that they have to have to succeed
165
00:09:27.291 --> 00:09:31.390
in the next class, or later in the course, whatever,
166
00:09:31.390 --> 00:09:33.937
those are judgments that have to be made.
167
00:09:33.937 --> 00:09:35.354
How to assess it.
168
00:09:37.868 --> 00:09:40.293
When my daughter, when my older daughter
169
00:09:40.293 --> 00:09:42.210
was taking driver's ed,
170
00:09:43.446 --> 00:09:45.569
they had one car that they would take out
171
00:09:45.569 --> 00:09:48.498
and the rest of the kids would be in study hall.
172
00:09:48.498 --> 00:09:51.306
And so, three kids and the instructor
173
00:09:51.306 --> 00:09:52.946
would be out at any one time,
174
00:09:52.946 --> 00:09:55.426
and she was out on one of these excursions,
175
00:09:55.426 --> 00:09:58.394
and you probably remember how that goes.
176
00:09:58.394 --> 00:10:01.227
They swap driving responsibilities
177
00:10:03.301 --> 00:10:05.676
during the cruise and then come back
178
00:10:05.676 --> 00:10:07.975
and then get another batch, and so forth.
179
00:10:07.975 --> 00:10:11.594
Well, the instructor, a coach, thought this would be
180
00:10:11.594 --> 00:10:15.983
a teachable moment for how to change a spare tire
181
00:10:15.983 --> 00:10:20.154
on this one day on this one loop that they were making.
182
00:10:20.154 --> 00:10:23.125
So, he tells the driver, "Young man,
183
00:10:23.125 --> 00:10:28.125
"pull the car over, and let's see you change the tire."
184
00:10:28.125 --> 00:10:31.933
And the student is game for anything.
185
00:10:31.933 --> 00:10:34.263
So he says, "Yes, sir."
186
00:10:34.263 --> 00:10:36.654
Pulls the car over,
187
00:10:36.654 --> 00:10:40.566
finds the jack, equipment, successfully,
188
00:10:40.566 --> 00:10:41.707
and fortunately, this is a car
189
00:10:41.707 --> 00:10:44.008
that actually has a spare tire in it.
190
00:10:44.008 --> 00:10:46.715
Not all of them do, these days.
191
00:10:46.715 --> 00:10:49.798
Puts the jack in place, cranks it up,
192
00:10:51.495 --> 00:10:55.267
and is ready to proceed with other parts of the process,
193
00:10:55.267 --> 00:10:59.434
when one of the other students says, "Coach Blank,
194
00:11:01.064 --> 00:11:03.314
"I see something dripping."
195
00:11:04.395 --> 00:11:09.195
Sure enough, the kid had put the jack on the gas line,
196
00:11:09.195 --> 00:11:13.278
broken the gas line, and gasoline was coming out.
197
00:11:14.885 --> 00:11:18.705
What went wrong with that, what could be more authentic
198
00:11:18.705 --> 00:11:21.094
than telling someone to change the tire,
199
00:11:21.094 --> 00:11:23.016
to see if they know how to change a tire.
200
00:11:23.016 --> 00:11:25.849
That is as high fidelity a measure
201
00:11:28.075 --> 00:11:32.075
as you could possibly have, but what went wrong?
202
00:11:33.700 --> 00:11:36.508
(audience murmuring)
203
00:11:36.508 --> 00:11:38.405
Didn't teach 'em how, first.
204
00:11:38.405 --> 00:11:41.155
Well, maybe they should have taught 'em how first.
205
00:11:41.155 --> 00:11:43.363
Yeah, I like that.
206
00:11:43.363 --> 00:11:45.395
But the fact of the matter is,
207
00:11:45.395 --> 00:11:48.645
the instructor wasn't paying attention,
208
00:11:49.670 --> 00:11:52.520
step by step, to what was going on
209
00:11:52.520 --> 00:11:55.115
and correcting it if it was wrong.
210
00:11:55.115 --> 00:11:58.085
Just letting the kid go with the flow.
211
00:11:58.085 --> 00:12:00.576
Okay, and you think, "Oh gee, nobody does that."
212
00:12:00.576 --> 00:12:01.656
Oh, yeah.
213
00:12:01.656 --> 00:12:04.757
Okay, test construction, performance assessment,
214
00:12:04.757 --> 00:12:07.608
in other words, what kinds of work products or samples
215
00:12:07.608 --> 00:12:09.397
would be helpful for making decisions
216
00:12:09.397 --> 00:12:13.819
about student proficiency, grading and marking.
217
00:12:13.819 --> 00:12:17.986
How many of you believe in the grading on the curve?
218
00:12:20.627 --> 00:12:22.517
(audience murmuring)
219
00:12:22.517 --> 00:12:23.517
Let me change the question.
220
00:12:23.517 --> 00:12:25.416
How many of you have ever been in a class
221
00:12:25.416 --> 00:12:28.219
where the instructor has graded on the curve?
222
00:12:28.219 --> 00:12:30.219
Holy cats, look at that.
223
00:12:31.425 --> 00:12:35.042
What's going on when that happens?
224
00:12:35.042 --> 00:12:36.736
Most people are doing bad.
225
00:12:36.736 --> 00:12:37.569
Say again?
226
00:12:37.569 --> 00:12:38.665
Most people are doing bad.
227
00:12:38.665 --> 00:12:41.658
Oh, that's a very good observation.
228
00:12:41.658 --> 00:12:44.109
A lot of instructors say to themselves,
229
00:12:44.109 --> 00:12:45.489
"Well, I'm not gonna do that."
230
00:12:45.489 --> 00:12:47.600
Until they get the first test results back,
231
00:12:47.600 --> 00:12:49.451
and they find that, as a group,
232
00:12:49.451 --> 00:12:51.731
the class has kind of crashed and burned.
233
00:12:51.731 --> 00:12:54.981
So they say, "Okay, 70 is the new 100."
234
00:12:56.520 --> 00:12:59.138
(laughing)
235
00:12:59.138 --> 00:13:01.088
And on it goes.
236
00:13:01.088 --> 00:13:03.505
The problem with that system?
237
00:13:06.046 --> 00:13:07.504
It's not actually an assessment
238
00:13:07.504 --> 00:13:10.091
of the capabilities (mumbles off mic).
239
00:13:10.091 --> 00:13:11.091
Very good.
240
00:13:12.051 --> 00:13:14.689
Depending on whether you, as an individual,
241
00:13:14.689 --> 00:13:17.022
happen in a very able group,
242
00:13:18.381 --> 00:13:21.594
your standards of performance are gonna be much higher
243
00:13:21.594 --> 00:13:25.761
if you're lucky enough to be in a bunch of beanheads,
244
00:13:26.647 --> 00:13:29.308
then your standards are gonna be much lower,
245
00:13:29.308 --> 00:13:32.075
and the problem is the standards are gonna fluctuate
246
00:13:32.075 --> 00:13:33.434
from class to class.
247
00:13:33.434 --> 00:13:35.375
So obviously, when you go to bed at night,
248
00:13:35.375 --> 00:13:38.141
you pray for dumber classmates, right?
249
00:13:38.141 --> 00:13:40.085
(audience laughter)
250
00:13:40.085 --> 00:13:43.826
All right, feedback to students about performance.
251
00:13:43.826 --> 00:13:46.477
What do you think the literature says,
252
00:13:46.477 --> 00:13:49.127
the research says, about feedback?
253
00:13:49.127 --> 00:13:51.077
Good thing or a bad thing?
254
00:13:51.077 --> 00:13:52.218
Good thing.
255
00:13:52.218 --> 00:13:54.416
Very good, it is a good thing.
256
00:13:54.416 --> 00:13:57.007
How quickly should students get feedback
257
00:13:57.007 --> 00:13:59.007
about their performance?
258
00:13:59.007 --> 00:14:02.386
(audience speaking at once)
259
00:14:02.386 --> 00:14:05.186
How soon is, as soon as possible?
260
00:14:05.186 --> 00:14:08.055
When they get their grades back.
261
00:14:08.055 --> 00:14:10.472
How about letting them know
262
00:14:11.637 --> 00:14:13.749
two minutes after they've turned in the test
263
00:14:13.749 --> 00:14:16.274
or completed the test, if it's online?
264
00:14:16.274 --> 00:14:18.265
It's not possible to get online
265
00:14:18.265 --> 00:14:20.399
because the computer graded it right after it.
266
00:14:20.399 --> 00:14:22.399
There's a good question.
267
00:14:22.399 --> 00:14:25.688
The fact of the matter is, instant or near-instant
268
00:14:25.688 --> 00:14:28.760
feedback turns out not to be nearly as effective
269
00:14:28.760 --> 00:14:32.510
as feedback that's delayed for say, 24 hours.
270
00:14:33.630 --> 00:14:35.130
Can you guess why?
271
00:14:37.621 --> 00:14:39.821
We haven't had time to actually look at
272
00:14:39.821 --> 00:14:42.043
the same paper in depth.
273
00:14:42.043 --> 00:14:44.112
That's a good point, but the other side
274
00:14:44.112 --> 00:14:47.549
of that equation is the student hasn't had time
275
00:14:47.549 --> 00:14:50.672
to process what they've been through.
276
00:14:50.672 --> 00:14:54.182
If you simply tell 'em, "Boy, your score is blah,"
277
00:14:54.182 --> 00:14:56.782
they're gonna stop the processing right then and there.
278
00:14:56.782 --> 00:14:59.102
And they'll move on to the next thing.
279
00:14:59.102 --> 00:15:02.811
So, in point of fact, you know, prompt feedback
280
00:15:02.811 --> 00:15:05.158
is a good thing, and by prompt, I mean in the next
281
00:15:05.158 --> 00:15:09.758
day or two, but as opposed to the next semester.
282
00:15:09.758 --> 00:15:11.864
(audience laughter)
283
00:15:11.864 --> 00:15:14.710
But instant feedback, not so much.
284
00:15:14.710 --> 00:15:19.531
Cheating, appeals, and other issues, those are gonna happen.
285
00:15:19.531 --> 00:15:23.489
So if you have responsibilities associated with the course,
286
00:15:23.489 --> 00:15:27.920
the recommendation is, hold on to student papers
287
00:15:27.920 --> 00:15:30.251
for at least a semester.
288
00:15:30.251 --> 00:15:33.421
There are gonna be folks that recommend other things.
289
00:15:33.421 --> 00:15:37.094
The great appeal process at MSU has a timeline
290
00:15:37.094 --> 00:15:39.641
associated with it, so you don't have to hold things
291
00:15:39.641 --> 00:15:41.654
seven years later, and so forth,
292
00:15:41.654 --> 00:15:44.275
but do hold on to some things.
293
00:15:44.275 --> 00:15:47.504
Okay, so what's important to assess?
294
00:15:47.504 --> 00:15:52.184
There's some, there's a, I guess, a fair trade off
295
00:15:52.184 --> 00:15:56.484
between everything in the book, which is probably overkill,
296
00:15:56.484 --> 00:15:59.754
and then little or nothing, under the assumption
297
00:15:59.754 --> 00:16:03.787
that they'll remember little or nothing afterwards.
298
00:16:03.787 --> 00:16:07.038
But to me, what are they gonna use later in the class,
299
00:16:07.038 --> 00:16:11.205
what are they gonna use later in subsequent classes?
300
00:16:12.638 --> 00:16:14.856
All right, let me reminisce a bit.
301
00:16:14.856 --> 00:16:17.707
My freshman orientation at college,
302
00:16:17.707 --> 00:16:21.987
I was going to a school that was on the coast of Florida.
303
00:16:21.987 --> 00:16:26.608
Every student had to take a swimming proficiency exam.
304
00:16:26.608 --> 00:16:28.379
Didn't matter what your major was,
305
00:16:28.379 --> 00:16:31.158
didn't matter whether you were able-bodied or disabled,
306
00:16:31.158 --> 00:16:34.769
every student had to take a swimming proficiency exam.
307
00:16:34.769 --> 00:16:37.528
Why would they take important time
308
00:16:37.528 --> 00:16:40.111
out of the universe to do that?
309
00:16:42.119 --> 00:16:43.761
(audience murmuring)
310
00:16:43.761 --> 00:16:44.884
Location.
311
00:16:44.884 --> 00:16:45.717
Sir.
312
00:16:45.717 --> 00:16:46.550
Location.
313
00:16:46.550 --> 00:16:47.633
Location, location, location,
314
00:16:47.633 --> 00:16:49.441
just like the real estate agent.
315
00:16:49.441 --> 00:16:52.978
And they knew what college kids are all about.
316
00:16:52.978 --> 00:16:55.728
Okay, they know that some of them
317
00:16:57.014 --> 00:16:58.622
might get close to the water
318
00:16:58.622 --> 00:17:01.361
in less than full possession of their faculties,
319
00:17:01.361 --> 00:17:03.430
and they wanted to know that,
320
00:17:03.430 --> 00:17:06.132
you know, you could get to safety, if necessary.
321
00:17:06.132 --> 00:17:08.881
In point of fact, if you didn't pass that exam,
322
00:17:08.881 --> 00:17:12.689
you were signed up for swimming your first semester.
323
00:17:12.689 --> 00:17:15.478
No ifs, no ands, no buts.
324
00:17:15.478 --> 00:17:17.190
It was that important to them.
325
00:17:17.190 --> 00:17:20.009
So, clearly, one of the things you wanna be thinking about,
326
00:17:20.009 --> 00:17:21.708
or any instructor should be thinking about,
327
00:17:21.708 --> 00:17:26.030
is what are the important take-aways for this course?
328
00:17:26.030 --> 00:17:29.313
Because if you're gonna be spending time on it,
329
00:17:29.313 --> 00:17:31.612
in class, if you're gonna be spending time on it,
330
00:17:31.612 --> 00:17:33.233
looking at student work,
331
00:17:33.233 --> 00:17:34.673
if you're gonna be spending time on it
332
00:17:34.673 --> 00:17:38.084
assessing student skills, then you know,
333
00:17:38.084 --> 00:17:41.724
you're giving up something else, so prioritize.
334
00:17:41.724 --> 00:17:45.964
Okay, objectives are a big way, a simple way,
335
00:17:45.964 --> 00:17:49.094
to help students figure out what's important to you.
336
00:17:49.094 --> 00:17:52.345
Just like old tests are a good way to let students know
337
00:17:52.345 --> 00:17:53.733
what are important to you.
338
00:17:53.733 --> 00:17:56.015
Or study guides are a good way to let students know
339
00:17:56.015 --> 00:17:57.904
what's important to you.
340
00:17:57.904 --> 00:18:00.755
In fact, the day before a test,
341
00:18:00.755 --> 00:18:02.625
or the class meeting before the test,
342
00:18:02.625 --> 00:18:06.366
what's the most common question, do you suppose?
343
00:18:06.366 --> 00:18:07.665
Tell me the questions--
344
00:18:07.665 --> 00:18:10.457
Is this gonna be on the test?
345
00:18:10.457 --> 00:18:13.473
Why do you suppose students ask that?
346
00:18:13.473 --> 00:18:15.100
(audience speaking off microphone)
347
00:18:15.100 --> 00:18:18.988
They want to know what to study, for goodness sake, yes.
348
00:18:18.988 --> 00:18:21.889
Of course it makes sense from that point of view.
349
00:18:21.889 --> 00:18:25.729
Okay, so should it be kept a deep, dark secret?
350
00:18:25.729 --> 00:18:29.289
Oh, it's gonna come from the chapters.
351
00:18:29.289 --> 00:18:30.996
Or not (laughs).
352
00:18:30.996 --> 00:18:33.387
(audience laughter)
353
00:18:33.387 --> 00:18:34.679
That's pretty good.
354
00:18:34.679 --> 00:18:36.494
Nothing puts students at ease more
355
00:18:36.494 --> 00:18:39.454
than that kind of coaching, okay.
356
00:18:39.454 --> 00:18:43.225
So, think, just think about, how do I communicate to them
357
00:18:43.225 --> 00:18:45.865
what the heck I think is important for them?
358
00:18:45.865 --> 00:18:49.296
Important enough to guide their study efforts,
359
00:18:49.296 --> 00:18:51.446
important enough to assess them on,
360
00:18:51.446 --> 00:18:55.168
and important enough to make decisions about them.
361
00:18:55.168 --> 00:18:58.388
Okay, so there's all kinds of ways to make that happen.
362
00:18:58.388 --> 00:19:01.382
How to assess, we talked a little bit
363
00:19:01.382 --> 00:19:05.717
in the tire changing about task authenticity.
364
00:19:05.717 --> 00:19:08.192
The more authentic the task,
365
00:19:08.192 --> 00:19:12.058
the less people can bitch and moan, and say,
366
00:19:12.058 --> 00:19:15.230
"Oh, that wasn't realistic, that wasn't real world."
367
00:19:15.230 --> 00:19:18.161
Okay, you can make tasks as real world as you like,
368
00:19:18.161 --> 00:19:21.328
but what you give up when you do that?
369
00:19:25.033 --> 00:19:26.433
There's several things that, unfortunately,
370
00:19:26.433 --> 00:19:28.246
you have to give up when you do that.
371
00:19:28.246 --> 00:19:31.996
Every other student in that car, on that day,
372
00:19:34.107 --> 00:19:37.107
learned something from that mistake.
373
00:19:38.672 --> 00:19:41.974
So, if the coach was to take the other students
374
00:19:41.974 --> 00:19:44.691
and say, "Okay, now you change the tire."
375
00:19:44.691 --> 00:19:47.251
Are they starting from the same baseline
376
00:19:47.251 --> 00:19:49.443
that the first student did?
377
00:19:49.443 --> 00:19:53.683
No, so in that sense, it's not fair to everybody
378
00:19:53.683 --> 00:19:54.982
to do it that way.
379
00:19:54.982 --> 00:19:57.683
Just like, if I go around the class,
380
00:19:57.683 --> 00:20:01.534
and ask everyone, "What's the capital of Mississippi?"
381
00:20:01.534 --> 00:20:02.367
Jackson.
382
00:20:02.367 --> 00:20:05.752
Jackson, beautiful, what's the capital of Mississippi?
383
00:20:05.752 --> 00:20:06.911
Jackson.
384
00:20:06.911 --> 00:20:09.653
Perfect, isn't that great, everybody has learned
385
00:20:09.653 --> 00:20:13.094
the capital, okay, you know what's wrong with that system.
386
00:20:13.094 --> 00:20:16.372
Okay, so figuring out how to sort it out
387
00:20:16.372 --> 00:20:19.039
in a way that makes it feasible,
388
00:20:21.024 --> 00:20:24.051
and something you can handle.
389
00:20:24.051 --> 00:20:26.353
Yeah, you have to work on it.
390
00:20:26.353 --> 00:20:30.900
Okay, at the bottom, there's a lot of information
391
00:20:30.900 --> 00:20:33.983
about, well, let me rewind that tape.
392
00:20:35.611 --> 00:20:37.563
Have you ever been in a class
393
00:20:37.563 --> 00:20:40.033
where the instructor asked,
394
00:20:40.033 --> 00:20:41.783
or gave you the task,
395
00:20:43.100 --> 00:20:47.439
"Tell me what grade you think earned, and why?"
396
00:20:47.439 --> 00:20:50.272
Have you ever gotten one of those?
397
00:20:51.690 --> 00:20:53.663
So, what do you say?
398
00:20:53.663 --> 00:20:54.887
A.
399
00:20:54.887 --> 00:20:56.238
(laughing)
400
00:20:56.238 --> 00:20:57.155
Shoot high!
401
00:21:00.092 --> 00:21:01.882
I learned so much in this course,
402
00:21:01.882 --> 00:21:04.610
I never knew how important this material was.
403
00:21:04.610 --> 00:21:08.060
Oh, my gracious, sakes alive, I am virtually
404
00:21:08.060 --> 00:21:12.227
self-fulfilled, thanks to having completed this class.
405
00:21:13.623 --> 00:21:17.131
The fact of the matter is, college-age students,
406
00:21:17.131 --> 00:21:20.154
and I'm not saying anything that isn't supported
407
00:21:20.154 --> 00:21:22.514
by the literature here, more or less,
408
00:21:22.514 --> 00:21:25.503
don't have a lot of self-appraisal skills
409
00:21:25.503 --> 00:21:28.488
that are highly accurate, that,
410
00:21:28.488 --> 00:21:31.436
the fancy word we use for that is, calibrated.
411
00:21:31.436 --> 00:21:33.295
They're not well-calibrated to figuring out
412
00:21:33.295 --> 00:21:35.964
what they know and what they don't know.
413
00:21:35.964 --> 00:21:39.674
In point of fact, the people who know the least
414
00:21:39.674 --> 00:21:43.295
tend to overestimate what they know,
415
00:21:43.295 --> 00:21:45.594
and the people who know the most,
416
00:21:45.594 --> 00:21:48.474
tend to underestimate what they know.
417
00:21:48.474 --> 00:21:51.330
So you're getting penalized at both,
418
00:21:51.330 --> 00:21:54.842
you know, biased at both ends of the continuum.
419
00:21:54.842 --> 00:21:57.671
So that's probably not a good strategy.
420
00:21:57.671 --> 00:21:59.838
All right, open book test.
421
00:22:01.602 --> 00:22:04.132
What to you think, a good thing or a bad thing?
422
00:22:04.132 --> 00:22:06.277
(audience speaking at once)
423
00:22:06.277 --> 00:22:08.259
Oh, who said, "I hate 'em?"
424
00:22:08.259 --> 00:22:09.511
How come?
425
00:22:09.511 --> 00:22:11.720
The questions are usually more detailed,
426
00:22:11.720 --> 00:22:14.742
and they take longer, and I'm gonna study regardless,
427
00:22:14.742 --> 00:22:17.422
and I don't like, all these people that didn't study
428
00:22:17.422 --> 00:22:19.800
get to use the book and feels like they get to cheat.
429
00:22:19.800 --> 00:22:22.769
Oh, okay, all right,
430
00:22:22.769 --> 00:22:27.039
so there's an equity issue associated with that.
431
00:22:27.039 --> 00:22:29.311
Any other opinions, yes ma'am?
432
00:22:29.311 --> 00:22:30.849
I thought my score when I took
433
00:22:30.849 --> 00:22:32.721
an open book test, students tend to perform less on,
434
00:22:32.721 --> 00:22:34.320
I don't think they prepared, and plus,
435
00:22:34.320 --> 00:22:38.350
they spend too much time looking for (mumbles off mic).
436
00:22:38.350 --> 00:22:42.517
(chuckles) The last part is absolutely true.
437
00:22:45.180 --> 00:22:49.013
If we could broaden that scope to just saying,
438
00:22:50.431 --> 00:22:54.599
"Are you gonna allow students access to resources?"
439
00:22:54.599 --> 00:22:57.489
A lot of times, the answer is, no.
440
00:22:57.489 --> 00:23:00.647
And for some instructors, the answer is, "Well, yes,
441
00:23:00.647 --> 00:23:04.497
"because in real life you would have access to resources."
442
00:23:04.497 --> 00:23:07.487
When I give tests in my class, they are open book,
443
00:23:07.487 --> 00:23:11.887
open note, open anything, except another human being.
444
00:23:11.887 --> 00:23:15.545
Okay, and to me, that includes the interweb.
445
00:23:15.545 --> 00:23:17.462
So, they can't do that.
446
00:23:18.974 --> 00:23:22.612
Why, because to my way of thinking,
447
00:23:22.612 --> 00:23:24.860
that's they're gonna do it in real life,
448
00:23:24.860 --> 00:23:27.753
the other thing, however, is before a test,
449
00:23:27.753 --> 00:23:30.785
I coach 'em, or I try my best to coach 'em
450
00:23:30.785 --> 00:23:34.233
to how to organize their materials and their notes
451
00:23:34.233 --> 00:23:37.345
so they won't be fumbling through 500 pages,
452
00:23:37.345 --> 00:23:41.007
looking for something, and if they fail to do that,
453
00:23:41.007 --> 00:23:42.945
well, it's on them, isn't it.
454
00:23:42.945 --> 00:23:47.112
Okay, and yes, I give questions that don't lend themselves
455
00:23:48.134 --> 00:23:51.777
to looking up the sentence in the book that says,
456
00:23:51.777 --> 00:23:56.113
"The career of Hitler was," (laughs) "mixed."
457
00:23:56.113 --> 00:23:59.503
(audience laughter)
458
00:23:59.503 --> 00:24:01.670
All right, take home test.
459
00:24:03.052 --> 00:24:05.703
All the people who have studied this
460
00:24:05.703 --> 00:24:08.633
are convinced based on evidence that students
461
00:24:08.633 --> 00:24:10.873
don't learn as much of the material,
462
00:24:10.873 --> 00:24:14.873
nor do they retain it as well as in class tests,
463
00:24:15.915 --> 00:24:19.668
or tests that you take as part of the class.
464
00:24:19.668 --> 00:24:23.835
Even if it's online, maybe under a timeframe, same effect,
465
00:24:25.018 --> 00:24:29.184
not, take home is not nearly as effective
466
00:24:29.184 --> 00:24:31.149
as a learning tool.
467
00:24:31.149 --> 00:24:33.399
Why do you suppose that is?
468
00:24:34.471 --> 00:24:36.041
Distractions.
469
00:24:36.041 --> 00:24:38.708
Distractions, sure, what else?
470
00:24:40.557 --> 00:24:43.923
They take it and just (mumbles off mic)
471
00:24:43.923 --> 00:24:45.686
and they don't actually learn it, they just.
472
00:24:45.686 --> 00:24:46.519
(mumbles off mic)
473
00:24:46.519 --> 00:24:47.352
Exactly.
474
00:24:47.352 --> 00:24:49.374
They give it to a friend to take.
475
00:24:49.374 --> 00:24:52.824
(laughs) Even more plausible.
476
00:24:52.824 --> 00:24:54.566
The fact of the matter is, yes,
477
00:24:54.566 --> 00:24:57.595
what do they focus on, they focus on the specific tasks
478
00:24:57.595 --> 00:25:00.614
in the take home test, they work just as long
479
00:25:00.614 --> 00:25:03.358
as they think it takes to address those tasks,
480
00:25:03.358 --> 00:25:08.103
and no others, and that's it, close the book, I'm done.
481
00:25:08.103 --> 00:25:10.307
Okay, so it's not surprising that the breadth
482
00:25:10.307 --> 00:25:12.865
of what they learn isn't as good,
483
00:25:12.865 --> 00:25:16.574
and it's not surprising that they don't take it away
484
00:25:16.574 --> 00:25:18.594
and retain it as well.
485
00:25:18.594 --> 00:25:20.515
So, something to think about.
486
00:25:20.515 --> 00:25:22.023
Oh, collaborative test.
487
00:25:22.023 --> 00:25:23.662
Letting students take tests together.
488
00:25:23.662 --> 00:25:27.062
What do you think the outcome of that is?
489
00:25:27.062 --> 00:25:29.834
(audience speaking of microphone)
490
00:25:29.834 --> 00:25:31.809
All right, Question Number One,
491
00:25:31.809 --> 00:25:33.321
do you think students like it?
492
00:25:33.321 --> 00:25:34.251
Yes.
493
00:25:34.251 --> 00:25:38.369
(audience talking at once)
494
00:25:38.369 --> 00:25:41.710
Very good, it actually turns out to be a mixed bag.
495
00:25:41.710 --> 00:25:44.401
On average, yes, students will say,
496
00:25:44.401 --> 00:25:47.733
"I love this format, thank you, instructor."
497
00:25:47.733 --> 00:25:50.372
However, some students are going to be
498
00:25:50.372 --> 00:25:53.566
unhappy with that format, why?
499
00:25:53.566 --> 00:25:54.995
(audience talking at once)
500
00:25:54.995 --> 00:25:57.693
Exactly, it's the same thing that happens oftentimes
501
00:25:57.693 --> 00:26:00.966
with group work, some people ending up doing
502
00:26:00.966 --> 00:26:02.635
the heavy lifting, and the others are just
503
00:26:02.635 --> 00:26:03.974
along for the ride.
504
00:26:03.974 --> 00:26:06.124
And some folks think that's an unfair system,
505
00:26:06.124 --> 00:26:07.707
and in fact, it is.
506
00:26:10.217 --> 00:26:12.518
What do you think happens in terms of the overall
507
00:26:12.518 --> 00:26:15.176
amount of learning or retention?
508
00:26:15.176 --> 00:26:18.457
(audience murmuring)
509
00:26:18.457 --> 00:26:22.518
For the class as a whole, not as good, not as good.
510
00:26:22.518 --> 00:26:23.351
Okay.
511
00:26:25.693 --> 00:26:28.055
So if you're looking for good ratings,
512
00:26:28.055 --> 00:26:30.726
at the end of the semester, give 'em.
513
00:26:30.726 --> 00:26:33.886
If you're looking for better retention,
514
00:26:33.886 --> 00:26:37.384
and having learned more, don't use collaborative tests.
515
00:26:37.384 --> 00:26:41.484
Okay, frequency of testing, more often or less often?
516
00:26:41.484 --> 00:26:42.499
More.
517
00:26:42.499 --> 00:26:44.082
More, absolutely.
518
00:26:45.609 --> 00:26:49.262
All the surveys of students say they prefer that,
519
00:26:49.262 --> 00:26:52.011
and all the outcomes say
520
00:26:52.011 --> 00:26:54.243
that's actually more effective in the long run.
521
00:26:54.243 --> 00:26:56.493
Why do you suppose that is?
522
00:26:57.893 --> 00:26:59.091
Retaining the information,
523
00:26:59.091 --> 00:27:01.032
if you take it over a longer period of time,
524
00:27:01.032 --> 00:27:02.527
then you're gonna have a harder time
525
00:27:02.527 --> 00:27:04.763
remembering what happened two months ago, three months ago.
526
00:27:04.763 --> 00:27:08.104
Exactly, you focus their attention on
527
00:27:08.104 --> 00:27:12.286
more digestible chunks of material, that's one.
528
00:27:12.286 --> 00:27:14.376
What do you think is another reason why?
529
00:27:14.376 --> 00:27:15.888
(mumbling off microphone)
530
00:27:15.888 --> 00:27:17.829
We didn't have a chance to catch back up--
531
00:27:17.829 --> 00:27:21.127
Exactly, more opportunities to overcome a bad start.
532
00:27:21.127 --> 00:27:23.045
What was the comment up here?
533
00:27:23.045 --> 00:27:24.803
(mumbling) Across the course,
534
00:27:24.803 --> 00:27:26.434
that way, you don't have to worry about just one test.
535
00:27:26.434 --> 00:27:28.517
Absolutely, absolutely.
536
00:27:29.583 --> 00:27:32.646
It's not eggs in one basket time,
537
00:27:32.646 --> 00:27:35.871
like in the case of just a mid-term and a final,
538
00:27:35.871 --> 00:27:39.404
and, "You're on your own, my friend."
539
00:27:39.404 --> 00:27:43.291
So the students prefer it, it keeps them,
540
00:27:43.291 --> 00:27:45.171
you have a better chance of keeping them
541
00:27:45.171 --> 00:27:47.751
abreast with the material with frequent testing
542
00:27:47.751 --> 00:27:50.240
rather than with sparse testing.
543
00:27:50.240 --> 00:27:51.907
So think about that.
544
00:27:53.141 --> 00:27:53.974
Okay.
545
00:27:56.221 --> 00:27:58.471
See the arrow on the slide?
546
00:27:59.369 --> 00:28:03.536
This is it, I've digested books and books into three rules.
547
00:28:04.643 --> 00:28:06.810
What's a good test?
548
00:28:06.810 --> 00:28:11.066
Number One, it measures the intended behavior,
549
00:28:11.066 --> 00:28:12.983
and not something else.
550
00:28:13.848 --> 00:28:16.056
Not something else.
551
00:28:16.056 --> 00:28:20.390
Two, it clearly communicates, like bananas in bunches,
552
00:28:20.390 --> 00:28:22.390
the task to the student.
553
00:28:23.251 --> 00:28:27.874
And Three, it yields a reliable indication of performance.
554
00:28:27.874 --> 00:28:32.131
That is, you can use that score as an indicator
555
00:28:32.131 --> 00:28:34.585
and be confident that that's gonna tell you something
556
00:28:34.585 --> 00:28:37.158
about what the students have learned.
557
00:28:37.158 --> 00:28:41.579
Okay, those are it, the three rules for a good test.
558
00:28:41.579 --> 00:28:43.053
Okay.
559
00:28:43.053 --> 00:28:45.589
Now, how you get there, oh my goodness.
560
00:28:45.589 --> 00:28:47.718
There's a lot of stuff that goes behind that.
561
00:28:47.718 --> 00:28:50.885
Okay, these are what professionals do.
562
00:28:51.929 --> 00:28:55.762
Okay, and when professionals write test items,
563
00:28:56.834 --> 00:28:59.917
if 40% of the items they develop
564
00:29:00.779 --> 00:29:03.313
make it through all of these steps,
565
00:29:03.313 --> 00:29:07.480
they think they've done remarkably well, 40%.
566
00:29:09.493 --> 00:29:12.600
So for the rest of us, we have to,
567
00:29:12.600 --> 00:29:14.600
we have to be willing to
568
00:29:16.029 --> 00:29:18.845
know that some of the things we've written
569
00:29:18.845 --> 00:29:20.095
are not pearls.
570
00:29:21.004 --> 00:29:25.477
Like, "There have been great advances in the field of."
571
00:29:25.477 --> 00:29:27.113
Okay.
572
00:29:27.113 --> 00:29:30.871
All right, different kinds of test formats.
573
00:29:30.871 --> 00:29:33.039
Doesn't matter whether you're talking about paper, pencil
574
00:29:33.039 --> 00:29:36.640
or what have you, but two basic families.
575
00:29:36.640 --> 00:29:40.806
One, the answer is there, staring you in the face,
576
00:29:40.806 --> 00:29:42.359
and all you have to do is find it.
577
00:29:42.359 --> 00:29:45.020
That's called Selected response.
578
00:29:45.020 --> 00:29:47.393
Those are true-false, multiple choice,
579
00:29:47.393 --> 00:29:49.310
matching, and the like.
580
00:29:50.173 --> 00:29:54.839
The other family, you have to generate the response.
581
00:29:54.839 --> 00:29:57.743
It isn't there, staring you in the face.
582
00:29:57.743 --> 00:30:00.725
Those would include things like essays,
583
00:30:00.725 --> 00:30:04.642
projects, short answer, fill-ins, and so forth.
584
00:30:06.734 --> 00:30:08.992
Okay, they all have their virtues,
585
00:30:08.992 --> 00:30:12.222
and they all have their drawbacks, in point of fact,
586
00:30:12.222 --> 00:30:15.204
you can make just about any format work,
587
00:30:15.204 --> 00:30:18.065
in just about any situation, but it turns out
588
00:30:18.065 --> 00:30:21.297
that some formats are a little better suited
589
00:30:21.297 --> 00:30:23.506
for measuring certain kinds of outcomes.
590
00:30:23.506 --> 00:30:25.423
Okay, so in a nutshell,
591
00:30:27.464 --> 00:30:29.274
what are some of the general rules
592
00:30:29.274 --> 00:30:31.361
that we should be thinking about
593
00:30:31.361 --> 00:30:33.898
in terms of making up of tests?
594
00:30:33.898 --> 00:30:36.800
Number One, top of the list,
595
00:30:36.800 --> 00:30:39.458
have someone else review your work
596
00:30:39.458 --> 00:30:42.774
before you unleash it on humanity, okay?
597
00:30:42.774 --> 00:30:44.654
(audience laughter)
598
00:30:44.654 --> 00:30:48.677
That'll help with things like, "All Mexicans are blank."
599
00:30:48.677 --> 00:30:49.510
Okay.
600
00:30:54.447 --> 00:30:56.085
Be prepared to revise items.
601
00:30:56.085 --> 00:30:58.742
They're not all gonna be pearls.
602
00:30:58.742 --> 00:31:00.827
Include clear directions.
603
00:31:00.827 --> 00:31:05.126
It doesn't matter whether you tell 'em, give 'em examples,
604
00:31:05.126 --> 00:31:08.315
post a two-minute video on how your tests
605
00:31:08.315 --> 00:31:11.714
are gonna be conducted, or write it all down.
606
00:31:11.714 --> 00:31:13.944
But communicate what is it you want 'em to do
607
00:31:13.944 --> 00:31:17.587
and how is it you want 'em to do it.
608
00:31:17.587 --> 00:31:20.898
Stay away from straight quotations,
609
00:31:20.898 --> 00:31:22.739
unless that's what you're trying to measure.
610
00:31:22.739 --> 00:31:26.050
Their ability to remember quotations.
611
00:31:26.050 --> 00:31:27.528
Stay away from trick questions.
612
00:31:27.528 --> 00:31:29.363
I'm gonna show you one or two in a minute.
613
00:31:29.363 --> 00:31:33.086
And you'll see why you should stay away from 'em.
614
00:31:33.086 --> 00:31:36.274
The measurement itself should reflect the time
615
00:31:36.274 --> 00:31:40.521
and importance of the skills you're trying to assess.
616
00:31:40.521 --> 00:31:42.936
Okay, just 'cause it's easier to ask questions
617
00:31:42.936 --> 00:31:45.553
about Jackson being the capital of Mississippi,
618
00:31:45.553 --> 00:31:48.498
doesn't necessarily mean that's the way the test should go.
619
00:31:48.498 --> 00:31:52.483
If what you want the students to understand is,
620
00:31:52.483 --> 00:31:56.592
how is Mississippi different from other states?
621
00:31:56.592 --> 00:31:58.472
Okay, knowing the capital isn't going to tell you
622
00:31:58.472 --> 00:32:00.434
a heck of a lot in that regard.
623
00:32:00.434 --> 00:32:04.312
And don't do no double negatives, okay.
624
00:32:04.312 --> 00:32:06.889
All right, let's take a quiz.
625
00:32:06.889 --> 00:32:09.218
You've never studied this before.
626
00:32:09.218 --> 00:32:12.243
I've given you the answer with the asterisk,
627
00:32:12.243 --> 00:32:14.693
so here's what I want you to do.
628
00:32:14.693 --> 00:32:17.884
Tell me why that's the correct answer,
629
00:32:17.884 --> 00:32:20.884
even though we haven't studied this.
630
00:32:25.950 --> 00:32:30.117
(audience chuckling and murmuring)
631
00:32:32.576 --> 00:32:34.243
Oops, let's go back.
632
00:32:35.211 --> 00:32:38.319
Exactly, there's a clue there.
633
00:32:38.319 --> 00:32:41.543
If you don't know jack about the question,
634
00:32:41.543 --> 00:32:44.232
you start a whole new set of strategies
635
00:32:44.232 --> 00:32:46.437
and one of those strategies is,
636
00:32:46.437 --> 00:32:49.790
is there anything in the stem of the item
637
00:32:49.790 --> 00:32:53.182
that matches anything in the alternatives?
638
00:32:53.182 --> 00:32:57.836
And there is, the first one has the word, cluss.
639
00:32:57.836 --> 00:33:01.003
Turns out that's the right answer, oh!
640
00:33:01.967 --> 00:33:04.000
How about the second one?
641
00:33:04.000 --> 00:33:04.969
(speaking off microphone)
642
00:33:04.969 --> 00:33:07.585
Oh, boy, you jumped right on that.
643
00:33:07.585 --> 00:33:11.795
Notice that the stem ends with the article, an.
644
00:33:11.795 --> 00:33:15.129
An, in English, that tells you that,
645
00:33:15.129 --> 00:33:16.928
if people know what they're doing,
646
00:33:16.928 --> 00:33:21.095
the next word will start with a vowel, not a consonant.
647
00:33:22.077 --> 00:33:24.736
And guess what, only one of those options
648
00:33:24.736 --> 00:33:27.903
starts with a vowel, hence the answer.
649
00:33:29.437 --> 00:33:30.971
Okay, and you say to yourself,
650
00:33:30.971 --> 00:33:34.206
"Oh gee, you're just making up tricky examples.
651
00:33:34.206 --> 00:33:37.206
"No one would ever figure that out."
652
00:33:38.072 --> 00:33:40.822
Yeah, fourth graders can do this.
653
00:33:42.407 --> 00:33:45.189
So, I'm confident even college students can.
654
00:33:45.189 --> 00:33:49.022
Okay, all right, let's try a couple of others.
655
00:33:52.456 --> 00:33:54.456
How about the first one?
656
00:33:59.405 --> 00:34:00.572
Mitushito was,
657
00:34:01.447 --> 00:34:05.530
we have to go back to World War II time for this.
658
00:34:08.070 --> 00:34:09.570
Why do you suppose
659
00:34:10.847 --> 00:34:12.597
this item needs help?
660
00:34:13.530 --> 00:34:15.580
There's only one.
661
00:34:15.580 --> 00:34:16.810
There's only one people to answer.
662
00:34:16.810 --> 00:34:20.643
Okay, if you look at the options,
663
00:34:20.643 --> 00:34:23.476
unfortunately, only A matches with
664
00:34:26.421 --> 00:34:29.256
what we believe Mitushito sounds like,
665
00:34:29.256 --> 00:34:32.201
it sounds like a Japanese surname, does it not?
666
00:34:32.201 --> 00:34:34.696
It doesn't sound like a Russian pilot,
667
00:34:34.696 --> 00:34:36.169
it doesn't, or a Russian journalist
668
00:34:36.169 --> 00:34:38.132
or an English pilot, or a U.S. delegate,
669
00:34:38.132 --> 00:34:41.324
I mean, it, you know, maybe it could be.
670
00:34:41.324 --> 00:34:43.863
But in this case, it wasn't.
671
00:34:43.863 --> 00:34:47.508
So you, what happens is, the item is artificially easy
672
00:34:47.508 --> 00:34:49.309
because of that clue.
673
00:34:49.309 --> 00:34:51.392
How about the second one?
674
00:34:53.964 --> 00:34:56.008
The word twice corresponds with poly.
675
00:34:56.008 --> 00:34:57.925
Very good, very good.
676
00:34:58.812 --> 00:35:03.309
If you look at the second, the first sentence,
677
00:35:03.309 --> 00:35:05.903
"May result in twice the number of chromosomes,"
678
00:35:05.903 --> 00:35:09.653
and if you look at the answers, "Poly," many.
679
00:35:10.936 --> 00:35:12.603
There's a cue, okay.
680
00:35:16.869 --> 00:35:21.036
So, again, inadvertently, the answer's been given away.
681
00:35:23.000 --> 00:35:25.212
All right, let's try some debatable questions
682
00:35:25.212 --> 00:35:28.402
and tell me what you think about those.
683
00:35:28.402 --> 00:35:30.486
First one, who's the most important governor
684
00:35:30.486 --> 00:35:31.736
of Mississippi?
685
00:35:39.487 --> 00:35:42.545
Now what could possibly be wrong with that question?
686
00:35:42.545 --> 00:35:44.683
(speaking off microphone)
687
00:35:44.683 --> 00:35:46.016
It's an opinion.
688
00:35:47.013 --> 00:35:49.670
Now, if what you're measuring,
689
00:35:49.670 --> 00:35:51.385
if what you're intending to measure
690
00:35:51.385 --> 00:35:54.494
is can they take a stand and defend that stand
691
00:35:54.494 --> 00:35:57.033
with factual information, okay, great.
692
00:35:57.033 --> 00:35:59.533
But that's not what this says.
693
00:36:00.917 --> 00:36:03.206
It just says, who was the most important governor
694
00:36:03.206 --> 00:36:04.148
of Mississippi?
695
00:36:04.148 --> 00:36:06.565
So, someone says, eh, Allain.
696
00:36:11.304 --> 00:36:13.798
It doesn't say, according to who,
697
00:36:13.798 --> 00:36:15.715
or based on what, okay.
698
00:36:16.863 --> 00:36:18.530
So that fails there.
699
00:36:21.772 --> 00:36:24.915
How about the third one, who's the better poet?
700
00:36:24.915 --> 00:36:27.315
Keats or Wordsworth?
701
00:36:27.315 --> 00:36:29.818
(audience murmuring)
702
00:36:29.818 --> 00:36:32.068
(laughing)
703
00:36:32.969 --> 00:36:36.802
It depends on who the instructor likes, right?
704
00:36:38.138 --> 00:36:41.411
No, again, what that does is, as it stands,
705
00:36:41.411 --> 00:36:43.335
is just an opinion question,
706
00:36:43.335 --> 00:36:45.075
and if someone just puts down,
707
00:36:45.075 --> 00:36:49.242
"I think the better poet was Keats," end of discussion.
708
00:36:50.189 --> 00:36:51.522
Or, enough said.
709
00:36:52.435 --> 00:36:55.668
Okay, but that's, the instructor's probably looking
710
00:36:55.668 --> 00:36:57.304
for something better than that,
711
00:36:57.304 --> 00:36:59.715
but that's not what they're asking for.
712
00:36:59.715 --> 00:37:02.632
All right, how about the first one?
713
00:37:03.475 --> 00:37:06.283
"List the results of World War II.
714
00:37:06.283 --> 00:37:07.735
(audience laughter)
715
00:37:07.735 --> 00:37:11.318
"Use both sides of the paper if necessary."
716
00:37:12.646 --> 00:37:17.391
So when you see that comment, what do you know?
717
00:37:17.391 --> 00:37:19.558
You better write both sides of the paper.
718
00:37:19.558 --> 00:37:20.621
(audience laughter)
719
00:37:20.621 --> 00:37:21.454
Okay.
720
00:37:23.345 --> 00:37:25.491
They're giving you a cue there,
721
00:37:25.491 --> 00:37:26.923
you better pay attention to it.
722
00:37:26.923 --> 00:37:30.155
So, what are the results of World War II?
723
00:37:30.155 --> 00:37:33.347
Yeah, a lot of things, holy cats, where do you start,
724
00:37:33.347 --> 00:37:35.025
and where do you stop?
725
00:37:35.025 --> 00:37:38.176
So it becomes a matter of trying to guess,
726
00:37:38.176 --> 00:37:40.670
you know, develop your psychic ability
727
00:37:40.670 --> 00:37:44.920
to divine what the instructor's actually after.
728
00:37:44.920 --> 00:37:47.498
And hope that you have sprayed enough of that
729
00:37:47.498 --> 00:37:49.543
onto the, both sides of the paper
730
00:37:49.543 --> 00:37:51.293
to get credit for it.
731
00:37:52.323 --> 00:37:57.273
Okay, all right, let's look at a trick question or two.
732
00:37:57.273 --> 00:38:01.403
First one, John L. Tukey developed a display
733
00:38:01.403 --> 00:38:03.656
known as the box and whisker plot.
734
00:38:03.656 --> 00:38:04.823
True or false?
735
00:38:07.133 --> 00:38:09.050
There's one vote, true.
736
00:38:17.607 --> 00:38:19.407
I will tell you that Professor Tukey,
737
00:38:19.407 --> 00:38:23.309
first of Bell Labs and later in Princeton University,
738
00:38:23.309 --> 00:38:25.758
contributed a great many things
739
00:38:25.758 --> 00:38:27.276
to the world of data analysis,
740
00:38:27.276 --> 00:38:29.197
including the box and whisker plot.
741
00:38:29.197 --> 00:38:32.697
But the answer to this question is, false.
742
00:38:33.620 --> 00:38:35.370
Does anyone know why?
743
00:38:36.961 --> 00:38:39.549
(speaking off microphone)
744
00:38:39.549 --> 00:38:42.607
Because his middle initial was not L.
745
00:38:42.607 --> 00:38:46.257
(audience laughter)
746
00:38:46.257 --> 00:38:49.490
Now, imagine the love in the classroom
747
00:38:49.490 --> 00:38:52.823
when you hand that back to the students.
748
00:38:53.785 --> 00:38:57.177
Well, of course it's false, his middle initial wasn't L.
749
00:38:57.177 --> 00:38:59.602
Weren't you paying attention?
750
00:38:59.602 --> 00:39:02.484
(speaking off microphone)
751
00:39:02.484 --> 00:39:04.299
(laughs)
752
00:39:04.299 --> 00:39:05.882
All right, let's...
753
00:39:07.819 --> 00:39:10.271
Okay, let's take the second one.
754
00:39:10.271 --> 00:39:11.962
About President Lincoln, sir?
755
00:39:11.962 --> 00:39:13.136
False.
756
00:39:13.136 --> 00:39:16.508
Okay, one vote false, any others?
757
00:39:16.508 --> 00:39:17.341
False.
758
00:39:17.341 --> 00:39:18.174
False.
759
00:39:21.837 --> 00:39:22.778
False.
760
00:39:22.778 --> 00:39:23.611
False.
761
00:39:23.611 --> 00:39:24.444
False.
762
00:39:24.444 --> 00:39:25.557
False.
763
00:39:25.557 --> 00:39:30.076
As it turns out, the answer might be, "It depends."
764
00:39:30.076 --> 00:39:31.589
States in rebellion.
765
00:39:31.589 --> 00:39:33.307
States in rebellion, very good.
766
00:39:33.307 --> 00:39:35.597
Okay, first, there's two bits of information here
767
00:39:35.597 --> 00:39:38.256
that the student is gonna be forced to judge.
768
00:39:38.256 --> 00:39:41.241
Number One, was he in fact the 16th president?
769
00:39:41.241 --> 00:39:42.991
Yes, so far, so good.
770
00:39:43.894 --> 00:39:47.644
Second, did he abolish slavery in the states?
771
00:39:50.679 --> 00:39:53.346
And the answer is, some of them.
772
00:39:54.276 --> 00:39:57.341
Some of them, but not all of them.
773
00:39:57.341 --> 00:40:00.885
So, is that true, or is that false?
774
00:40:00.885 --> 00:40:03.137
Could the instructor have just been a little careless
775
00:40:03.137 --> 00:40:05.302
in writing that question?
776
00:40:05.302 --> 00:40:06.219
Well, yeah.
777
00:40:07.513 --> 00:40:10.013
So, which one gets you credit?
778
00:40:11.073 --> 00:40:12.544
Okay.
779
00:40:12.544 --> 00:40:16.428
That's a problem, and that makes it artificially difficult.
780
00:40:16.428 --> 00:40:18.511
Okay, let's try this one.
781
00:40:24.481 --> 00:40:27.648
Notice I haven't given you the answer.
782
00:40:28.505 --> 00:40:32.303
(audience murmuring)
783
00:40:32.303 --> 00:40:34.503
What's the answer?
784
00:40:34.503 --> 00:40:35.674
E.
785
00:40:35.674 --> 00:40:38.147
E, very good, none of the above.
786
00:40:38.147 --> 00:40:40.230
All right, let's say that
787
00:40:41.848 --> 00:40:46.425
2/3 of the students in a class pick E.
788
00:40:46.425 --> 00:40:49.767
Do they know the capital of South Dakota?
789
00:40:49.767 --> 00:40:52.100
(audience murmuring)
790
00:40:52.100 --> 00:40:53.517
Maybe, maybe not.
791
00:40:56.808 --> 00:40:58.133
They know what it's not.
792
00:40:58.133 --> 00:41:00.506
They know it's not these four.
793
00:41:00.506 --> 00:41:04.386
But they may or may not know it's Pierre.
794
00:41:04.386 --> 00:41:07.944
Okay, that's a problem with this kind of item.
795
00:41:07.944 --> 00:41:10.938
None of the above, or no 'taz, as I call 'em.
796
00:41:10.938 --> 00:41:14.282
Okay, all right, evaluating answers.
797
00:41:14.282 --> 00:41:16.321
You're gonna need to stop and think about
798
00:41:16.321 --> 00:41:20.488
before you get started, what is it I'm looking for,
799
00:41:21.345 --> 00:41:23.984
what am I gonna give credit for,
800
00:41:23.984 --> 00:41:26.027
do I give partial credit,
801
00:41:26.027 --> 00:41:29.912
and then try to stick to that set of guidelines
802
00:41:29.912 --> 00:41:31.220
the whole way through.
803
00:41:31.220 --> 00:41:33.098
I'll tell you what tends to happen.
804
00:41:33.098 --> 00:41:36.195
In a big stack, your standards as you start at the top
805
00:41:36.195 --> 00:41:38.897
of the stack are gonna change by the time
806
00:41:38.897 --> 00:41:41.814
you get to the bottom of the stack.
807
00:41:43.435 --> 00:41:45.903
And you're gonna, and unfortunately,
808
00:41:45.903 --> 00:41:47.990
that's not fair to the students, okay.
809
00:41:47.990 --> 00:41:51.429
Think about that from the students' perspective.
810
00:41:51.429 --> 00:41:55.195
So, you really need to, ahead of time,
811
00:41:55.195 --> 00:41:57.156
have what you think is a model answer.
812
00:41:57.156 --> 00:41:59.118
What are the features you're looking for,
813
00:41:59.118 --> 00:42:02.535
what are you gonna give credit for and...
814
00:42:03.770 --> 00:42:06.836
Take breaks, rub your eyes, stretch,
815
00:42:06.836 --> 00:42:09.491
get up, sit back down, okay.
816
00:42:09.491 --> 00:42:11.451
All right, so here's some examples,
817
00:42:11.451 --> 00:42:15.335
and let's talk about whether they deserve credit or not.
818
00:42:15.335 --> 00:42:18.002
Here's one from a physics class.
819
00:42:19.219 --> 00:42:22.651
Okay, how do you measure the height of a tall building
820
00:42:22.651 --> 00:42:24.151
using a barometer?
821
00:42:28.459 --> 00:42:31.542
(audience murmuring)
822
00:42:33.408 --> 00:42:35.776
(audience laughing)
823
00:42:35.776 --> 00:42:39.542
Okay, here's what the professor's looking for.
824
00:42:39.542 --> 00:42:42.854
Measure the air pressure at the base of the building,
825
00:42:42.854 --> 00:42:46.087
measure the air pressure at the top of the building.
826
00:42:46.087 --> 00:42:48.051
There's a simple conversion formula
827
00:42:48.051 --> 00:42:50.868
to figure the altitude from that.
828
00:42:50.868 --> 00:42:52.952
That's what the professor's looking for.
829
00:42:52.952 --> 00:42:57.119
Now, does he or she give credit for answers like these?
830
00:42:59.914 --> 00:43:02.914
(audience laughter)
831
00:43:07.226 --> 00:43:10.309
It would work, or how about this one?
832
00:43:12.194 --> 00:43:15.194
(audience laughter)
833
00:43:20.496 --> 00:43:21.663
It would work.
834
00:43:22.640 --> 00:43:24.973
Okay, or how about this one?
835
00:43:26.098 --> 00:43:29.098
(audience laughter)
836
00:43:31.822 --> 00:43:33.405
It would work, too.
837
00:43:36.611 --> 00:43:37.944
I love this one.
838
00:43:43.316 --> 00:43:47.118
Use proportions, measure the shadows, use proportions.
839
00:43:47.118 --> 00:43:49.901
It works, unless there's a cloud over the sky.
840
00:43:49.901 --> 00:43:52.318
Okay, and last but not least.
841
00:43:53.256 --> 00:43:56.256
(audience laughter)
842
00:44:02.729 --> 00:44:05.361
I will give you this fine barometer.
843
00:44:05.361 --> 00:44:06.587
(audience laughter)
844
00:44:06.587 --> 00:44:08.877
Okay, now do you give credit for those,
845
00:44:08.877 --> 00:44:12.517
because any of those could work, okay?
846
00:44:12.517 --> 00:44:16.596
But that's the difference between posing the task
847
00:44:16.596 --> 00:44:19.948
in a way that communicates what you're after,
848
00:44:19.948 --> 00:44:21.951
versus just throwing it out there,
849
00:44:21.951 --> 00:44:25.465
and letting the students' imagination take flight.
850
00:44:25.465 --> 00:44:28.382
Okay, let's try a couple of others.
851
00:44:29.826 --> 00:44:32.826
(audience laughter)
852
00:44:35.536 --> 00:44:38.192
Where was the Declaration signed, at the bot,
853
00:44:38.192 --> 00:44:40.253
well, where else would you sign it, of course!
854
00:44:40.253 --> 00:44:41.086
Okay.
855
00:44:42.395 --> 00:44:45.395
(audience laughter)
856
00:44:47.356 --> 00:44:52.019
Hard water, ice, it doesn't get much harder than that.
857
00:44:52.019 --> 00:44:53.102
Let's expand.
858
00:44:55.293 --> 00:44:58.293
(audience laughter)
859
00:45:03.923 --> 00:45:08.218
Let's expand the function, and sure enough, he did.
860
00:45:08.218 --> 00:45:09.051
Okay, or.
861
00:45:11.861 --> 00:45:14.861
(audience laughter)
862
00:45:16.118 --> 00:45:20.453
You know, does it continue to go after it comes to rest?
863
00:45:20.453 --> 00:45:22.826
And the answer is no, 'cause he drew an elephant,
864
00:45:22.826 --> 00:45:27.187
and he says there's an elephant in the way.
865
00:45:27.187 --> 00:45:30.187
(audience laughter)
866
00:45:31.106 --> 00:45:31.939
Find x.
867
00:45:33.269 --> 00:45:36.635
Well, give him credit for map skills, there it is.
868
00:45:36.635 --> 00:45:38.520
(audience laughter)
869
00:45:38.520 --> 00:45:42.241
Okay, with this one, what would fix that?
870
00:45:42.241 --> 00:45:44.165
Value of x.
871
00:45:44.165 --> 00:45:46.248
What is the value of x?
872
00:45:47.268 --> 00:45:50.576
Okay, or solve for the value of x, not find x.
873
00:45:50.576 --> 00:45:51.803
For goodness sake.
874
00:45:51.803 --> 00:45:55.567
Okay, all right, or here's some examples,
875
00:45:55.567 --> 00:45:59.400
Darwin was the author of organ of the species.
876
00:46:01.044 --> 00:46:04.377
Not origin, do you give credit for that?
877
00:46:06.080 --> 00:46:09.010
(audience murmuring)
878
00:46:09.010 --> 00:46:12.927
The dodo is a bird that's almost decent by now.
879
00:46:15.509 --> 00:46:17.512
The next to the last is my personal favorite.
880
00:46:17.512 --> 00:46:19.929
If you smell an odorless gas.
881
00:46:21.094 --> 00:46:22.975
(audience laughter)
882
00:46:22.975 --> 00:46:25.475
It's probably carbon monoxide.
883
00:46:28.814 --> 00:46:33.356
I mean, they've got some of the stuff there, right?
884
00:46:33.356 --> 00:46:35.606
It's odorless, sure enough.
885
00:46:36.586 --> 00:46:38.904
But you're never gonna, anyway.
886
00:46:38.904 --> 00:46:42.833
All right, there's other kinds of ways of evaluating
887
00:46:42.833 --> 00:46:45.534
besides tests, besides questions in class,
888
00:46:45.534 --> 00:46:47.784
besides take home projects.
889
00:46:49.501 --> 00:46:52.896
And you should give thought to those as options.
890
00:46:52.896 --> 00:46:55.594
You should also think about your grading philosophy.
891
00:46:55.594 --> 00:46:57.928
If you're gonna be giving marks that are gonna
892
00:46:57.928 --> 00:47:02.837
walk through the students' transcripts forever and a day,
893
00:47:02.837 --> 00:47:06.758
it's a big deal, so what goes into those?
894
00:47:06.758 --> 00:47:10.272
What doesn't go into those, okay?
895
00:47:10.272 --> 00:47:12.558
Think about those.
896
00:47:12.558 --> 00:47:14.890
What about borderline cases?
897
00:47:14.890 --> 00:47:19.057
You know, remember what we said about grading on the curve.
898
00:47:20.449 --> 00:47:23.720
If you ask experts what a grade should do,
899
00:47:23.720 --> 00:47:25.727
they'll tell you, it should do one thing.
900
00:47:25.727 --> 00:47:28.477
It should communicate achievement
901
00:47:29.506 --> 00:47:32.173
or proficiency over the subject,
902
00:47:34.203 --> 00:47:35.922
over the skills of interest.
903
00:47:35.922 --> 00:47:39.106
Nothing else, it's not enthusiasm,
904
00:47:39.106 --> 00:47:40.988
it's not motivation.
905
00:47:40.988 --> 00:47:43.478
It's not big smile on your face.
906
00:47:43.478 --> 00:47:44.886
Personally, as an instructor,
907
00:47:44.886 --> 00:47:49.053
I love those attributes in students, okay, but anyway.
908
00:47:50.130 --> 00:47:51.644
Think about that.
909
00:47:51.644 --> 00:47:55.567
How do you combine scores into grades?
910
00:47:55.567 --> 00:47:58.550
And so forth, cheating, it's gonna happen.
911
00:47:58.550 --> 00:48:00.349
It's gonna happen at about the same rate
912
00:48:00.349 --> 00:48:04.266
for online classes as for face to face classes.
913
00:48:05.824 --> 00:48:07.625
So just be aware of it.
914
00:48:07.625 --> 00:48:10.625
It's gonna happen, MSU has a policy,
915
00:48:11.545 --> 00:48:14.741
all the recommendations of folks who study this
916
00:48:14.741 --> 00:48:17.536
start with, "Let the students know
917
00:48:17.536 --> 00:48:19.540
"what your expectations are."
918
00:48:19.540 --> 00:48:22.115
That's what they wanna know.
919
00:48:22.115 --> 00:48:25.220
What, to this instructor, is plagiarism?
920
00:48:25.220 --> 00:48:28.407
If I borrow from a paper I wrote last semester,
921
00:48:28.407 --> 00:48:30.574
is that plagiarism?
922
00:48:30.574 --> 00:48:33.251
To a lot of instructors, the answer is, "Yes."
923
00:48:33.251 --> 00:48:37.317
To other instructors, the answer is, "No, heck use it."
924
00:48:37.317 --> 00:48:41.654
So, think about what's gonna be puzzling your students
925
00:48:41.654 --> 00:48:43.592
at the beginning of the semester,
926
00:48:43.592 --> 00:48:46.210
those are probably the things that should be addressed
927
00:48:46.210 --> 00:48:47.888
at the beginning of the semester.
928
00:48:47.888 --> 00:48:51.445
Okay, why do people rationalize cheating?
929
00:48:51.445 --> 00:48:55.445
How can they look at the mirror when they do it?
930
00:48:57.020 --> 00:49:00.853
Well, here are the top reasons that they give.
931
00:49:02.994 --> 00:49:06.515
It's easy to do, I was too pressed,
932
00:49:06.515 --> 00:49:10.398
a friend needed help, so of course I helped him or her.
933
00:49:10.398 --> 00:49:12.148
I have to pass, okay.
934
00:49:13.666 --> 00:49:17.631
Everybody else does it, that's the Niagara Falls argument.
935
00:49:17.631 --> 00:49:20.946
No one else cares if I cheat, uh oh.
936
00:49:20.946 --> 00:49:23.071
That's expectations.
937
00:49:23.071 --> 00:49:27.238
Okay, somebody stole my work, well that can happen.
938
00:49:28.183 --> 00:49:31.293
The course is too hard, or the teacher's just unfair.
939
00:49:31.293 --> 00:49:34.357
Unfortunately, that can happen, too.
940
00:49:34.357 --> 00:49:37.857
But the worst one, to me, is the last one.
941
00:49:39.613 --> 00:49:42.481
Think about it, if we can't even sell our course
942
00:49:42.481 --> 00:49:46.564
as being worthwhile, maybe we need to rethink it.
943
00:49:47.673 --> 00:49:50.739
So, yeah, we should all be advocates for
944
00:49:50.739 --> 00:49:52.131
what we're all about.
945
00:49:52.131 --> 00:49:53.847
So what do you do to reduce cheating?
946
00:49:53.847 --> 00:49:56.586
Well, if you stop and think about those rationalizations,
947
00:49:56.586 --> 00:50:00.548
then some of the proactive steps become fairly obvious.
948
00:50:00.548 --> 00:50:03.715
Okay, try to build good relationships.
949
00:50:04.684 --> 00:50:07.423
From the get-go, make it clear what to you
950
00:50:07.423 --> 00:50:10.671
is inappropriate behavior versus appropriate behavior.
951
00:50:10.671 --> 00:50:13.777
Okay, students'll feel happier, knowing the difference,
952
00:50:13.777 --> 00:50:15.535
you'll be happier, knowing the difference.
953
00:50:15.535 --> 00:50:17.843
Okay, and so forth.
954
00:50:17.843 --> 00:50:21.206
By the way, all of these slides, Ms. McMullan,
955
00:50:21.206 --> 00:50:22.840
I think is gonna post them in a way
956
00:50:22.840 --> 00:50:24.313
that if you wanna refer to them later,
957
00:50:24.313 --> 00:50:27.504
you are certainly welcome to do so.
958
00:50:27.504 --> 00:50:29.406
Appeals will happen.
959
00:50:29.406 --> 00:50:31.115
You just got through, I think,
960
00:50:31.115 --> 00:50:33.365
a presentation about a syllabus.
961
00:50:33.365 --> 00:50:36.282
The simplest way to defend yourself
962
00:50:37.330 --> 00:50:40.269
is follow the guidelines you put in your syllabus.
963
00:50:40.269 --> 00:50:43.577
Don't change 'em up midway in the course.
964
00:50:43.577 --> 00:50:45.578
Don't change 'em for one student because
965
00:50:45.578 --> 00:50:47.992
it turned out to be, you know, a situation
966
00:50:47.992 --> 00:50:52.159
you didn't anticipate, stick to the syllabus guidelines.
967
00:50:53.396 --> 00:50:57.773
Okay, document your discussions, know MSU's policies
968
00:50:57.773 --> 00:51:01.537
about appeals, about the honor code, and so forth,
969
00:51:01.537 --> 00:51:03.952
that will really help.
970
00:51:03.952 --> 00:51:06.241
Don't forget about these kinds of things in a syllabus.
971
00:51:06.241 --> 00:51:08.205
What are you gonna do about late assignments,
972
00:51:08.205 --> 00:51:09.883
what are you gonna do about test dates,
973
00:51:09.883 --> 00:51:12.579
if there really is a legitimate excuse
974
00:51:12.579 --> 00:51:15.405
for a student to have to miss test.
975
00:51:15.405 --> 00:51:18.349
Some instructors handle that with drop grades.
976
00:51:18.349 --> 00:51:21.825
Others will give make up opportunities.
977
00:51:21.825 --> 00:51:24.408
Okay, some'll just say, "Sorry,
978
00:51:25.243 --> 00:51:28.792
"I said be there or be square and you weren't."
979
00:51:28.792 --> 00:51:30.839
But you need to stop and think about the impact
980
00:51:30.839 --> 00:51:32.518
that would have.
981
00:51:32.518 --> 00:51:35.338
Here are some examples, if you want to look 'em up online,
982
00:51:35.338 --> 00:51:38.202
about a syllabus, and decide for yourself,
983
00:51:38.202 --> 00:51:41.270
did they do a good job, or oh my goodness,
984
00:51:41.270 --> 00:51:43.316
would I want to stay the heck away from that class?
985
00:51:43.316 --> 00:51:44.581
Based on the syllabus?
986
00:51:44.581 --> 00:51:46.259
You're certainly welcome to do so.
987
00:51:46.259 --> 00:51:47.842
Here is an example.
988
00:51:49.001 --> 00:51:52.558
I'll let you ponder that at your leisure.
989
00:51:52.558 --> 00:51:54.191
If you're gonna count attendance,
990
00:51:54.191 --> 00:51:55.829
make it a carrot, not a stick.
991
00:51:55.829 --> 00:51:57.579
That's my suggestion.
992
00:51:59.676 --> 00:52:03.426
By the way, MSU has, in the last three years,
993
00:52:05.160 --> 00:52:10.067
really honed in on attendance as a big darn deal.
994
00:52:10.067 --> 00:52:14.609
Especially among underclassmen, freshmen and sophomores.
995
00:52:14.609 --> 00:52:17.471
Why do you think that is?
996
00:52:17.471 --> 00:52:19.186
'Cause they're new to the game.
997
00:52:19.186 --> 00:52:20.019
Say it again?
998
00:52:20.019 --> 00:52:20.852
They're new to the game.
999
00:52:20.852 --> 00:52:23.518
They are absolutely new to the game, number one.
1000
00:52:23.518 --> 00:52:27.603
And number two, it turns out that attendance becomes
1001
00:52:27.603 --> 00:52:31.353
a good predictor of performance in the class.
1002
00:52:32.557 --> 00:52:35.502
It's an inverse relationship.
1003
00:52:35.502 --> 00:52:38.169
Higher attendance, lower grades.
1004
00:52:39.657 --> 00:52:41.389
(laughs)
1005
00:52:41.389 --> 00:52:44.086
Higher absences, lower grades.
1006
00:52:44.086 --> 00:52:46.377
Fewer absences, higher grades, okay.
1007
00:52:46.377 --> 00:52:47.976
It's an inverse relationship.
1008
00:52:47.976 --> 00:52:50.726
So think about that as important.
1009
00:52:52.768 --> 00:52:55.271
Oh, this was a one that really, really,
1010
00:52:55.271 --> 00:52:57.479
I found interesting.
1011
00:52:57.479 --> 00:53:00.339
An entire class, you'd fail the entire class
1012
00:53:00.339 --> 00:53:03.901
if you didn't show up for the final exam.
1013
00:53:03.901 --> 00:53:06.568
Didn't matter how you performed.
1014
00:53:08.030 --> 00:53:10.613
Just stop and think about that.
1015
00:53:11.628 --> 00:53:12.961
Okay, reporting.
1016
00:53:15.943 --> 00:53:17.889
Grades are protected by FERPA.
1017
00:53:17.889 --> 00:53:19.222
You're not supposed to divulge 'em
1018
00:53:19.222 --> 00:53:21.834
or disclose 'em to other people, you knew that.
1019
00:53:21.834 --> 00:53:23.997
Okay, and you wouldn't want your grades,
1020
00:53:23.997 --> 00:53:25.348
if you were taking a class, you wouldn't want
1021
00:53:25.348 --> 00:53:28.330
your grades posted in a way that others could see.
1022
00:53:28.330 --> 00:53:31.413
So, you know, don't do it for others.
1023
00:53:32.912 --> 00:53:37.214
The mycourses system is perfectly fine for that,
1024
00:53:37.214 --> 00:53:39.131
and it works very well.
1025
00:53:41.300 --> 00:53:44.656
There's lots of resources you can turn to for advice,
1026
00:53:44.656 --> 00:53:48.656
for examples, some good, some bad, and so forth.
1027
00:53:49.765 --> 00:53:52.352
The one thing I would urge you to do is,
1028
00:53:52.352 --> 00:53:56.280
if you're using like test banks for a textbook,
1029
00:53:56.280 --> 00:53:59.922
pay real close attention to those items
1030
00:53:59.922 --> 00:54:02.167
because lots of times, those are written by people
1031
00:54:02.167 --> 00:54:04.578
that had nothing to do with the class
1032
00:54:04.578 --> 00:54:07.524
or the text, and they were hired guns
1033
00:54:07.524 --> 00:54:10.185
and told, "Create a test bank."
1034
00:54:10.185 --> 00:54:13.130
And so, as a result, items may be in the test bank
1035
00:54:13.130 --> 00:54:15.666
that don't even pertain to the text.
1036
00:54:15.666 --> 00:54:17.301
And that's a tough one to explain away,
1037
00:54:17.301 --> 00:54:20.285
when the students challenge you.
1038
00:54:20.285 --> 00:54:24.035
Okay, and then finally, remember this prayer.
1039
00:54:32.718 --> 00:54:35.801
(audience murmuring)
1040
00:54:44.719 --> 00:54:46.356
"If I should die before I wake,
1041
00:54:46.356 --> 00:54:49.626
"that's one less test I'll have to take."
1042
00:54:49.626 --> 00:54:51.714
Anyway, some resources that you can look at,
1043
00:54:51.714 --> 00:54:54.128
if you're interested, resources on test construction,
1044
00:54:54.128 --> 00:54:55.928
you can look at if you're interested.
1045
00:54:55.928 --> 00:55:00.182
Some course evaluation comments that are kind of funny.
1046
00:55:00.182 --> 00:55:02.024
The text makes a satisfying thud
1047
00:55:02.024 --> 00:55:04.806
when I drop it on the floor.
1048
00:55:04.806 --> 00:55:06.889
Some cheating references.
1049
00:55:08.326 --> 00:55:09.326
And at the end of it all,
1050
00:55:09.326 --> 00:55:12.023
I hope you have a terrific semester, thanks.
1051
00:55:12.023 --> 00:55:14.273
(applause)