Aug 312017

Last week I gave all my classes a test – 10 classes in all. They were all written tests which I had to mark by hand. That’s a lot of work, but it’s part of the job. I can actually get my assistants to do the marking, but I’m not going to do that. I take my teaching seriously, and testing is very important. I hate marking, and my students hate tests. So why do I do it? I’ve hated taking tests all my life. Couldn’t I just find an easier way out? Tempting – but no.

At the start of all my undergraduate classes I used to give my students a little pep talk about my methods, especially about testing. I used to tell them that I hate testing and marking, but I don’t do it because I am a mean, nasty, horrible person who enjoys inflicting pain. I do it because it is intrinsically worthwhile – but only if done right. Some ways of testing are worse than useless, they are counterproductive; some ways are invaluable to the learning process. The trick is knowing which is which.

Schools (final exams) at Oxford are probably the most brutal exams I have ever sat for. I had to do 13 exams, each 3 hours long, over the course of a week (with Sunday off) – one in the morning, and one in the afternoon. They tested everything I had studied for 3 years. Unlike the US system where you take a course and get tested at the end of it, at Oxford there are no tests per subject as you go along; just one set of final exams on everything at the very end of your undergraduate years. In your final term you have no tutorials or lectures. You simply look back over every essay you’ve written, and all of your notes, and cram as much in as possible, over a 5-week period, ready to regurgitate it all on paper over a grueling week, 3 hours at a pop. It didn’t help that we had to sit the exams dressed in black suit, white shirt with white bow tie, and academic gown (carrying a mortar board), but that was a minor inconvenience. The point was that for a brief moment in time you had the whole subject matter at your fingertips.

Having everything to hand is an important part of the intellectual process for me. By “to hand” and “at your fingertips” does not mean being able to look it up quickly via the internet or whatever. Everyone has that capacity. I mean having the information IN YOUR HEAD. That ability is vital to the thinking process. Testing in the right way encourages that style of thinking: get students to internalize as much as possible all at once, not so that they can simply regurgitate it, but so they can put it to practical use.

Some tests are the absolute opposite of this style, notably the standardized tests for entry to undergraduate and graduate programs of learning. The classic SAT and GRE exams used by the vast majority of universities in the US had 2 sections – verbal and mathematics – and now they have logical reasoning and essay sections. Supposedly the latter two make the tests more diagnostically useful. The fact is that repeated analysis of SAT scores has shown that the test cannot even accurately predict success or failure in the FIRST year of college, let alone after four years. They are worthless as predictors. Universities use them for only one reason: they are swamped with applications and so they need a simple measure to select among the applicants. They set parameters among the test scores and separate applicants into those they will accept and those they reject. Then they use other measures such as essays or interviews for the fine tuning. So, they start off with a worthless test that is not remotely related to the subject matter the student is applying to study, and which has no predictive power concerning success, and use that as their arbiter for who gets in and who doesn’t simply because it gives them a numerical score they can use to compare applicants. An actual test is too much work.

When I applied to Oxford I had to first sit five exams, three hours each, on the subject I was applying to study, and was marked by the relevant tutors at the college I was applying to. When I cleared that hurdle, I went to the college for an hour-long interview by the tutors. That is a reasonable test. When I applied to the doctoral program in anthropology at the University of North Carolina I was accepted on my written work – very commendable – but scholarships were assigned solely on GRE results. My scores were respectable, but not great, because I sat them without knowing what they were for. I was told to take them, and I did. I had never taken a multiple-choice test before in my life, and just did the best I could. I did not get a scholarship and spent 5 years studying at my own expense. Two of my classmates did get full scholarships, however, because they scored very well on the GRE. (They also got major funding from the National Science Foundation for their fieldwork. I had to take out a bank loan for mine.) The upshot? Neither of them finished the Ph.D. – one of them because he couldn’t string together a coherent sentence in writing !!! So much for standardized testing. Standardized testing tests only the ability to take standardized tests.

Standardized tests are popular because they can be machine graded and because they provide a simple numerical score that can be used to compare one student (or candidate) with another. The degree to which standardized tests provide meaningful results depends on the skill of the tester in many ways. First, there is the nature of the questions themselves. On the GRE I had to answer such questions as “ball is to bat as (something) is to (something)” with 5 choices of answers. How exactly will that question tell you whether I will be a good anthropologist or not? Second, if you look at the array of answers on a particular question you can decide whether your questions are too hard or too easy. If 100% of people being tested get the right answer the question is too easy; if the answers are spread more or less evenly over all the possible answers the people being tested are guessing, therefore the question is too hard. Professional test-writing companies spend a lot of time and money researching their testing methods. Your average university professor generally has no clue about this stuff when making up a multiple-choice test.

I used to use multiple-choice tests in very large lecture classes to test certain base level knowledge. I think this is fair as long as it’s not the only method of testing students. When I first trained as an emergency paramedic I had to take a grueling standardized test to certify. It tested basic knowledge in a host of medical areas. But it was only ONE part of a series of tests. I also had to show live examiners that I knew how to do a variety of things such as insert an IV line, intubate a patient, identify a fibrillating heart from an EKG and defibrillate, and splint a broken bone. You were allowed several attempts at each but if you failed any station you had to redo the test at a later date after more training. Furthermore, I had to work as a trainee under a supervisor on ambulances and take several 12-hour rotations in an emergency room, all of which were assessed and graded. So, the standardized test was part of a mix of tests. One hopes that the result is competent paramedics.

Bottom line: testing is an essential component of teaching/learning, but to be useful you have to know what you are doing, and too few teachers do.

Sorry, the comment form is closed at this time.