A number of years ago now, I was working as a test specialist when my boss decided to switch me over to the computer section. Since I had no computer training, and since all our computer programs were written in FORTRAN, I signed up for a university level correspondence course on FORTRAN programming. By the time the course materials had arrived in the mail two weeks later, however, my boss had changed his mind and switched me back to writing tests. I therefore had no further need to learn FORTRAN, and I pushed the unopened box of course materials under my bed and forgot about it.

Six months later I received a notice in the mail saying the examination for the computer course would be at such and such location in a week's time. And I thought, "Well, I've already paid for the course, why not take the test?" Not, you understand, that I had ever opened the box or looked at any of the course materials. I just wondered if I could pass the test on my knowledge of test design.

The examination, if I recall, was 70% multiple-choice and 30% written response. I had no hope of answering the written response part, since I had absolutely no idea how to write a FORTRAN program. I did attempt a line or two of code based on examples from the multiple-choice part of the test, but since I had no idea what it was I was copying, I doubt it made much sense.

The multiple-choice questions, however, were a different matter. As a test specialist, I was able to examine each question for the tiny flaws that gave the answer away. Examining the alternatives for the odd one out, or for one answer that was significantly longer than the others, and so on, I gave it my best college try, even though I often had no idea what the question was asking, let alone what the correct answer might be.

Three weeks later I received a letter saying I had obtained 70% on the test; which considering I had left the written response essentially blank, means I must have aced the multiple-choice. Not bad for a topic I literally knew nothing about!

[I subsequently wrote the institution in question pointing out this weakness in their tests, and sometime later they hired me to give a one day workshop on the do's and don'ts of multiple-choice item writing. There is, therefore, no use asking me how to register for that course; these techniques won't work to pass that test again!]

This was probably an extreme case; and, to be fair, such tests are not intended for people who make their living designing multiple-choice tests. Most people would not have found the flaws quite so obvious.

More recently, I sat down with a copy of the Alberta Grade 12 Mathematics Diploma Examination to see what I could get using these test-wise tricks. In this case, I only managed 23%, slightly less than one would expect by pure chance. This test had been written by experts and it was impossible for me to get through it without actually knowing the course content.

Most classroom teachers' tests are somewhere in between these two extremes.