Keying The Item

  1. Make certain that intended answer is only correct or clearly the best
      -pretty obvious, but some kids can think their way around anything

    Example

  2. Vary the distribution of the keyed item in random manner
      -should have same number of a's, b's, c's and d's as correct answer

    -most kids know that if in doubt you choose "c"

      -because we all tend to want to "hide" keyed answer in middle

      -because after you've gone to all that work of thinking up wrong answers want to make sure the little monsters read all of them

    -this is very seductive -- subconscious even when at the test branch so we used:

      -random tables to force yourself

      -or dice

      -or use pyramiding

  3. Distracters should be arranged in ascending or descending order.
  4. Example
    • numbers in order
    • dates in chronological order
    • lines from a passage in order they appear in passage
    • alphabetize
    • if nothing else, others in length (pyramiding) --not vital, last choice
  5. Make sure that every item is independent of every other item
    • that answer to one question doesn't tip them off on another one
    • or that wrong answers confuses them into choosing wrong answer on another question --> though that's a bit harder to spot
    • answer to one cannot depend on getting answer to previous one right
        -e.g.., can't break 4 step math problem into four questions because error on first step costs four marks, even if he got last three steps right
    • ALL of which the department found away around---> we developed families of items --> bit more advanced....
        but you can simulate 4 step math
          step one, what is the answer step two: "another student doing this experiment got 148 grams. Had your answer been 148, the next step would be..."
        this was an Alberta invention, now used everywhere
  6. Each item should be worth the same number of marks:
      • no surprises means no fair telling student #37 was worth 12 marks
      • confusing
      • on what basis would you decide? Discriminating against students who don't do well on that one concept
      • --> if you think concept is twice as important as another, you ask twice as many questions about it
    • since each item is worth the same, you have to try to make them all about equal work --> in terms of time to do them, not difficulty
      • can't spend two hours reading booklet to answer one question....