Take 'top' pile, count number of students who responded to each alternative
- fast way is simply to sort piles into "A", "B", "C", "D" // or true/false or type of error you get for short answer, fill-in-the-blank
OR set up on spread sheet if you're familiar with computers
ITEM ANALYSIS FORM TEACHER CONSTRUCTED TESTS
CLASS SIZE = 30
ITEM UPPER LOWER DIFFERENCE D TOTAL DIFFICULTY
1. A 0
*B 4
C 1
D 1
O
*=Keyed Answer
|
ITEM ANALYSIS FORM TEACHER CONSTRUCTED TESTS
CLASS SIZE = 30
ITEM UPPER LOWER DIFFERENCE D TOTAL DIFFICULTY
1. A 0
*B 4 2
C 1
D 1
O
*=Keyed Answer
|
- this is the time consuming part --> but not that bad, can do it while watching TV, because you're just sorting piles
THREE POSSIBLE SHORT CUTS HERE (STEP 4)
(A) If you have a large sample of around 100 or more, you can cut down
the sample you work with
- take top 27% (27 out of 100); bottom 27% (so only dealing with 54, not all 100)
- put middle 46 aside for the moment
- larger the sample, more accurate, but have to trade off against labour; using top 1/3 or so is probably good enough by the time you get to 100; --27% magic figure statisticians tell us to use
- I'd use halves at 30, but you could just use a sample of top 10 and bottom 10 if you're pressed for time
- but it means a single student changes stats by 10%
- trading off speed for accuracy...
- but I'd rather have you doing ten and ten than nothing
(B) Second short cut, if you have access to photocopier (budgets)
- photocopy answer sheets, cut off identifying info
(can't use if handwriting is distinctive)
- colour code high and low groups --> dab of marker pen color
- distribute randomly to students in your class so they don't know whose answer sheet they have
- get them to raise their hands
- for #6, how many have "A" on blue sheet?
how many have "B"; how many "C"
- for #6, how many have "A" on red sheet....
- some reservations because they can screw you up if they don't take it seriously
- another version of this would be to hire kid who cuts your lawn to do the counting, provided you've removed all identifying information
- I actually did this for a bunch of teachers at one high school in Edmonton when I was in university for pocket money
(C) Third shortcut, IF you can't use separate answer sheet, sometimes faster to type than to sort
SAMPLE OF TYPING FORMAT
FOR ITEM ANALYSIS
ITEM # 1 2 3 4 5 6 7 8 9 10
KEY T F T F T A D C A B
STUDENT
Kay T T T F F A D D A C
Jane T T T F T A D C A D
John F F T F T A D C A B
|
Subtract the number of students in lower group who got question right from number of high group students who got it right
- quite possible to get a negative number
ITEM ANALYSIS FORM TEACHER CONSTRUCTED TESTS
CLASS SIZE = 30
ITEM UPPER LOWER DIFFERENCE D TOTAL DIFFICULTY
1. A 0
*B 4 2 2
C 1
D 1
O
*=Keyed Answer
|
Divide the difference by number of students in upper or lower group
- in this case, divide by 15
- this gives you the "discrimination index" (D)
ITEM ANALYSIS FORM TEACHER CONSTRUCTED TESTS
CLASS SIZE = 30
ITEM UPPER LOWER DIFFERENCE D TOTAL DIFFICULTY
1. A 0
*B 4 2 2 0.333
C 1
D 1
O
*=Keyed Answer
|
Total number who got it right
ITEM ANALYSIS FORM TEACHER CONSTRUCTED TESTS
CLASS SIZE = 30
ITEM UPPER LOWER DIFFERENCE D TOTAL DIFFICULTY
1. A 0
*B 4 2 2 0.333 6
C 1
D 1
O
*=Keyed Answer
|
If you have a large class and were only using the 1/3 sample for top and bottom groups, then you have to NOW count number of middle group who got each question right (not each alternative this time, just right answers)
Sample Form Class Size= 100.
if class of 30, upper and lower half, no other column here
Divide total by total number of students
- difficulty = (proportion who got it right (p) )
ITEM ANALYSIS FORM TEACHER CONSTRUCTED TESTS
CLASS SIZE = 30
ITEM UPPER LOWER DIFFERENCE D TOTAL DIFFICULTY
1. A 0
*B 4 2 2 0.333 6 .42
C 1
D 1
O
*=Keyed Answer
|
You will NOTE the complete lack of complicated statistics --> counting, adding, dividing --> no tricky formulas required for this
- not going to worry about corrected point biserials etc.
- one of the advantages of using fixed number of alternatives