Advance Techiniques for Testing Higher Level Thinking
  1. TO ACCESS HIGHER LEVELS IN BLOOM'S TAXONOMY

    2. DON'T CONFUSE "HIGHER THINKING SKILLS" WITH "DIFFICULTY" OR COMPLICATED

    • I'll use some Grade 3 examples to prove this...

      • use data or pictures to go beyond recall

      • use multiple choice to get at skill questions

    Objective: "read and interpret a chart" -- HOW WOULD YOU DO IT? Objective: "create a chart"

    Okay, harder, how do you get them to create a chart in multiple-choice?

    Objective: "cause and effect": why settlers chose to settle where they did

      Map Use
    • Use &#34Before"and &#34After"pictures
    • similarly, map reading:
    • Map Example
    • ALWAYS MAKE SURE information on question is on same page or facing page, no page flipping allowed!
    • pictures also make examination more appealing visually
      • straight text is pretty intimidating --> grade 3 exam is appealing
      • but DO NOT include pictures JUST for sake of pictures, as --takes students time to examine and relate to questions --and if not related to question will get hopelessly confused
          --> "What am I supposed to get from picture/cartoon etc.?!"
      • figure you will put picture at start of test to relax kids, but has opposite effect for that one kid who panics and goes, I &#34can'even figure out what she'asking!"
      • if must put cartoon, put on cover with instructions....
  3. FAMILIES OF ITEMS
      INQUIRY PROCESS: not an example but rather a flow chart or figure Do we want it in here Y or N?
    • Whole process, flow of ideas
    • This is the curriculum with which I was presented on joining the newly formed Student Evaluation Branch, with the mandate of building tests to the curriculum
    • Curriculum Branch (for various reasons) wanted to block the tests and so said would only accept if we could cope with process, not just individual factoids
    • HOW WOULD YOU SIMULATE THIS?
    • But as you remember, multiple choice exams must be COMPLETELY independent of each other
      • answer to one mustn't give away answer to another
      • answer to one cannot depend on getting answer to previous one right
    • I knew it couldn'be done, because rule I gave you is that all questions must be independent of each other
    • But we didn'tell our teacher item writers it couldn'be done so they invented a way to do it....
      -The &#34but"was, several questions can share the same data
      Example:
    • BUT no more than 5 or 6 questions to a family
        -- too much on one topic, start to lose the advantage of being able to survey broad section of curriculum -- too much on one data set, student is murdered if doesn't "get it"
      (A) Simulates going through inquiry process, step by step ((REFER BACK TO OVERHEAD OF INQUIRY PROCESS) (B) If you are going to make a student read all that data to answer question 49, have to make it worth his while by asking additional questions on same data
        * Question 50 addresses advanced skill of organizing a good research question
      EACH QUESTION IS WORTH ONE MARK SO SHOULD BE ROUGHLY EQUAL IN TIME
      • SO question 50 asks about Source III
        • don't have to get right answer to #49 to get #50
        • but have already read SOURCE III so faster this time so makes up for time invested in #49
        * Question 53 -- highest of Bloom'taxonomy: Evaluation
      • notice that we provide SPECIFIC CRITERIA on which it is to be evaluated, "mainly support the principle of the League of Nations"
      • notice that we can ask opinion question because it's not their opinion
      • but be cautious of stereotyping "a conservative"
      ExampleOVERHEAD: GRADE 9 FAMILY -one of mine, but an early one -- overly complex with X & Y in Source I
    USE ANALOGIES TO GET AT UNDERSTANDING
    • Objective is to know stages of industrial revolution, definition of mechanization, automation, etc.
    • Problem is student can just memorize definitions without understanding
    • Paraphrase is OK, but then risk introducing error or ambiguity in your definition
    • even then, may not understand relationship between concepts
      Example
      -must make sure that in analogy questions relationship of last two is not sufficient to give it away
      Example
    HAVE STUDENTS ANALYZE IMPLICIT ASSUMPTIONS IN MATERIALS THEY'VE COVERED
    • asking them to recall things the text made explicit is just recall, does not test understanding
    • but asking them to identify assumptions and analyze criteria does:
      Example
    PROVIDE FOR A CONDITION CONTRARY TO FACT -- "HYPOTHESIZE"
      Example