Visible thinking and complex examination questions

No students were permanently harmed during this lesson.

Since the introduction of the new A – Level psychology specifications, complex essay questions have become a prominent feature of the examination landscape. By ‘complex’ questions, I mean those that  impose several distinct skill requirements that must be addressed simultaneously in order to attract credit. For example, the Edexcel 2018 Paper 2 contained the following question in the ‘Criminological Psychology’ section:

Kylie witnessed a crime and had to go to the police station for an interview. The crime involved a robbery of a shop in a busy shopping centre. Kylie was walking past the shop with her friends when she heard the shopkeeper shouting for help, as the thief ran out of the shop. The police carried out a cognitive interview to gather as much information as possible from Kylie about what she witnessed.

To what extent would the cognitive interview be effective in gathering accurate information from Kylie about the crime she witnessed? You must make reference to the context in your answer. (16)

In order to answer this question effectively, the candidate must evaluate cognitive interviewing in the context of the crime witnessed by Kylie. This means she has to show knowledge and understanding (AO1) of how CI might be used with Kylie (AO2) and make judgements about its probable effectiveness in that context (AO3). This is rather more demanding than the ‘Describe and evaluate cognitive interviewing’ type questions that used to prevail at A – Level.

When these types of question started appearing my initial response was that they were needlessly difficult and represented nothing more than a new set of hoops I needed to train my students to jump through. However, a couple of years into the new specifications and I’m now more inclined to welcome them as a challenge for us and our students to embrace. After all, our task as psychology educators is to support our students in attaining mastery of core psychological concepts, research, methodologies and ways of thinking. Complex questions are a more valid test of mastery than straightforward ‘describe and evaluate’ questions as they are not amenable to a brute-force ‘rote learn the facts and the criticisms without understanding’ approach. More pertinently, it seems to me that by using these types of question as a teaching tool, we can support out students in becoming better psychological thinkers.

Because the context material that accompanies complex questions cannot be predicted in advance, they require students to construct their response on the fly, under examination conditions.  In other words, they have to think. As Dan Willingham (2010) memorably points out, thinking is difficult and students are disinclined to do it even under ideal conditions. The examination situation imposes substantial psychological demands that reduce students’ capacity to think effectively (Putwain & Symes, 2018). Consequently, it is our responsibility to teach our students to think in the right ways long before the exam, and support them in acquiring a degree of automaticity that will allow them to devote their already-stretched cognitive resources to engaging with the content of the question.

The trouble with thinking is that you can’t see it. That makes it difficult for us to explain the sorts of thinking we want our students to do. It also makes it difficult for us to access our students’ thinking processes so we can check whether they’re being directed in the right way.  In recent years, I’ve drawn a great deal on Ron Ritchhart’s notion of making thinking visible to support my students in learning how to think (see Ritchhart et al., 2011). Ritchhart’s approach relies on  manipulables (e.g. sticky-notes) to represent concepts and the use of spatial organisation to represent relationships between them. Together with simple, repeatable dialogues and structures, they present a powerful toolbox for reducing unhelpful cognitive load, establishing transferable routines for dealing with generic subject-specific thinking situations and getting students’ thinking out in the open, where we can see it.

A visible thinking routine for complex essay questions

Here’s how I’ve been using visible thinking to teach students how to address complex questions. I lay the groundwork by presenting a question and asking the students to consider how they should address it and what an answer should do.  For example:

Joe has been convicted of criminal damage.  The magistrate sentencing noted that Joe had been arrested a number of times for similar acts and had a record of disruptive behaviour going bad to his school days.  The magistrate accepted that most teenagers get into trouble but that most seem to ‘grow out of it’ whilst Joe had not.  When asked why he had committed this crime, Joe said, ‘mostly because I’m bored…but sometimes things just wind me up.  That day I was supposed to be meeting my mates but the bus didn’t come so I just lost it a bit, smashed the bus stop up a bit.’  Joe’s father and older brother both have a similar history of antisocial behaviour and offending. 

Evaluate personality theory as an explanation of Joe’s offending. (16)

I encourage my students to adopt a four-question routine to set themselves up to address the demands of the problem:

  1. What do I know about this topic?
  2. What’s relevant in the context?
  3. What am I making judgements about?
  4. How can I justify those judgements?

This type of subject-specific metacognition is best taught by modelling, in my experience. The aim is for the students to understand that, in order to address the question satisfactorily, they need to form a principled judgement (AO3) of whether personality theory (AO1) is a valid explanation of Joe’s offending (AO2).

Students are then given sheets of A3 paper and three colours of sticky note (in my case, green, blue and orange). Pairs or three work well. The visible thinking routine is as follows:

  • First, students recall as many facts as they can that represent the knowledge and understanding required to address the question. They write one fact per green sticky note. These are collected in the centre of the A3 sheet, arranging them such that more closely related ideas are grouped together on the page.
  • Second, the students are then asked to read the context material carefully and look for specific things in the text that relate clearly to the facts/ideas on the green sticky notes. Each of these is noted on a blue sticky note and added to the sheet, near to the relevant facts, but concentrically outward.
Different colours denote different skill elements/assessment objectives.
  • Third, the students are asked to identify material relevant to evaluating personality theory, for example, supporting or challenging research findings, conceptual strengths and weaknesses and so on. Each of these is added to an orange sticky note, again placed near to the relevant application (blue) and knowledge (green).

The students are encouraged to keep thinking and recalling more relevant facts, applications and evaluative points throughout the activity, as each point made and recorded may cue either recall of other material or provoke new links between the ideas, deepening understanding.

Lines of reasoning flow from the centre towards the edge.

The fact that all the ideas are present on the page reduces cognitive load, helps the students think more clearly, and tells the teacher where they can most incisively intervene.  The flexible nature of sticky notes allows the students to think and rethink by trying out different positioning and juxtapositionings of ideas. The different colours allow the students to keep track of the different skill demands of the question, allowing them to spot gaps and deploy material effectively. By now the students should be in a position to trace lines of reasoning about the question by working from the middle to the edge.

  • The final step is for students to reorganise the sticky notes into a linear plan from which they could write their response.  The different coloured notes help here, prompting the students to organise their writing into balanced paragraphs that address all the question requirements.
The linear plan supports thinking about the sequencing of material.

I’ve only recently started using this approach with my Year 13s in a consistent way. They have commented positively on how it is helping them keep track of task requirements and organise their ideas before writing. Of course, the long-term intention is to remove the physical placeholders and the prompts from the teacher that support the process, leaving a purely mental routine that the students can use independently and without prompting. My feeling is that the systematic withdrawal of the various elements will be a fairly straightforward thing to plan, and, at each stage, I can draw attention to what I’m removing and why (e.g. ‘last time I gave you three colours of sticky notes but time I’m not…’) so that the students can establish a conscious rationale for their own thinking when approaching this type of problem.

It’s much more complicated to explain this approach than it is to do it in practice; I hope the accompanying photographs make this clear. I believe that it has the potential to help more of my students access the higher essay mark bands in their examinations. More importantly, I also believe that it can play a part in helping my students to become better thinkers in and about psychology.

Thanks

The concentric planning approach on which this VTR draws was developed collaboratively with Charlotte Hubble.

References

Putwain, D. W. & Symes, W. (2018). Does increased effort compensate for performance debilitating test anxiety? School Psychology Quarterly, 33(3), 482-49

Ritchhart, R., Church, M. & Morrison, K. (2011).  Making thinking visible: How to promote engagement, understanding and independence for all learners. Hoboken, NJ: Jossey-Bass.

Willingham, D.T. (2010). Why don’t students like school?  A cognitive scientist answers questions about how the mind works and what it means for the classroom.  San Francisco: Jossey-Bass.

3 thoughts on “Visible thinking and complex examination questions”

  1. Hi Aidan,

    I’ve been using a similar strategy with post-its for essay planning since I got my head around the new spec; we found it especially useful when retraining y13 who had been taught to write everything in ‘blocks’ (ie all their AO1 followed by all their AO3, as would have fit the old style ‘Describe & Evaluate’ Qs you’ve mentioned). Once we got over the “This isn’t how we were taught it before, you mean Miss ___ taught us wrong…” it worked really well and Y12, having started out with this method are now much more confident with it in y13. Students found it really useful to learn how to ‘weave’ different strands together to form logical chains of reasoning which was holding them back from anything above L2 before. Ours tend to work in a more linear fashion (though I like the concentric idea and going to try it) – once they’ve identified all of their AO1 points they arrange them in a coherent order (ie that would suit a straight ‘describe’ question), then they start slotting in the various evaluation points. Not tried it yet with a question involving application too but giving it a go this week (with the Kylie question cited above). Only question in my mind is, in this particular context, (Cog Int) how much the evaluation will really logically flow from the AO1/AO2 (unlike a theory or study-based Q where you can fit points which are specifically relevant to individual features of the thing being presented). Going to throw it at my students and see what they come up with, will let you know!
    Cheers, Paul.

    1. Hi, Paul.

      Interested to hear how your students get on with the concentric approach. We’ve found it quite helpful, (and our English dept. colleagues have really run with it).

      In respect of the AO3 really flowing logically from the AO1/2, I think you’re right – it’s going to be a bit generic; we’d just emphasise to our students the need to keep referring back to Kylie e.g. ‘Geiselman et al. (1986) found that CI gave a 30% increase in correct detail compared to a standard interview, which suggests that Kylie will be able to provide better testimony. However, Fisher et al. (1987) also found that CI increased the number of errors by 28%. If the officers use an enhanced CI, and tell Kylie that they weren’t there and only she knows what really happened, the number of errors Kylie makes should be minimised (Fisher & Geiselman, 1992).’ It’s a bit forced, but I’d rather that than they miss out on credit because their response wasn’t contextualised enough (cf. that diagnosis Q on the 2018 P2).

      I’ve been setting problems that require more specific evaluation in order to provoke better retention of key ideas and less generic evaluation, but I’m not sure how far the Edexcel examiners will go with this as it depends somewhat on what teachers have chosen to teach, where the spec does not provide much detail. For example, I recently set this:

      An armed robbery took place at a gun shop in London. During the course of the robbery, a security guard was shot and injured. Witnesses were asked to give statements to the police. Assess the probability that the quality of testimony will be affected by the weapons focus effect. (16)

      Here, I was looking for the students to respond to the fact that the weapon focus effect does not necessarily occur in contexts where a weapon is expected (Pickel, 2008) and also for the students to make a connection with the Yuille and Cutshall (1986) research. What interested me about the responses is that about 80% of the students made these connections and wrote about them, but it did not occur to them to start with the basic evaluation issues e.g. is there any evidence that the WFE is real (e.g. Loftus et al, 1987)? so I’ve had to start thinking about how to help them start with the obvious before getting to the nuances.

      Will be interested to hear how you get on with all this.

      –Aidan

  2. Hi Aidan,

    Have now tried this with a couple of classes and in two different contexts (Cognitive Interview and Defendant Characteristics – using the example you’ve adapted from the SAMS). As discussed above, they struggled with the CI question when making their AO3 links effectively, although the overall structure of the activity did help them to organise their ideas well (and some of the struggling may well be to do with the fact that they were using this approach for the first time, and it was the first 16 mark application essay we’ve looked at). 2nd time through with DCs was much smoother, students seemed to really engage with the process and ended up with something decent. Definitely something we’ll continue to use, and I think there’ll be a lot of mileage for the synoptic questions from paper 1 and paper 3 (eg the ‘Mandy’ question from P1 2017), so will try with y12 towards the end of this year, and probably introduce earlier in a different context for the crime topic next year.

    What also prompted very interesting discussion was the example answer for the ‘Kylie’ question from the examiners’ report – students did an activity essentially modeled on examiner meetings/teacher INSET. Their analysis of the answer was thorough and well-considered, and they arrived at a consensus which pretty much met my own judgement on the answer. They, like me, were fairly surprised with the actual mark awarded although we turned this into a positive in terms of how much better their own answers should therefore be. Whether that one example can be taken as a fair representation of how examiners might assess future questions is a different matter…

    Have a good Christmas
    Paul

Leave a Reply

Your email address will not be published. Required fields are marked *