skip to Main Content

Doubting Ms Daisy

Daisy Christodoulou’s second book, Making Good Progress? (OUP 2016), is helpful on assessment but as deeply flawed on teaching and learning as her first, Seven Myths About Education (Curriculum Centre 2013/Routledge 2014).  This expressed her frustration with her training by Teach First at a London comprehensive school from 2007 to 2010.  She claims to have “enjoyed it immensely” (page 2), but anger about the kind of teaching she was expected to deliver informs the whole book.  She left teaching after three years to take an MA and returned to another London comprehensive for, it appears, a year before becoming Research and Development Manager at ARK where she has remained.  Her books make no mention of further teaching, observing lessons or discussing teaching and learning with practitioners, SLTs or other researchers.  Her views seem to derive wholly from her teaching experience, a great deal of reading and very strong personal conviction.

On this basis she argues that education in England is damaged by teaching pupils skills rather than knowledge.  The seven ‘myths’ all relate to this failing.  The book is written with great energy and has glowing forewords by E D Hirsch who promotes a knowledge-based agenda in the USA and by Dylan Wiliam, until recently a professor at the Institute of Education.  I will return to these.  More important, Christodoulou’s argument in Seven Myths About Education, which she repeats in the first chapters of Making Good Progress?, has three fundamental flaws of increasing seriousness which she disguises with omission and evasion.

Omissions and evasions

The first is a repeated claim that schools in England focus on teaching skills rather than knowledge.  This view is derived from Christodoulou’s experience of Teach First, the former National Curriculum and Ofsted subject reports which she quotes extensively as praising skills-based lessons.  If Christodoulou had observed a substantial number of ordinary lessons – not lessons specially devised to be observed by inspectors – she would have seen an enormous amount of factual knowledge being taught.  At all ages, from letter shapes to correct spellings to number bonds to foreign language vocabulary to similes and metaphors, teachers endlessly teach the knowledge without which further learning is impossible.  Christodoulou is actually aware of this, writing that “teachers can, and do, put on a show for Ofsted” (Myths, p.31) and gives an example that students’ “ability to spell in French is actually down to teacher instruction and explanation that happened prior to the Ofsted inspection” (p.38).  But she doesn’t accept the implication of this because it would fatally undermine her argument.

The former National Curriculum and Ofsted subject reports are no more than attempts by educational specialists to guide teachers to moderate their natural instinct to impart facts by taking account of skills.  But such a balanced view wouldn’t serve Christodoulou’s agenda.

Second, Christodoulou repeatedly argues that knowledge must precede skill – skills which engage the short-term memory must be able to draw on knowledge which is stored in the long-term memory and must therefore be taught first.  Christodoulou doesn’t mention Anders Ericsson in Seven Myths About Education, but this omission is scarcely possible in Making Good Progress? because he is, as she admits, “the foremost researcher on practice” (p.40), that is, the foremost researcher on how expertise is developed by practice; and in her second book Christodoulou adopts Ericsson’s term ‘deliberate practice’ with approval.  However, she cites only two early papers by Ericsson, from 1990 and 1993, in this second book.  She doesn’t mention Ericsson’s book, Peak (Bodley Head 2016) which gives a full account of his research, even though it was published in Britain well before 6 November 2016 which is given as her access date for online references in Making Good Progress?

Christodoulou omits any mention of Ericsson’s book because he shows repeatedly that knowledge is most effectively acquired through practicing skills, the reverse of Christodoulou’s belief.  As an example of this, Ericsson cites extensive research that experienced doctors and nurses are often less proficient that newly-qualified ones.  The reason for this is that experienced doctors and nurses regularly attend lectures at which new developments are explained and they take notes.  But the new knowledge remains just that – knowledge.  It doesn’t affect the doctors’ and nurses’ skills until it is used in practice, and many doctors and nurses fail to use it.  Research by Ericsson and others is gradually transforming medical CPD by introducing skills-based updating (Ericsson, pp.130 – 137).  Christodoulou is unable to give a fair account of Ericsson’s research because it would fatally undermine her argument.

Third and most serious, Christodoulou is virtually silent about the issue of student motivation.  Ericsson demonstrates that expertise is developed through breaking down complex skills into a series of components which are practiced until expertise is achieved, but he also shows that expertise is rare and achieved through exceptional motivation – most chess players don’t become Grand Masters.  But direct application of this model is inappropriate to schools where all students need to develop the highest level of skill of which they are capable – teachers need to raise attainment by all students, not only the most motivated.

Christodoulou gives little indication of how teachers are to deliver a knowledge-based curriculum except through repetition until the knowledge is securely acquired.  In both books the only significant example is Siegfried Englemann’s Expressive Writing programme, devised in the 1960s, which she praises as an example of direct instruction. This programme “breaks down the skill of writing into a number of quite small and specific tasks and then gets pupils to practice those over and over again in specific contexts” (Making Good Progress? p.46).  It is published as a series of textbooks comprising short exercises, carefully graded for difficulty, through which students work under the teacher’s guidance.

Christodoulou’s use of Englemann provides two major examples of evasion by omission.  In both her books she quotes John Hattie’s praise of Englemann in his Visible Learning : A Synthesis of Over 800 Meta-analyses Relating to Achievement (2009).  But she doesn’t admit that Hattie has drastically modified his support for direct instruction in his later book, Visible Learning for Teachers : Maximizing Impact on Learning (2012).  Hattie has withdrawn his support of Englemann’s approach: “It is unfortunate that many implementations of direct instruction are based on purchased, pre-scripted lessons, which certainly undermines one of its major advantages – that is, teachers working together to create the lesson planning” (p.65). Englemann and direct instruction now appear only as “a method of getting teachers to talk to each other about teaching” (p.65), i.e. as an aspect of CPD, not as a method of classroom instruction.

Hattie has revised his view of effective teaching in his second book and now focusses on influences which show higher impact than direct instruction which is ranked 29th in order of impact.  These include student self-assessment (1st), Piagetian programs (2nd), formative assessment (equal 4th), feedback (10th) and metacognitive strategies (14th) (p.251).  He has moved much closer to the view, outlined by Robert Coe in Improving education : a triumph of hope over experience http://www.cem.org/attachments/publications/ImprovingEducation2013.pdf – and in meta-analyses now accepted by the Education Endowment Foundation – https://educationendowmentfoundation.org.uk/resources/teaching-learning-toolkit – that the most effective ways of raising attainment relate to feedback, metacognition, peer-tutoring and student collaboration, matters (apart from feedback) on which Christodoulou has nothing to say.

A second example of serious evasion by omission is Christodoulou’s claim that “in a major American study of Englemann’s direct instruction method, pupils who were taught using this method outperformed their peers not just on their academic performance, but on affective measures such as self-esteem too.  Direct instruction is successful and pupils enjoy succeeding” (Seven Myths, p.41).  The reference is to the Follow Through project in the USA which involved over 75,000 children, but these were “economically disadvantaged students from kindergarten through grade 3”. i.e. ages 5 to 9.  Christodoulou conceals this vital information because younger children are, of course, more willing than adolescents to undertake repetitive work to please their teacher.  There is no evidence of direct instruction being used successfully in secondary schools other than with very small groups of students with learning disabilities.  Walker et al (2005) write “Research in Direct Instruction (DI) writing programs is promising, but limited at this time to a small number of group design studies” – http://onlinelibrary.wiley.com/doi/10.1111/j.1540-5826.2005.00131.x/abstract

On the basis of these evasions and omissions, Christodoulou recommends Englemann’s Expressive Writing and has introduced it in ARK secondary schools.  I have actually observed lessons using the programme over the years and found them invariably dire – visibly bored teachers working through exercises with equally bored students held in check with draconian discipline or, more imaginatively, with promises of sweets!  It’s possible to create more engaging language exercises, as in Groves’s and Grimshaw’s Smudge and Chewpen books, also of the 1960s, – but even these can only be used in a limited way before the boredom of repetition sets in.

Christodoulou clearly values knowledge for its own sake.  This is confirmed by her Wikipedia entry – https://en.wikipedia.org/wiki/Daisy_Christodoulou – which explains how she captained the University of Warwick team which won BBC’s University Challenge in 2006–7 with personal scores so high that she took part in a special Christmas edition of the show in 2011, her team again reaching the final.  Perhaps she actually overvalues knowledge – valuing it so highly that she is willing to misrepresent evidence in order to promote it.

Assessment for Learning

Sadly these flaws risk undermining the later chapters of Making Good Progress? which make valuable points about assessment.  Christodoulou argues convincingly that formative and summative assessments have different purposes and need to be structured differently; that descriptor-based and examination answers are very difficult to mark consistently; and that well-constructed multiple-choice tests have a useful part to play in formative assessment as quick to mark and showing misunderstandings precisely on which immediate feedback can be given.  Most valuably, Christodoulou outlines ways in which formative and summative assessment can be made more effective and less time-consuming.

This is supported in his foreword by Dylan Wiliam who relates formative assessment to grades in instrumental music and, by extension, to sports coaching and to the graded assessment programmes developed by ILEA and others in the 1980s.  A child learning the piano isn’t set a GCSE-type standard to which she must aspire; she practices scales even though music rarely includes scales and in due course takes Piano Grade 1, retaking if she fails, then moving on to Grade 2.  A child learning football isn’t set a competitive standard to which he must aspire; he is coached repeatedly in various skills until they become instinctive – only then will the quality of his play improve.  This approach enacts Dweck’s growth mindset, though Wiliam and Christodoulou don’t mention Dweck’s work.

Christodoulou and Wiliam argue that this incremental approach will succeed in ways that the current deficit model of tracking used in many schools doesn’t – how far is the student below their target grade and what must be done so they attain it?  I have no doubt Christodoulou and Wiliam are right.  I worked on the graded assessment programmes in the early 90s and saw how students thrived with them before Government refused to approve them as alternative paths to GCSE.  And I sat and later taught London English Language O Level multiple-choice comprehension and was trained in designing multi-choice tests.  Properly designed, these can test all the relevant cognitive skills – verbal understanding, inference, deduction, analysis, evaluation and comparison – quickly and precisely, allowing detailed feedback about misunderstandings.

I fear Christodoulou’s militant and misleading promotion of a knowledge-focussed curriculum will distract from acceptance of her other ideas.  Perhaps more seriously the glowing forewords for Seven Myths provided by E D Hirsch and Dylan Wiliam raise doubts about their judgement.  Hirsch suggests that the book deserves “to be nominated as the best book of 2013 on American education” because the issues apply equally to the USA and Wiliam writes “this may well be the most important book of the decade on teaching (and I reluctantly include my own works in this assessment)”.  Gushing commendations like this are rare in academia.  Hirsch and Wiliam are both experienced educational researchers and they might have been expected to check the claims, or at least the references, in a unrefereed book originally published by a small UK academy chain (Future Academies).  Wiliam at least has come to his senses.  His foreword to Making Good Progress? focusses wholly on assessment and says little more about the book than that it is “timely”, “wide-ranging and important”.

I agree and it would become more so if Christodoulou now apologised for misleading her readers and reissued the book with the evasions and omissions removed.  It would then be a valuable resource for rethinking assessment in schools.  Nothing can be done with Seven Myths About Education, alas, except disregard it as entirely misleading.

Leave a Reply

Your email address will not be published. Required fields are marked *

NB: This post has other comments. You can read them here.

Good news – Progress 8 will control Ofsted

Progress 8 (P8) figures for almost all the state secondary schools in England were published for the first time by the DfE on 19th January – https://www.compare-school-performance.service.gov.uk/. Each school is now judged by the value it has added in 5 years to the expected attainment of its Year 7 students entering in 2011 – a plus figure shows the value the school has added, a minus figure shows the reverse. P8 is now the headline figure for schools’ success and everyone can see how well each school has done compared with the other schools in its local authority area and other members of its academy chain.

The DfE has confirmed that decisions on inspections will now be taken in the light of P8. Schools with +1.0 won’t have a routine inspection in the calendar year following publication of P8 (an inspection could still be triggered by unusual concerns like a complaint, safeguarding or radicalisation). Schools with -0.5 or below are likely to be inspected; and schools with -0.25 or below with other factors for 3 consecutive years will be regarded as ‘coasting’ and liable to intervention – see DfE Progress 8 and Attainment 8, https://www.gov.uk/government/publications/progress-8-school-performance-measure pages 8/9.

A less noticed result of P8 is that it will now control Ofsted’s judgements.
Progress 8 is reported in 5 categories:

Well above average +0.50 and above About 5% of schools
Above average +0.49 to +0.20 approx[1] About 25% of schools
Average +0.20 approx to -0.20 approx About 40% of schools
Below average -0.20 approx to -0.50 About 20% of schools
Well below average -0.51 and below About 10% of schools

A school with Above average P8 can hardly now be judged as Requiring Improvement (except for unusual reasons like safeguarding or radicalisation). And a school with a Below average P8 can hardly be judged as Good, let alone Outstanding. Objective evidence of value added by the school’s work over 5 years must necessarily take precedence over the judgements typically of 5 inspectors carried out over 2 days.

The consequences can be striking. For example, a Let’s Think in English.school in a northern city assessed as Requiring improvement and attaining Above average P8 (+.0.33, the equal 6th highest of the city’s 32 schools for which P8 is available) cannot now reasonably be judged as less than Good. And a school assessed as Good in January 2016 attaining Below average P8 (-0.29, the lowest of the 14 non-special schools in its London borough for which P8 is available) can no longer be regarded as Good.

Ofsted inspectors are required to take statistical information about the school into account (School Inspection Handbook, paragraph 29). It is therefore very hard to see how, apart from unusual factors such as safeguarding or radicalisation, inspectors can now judge schools with Well above average P8 as other than Outstanding, schools with Above average P8 as other than Good or better and schools with negative P8 as other than Requiring improvement or, particularly if they have Well below average P8, as other than Inadequate.

It is clear that Ofsted’s internal quality control will scrutinise discrepancies between inspectors’ judgements and Progress 8 to avoid the risk of a judicial review which Ofsted would lose.

Benefits

This control of Ofsted through Progress 8 will have five benefits for teachers.

  1. It will remove the uncertainty of Ofsted inspections and therefore their ‘threat’ element which contributes to teachers’ stress. Knowing that Ofsted can’t reasonably come up with a judgement different from what’s shown by Progress 8 will make inspections little more than routine. As parents and local communities come to understand what P8 means, as they soon will, and as P8 scores are updated annually as Ofsted judgements can’t be, P8 rather than Ofsted will become the yardstick of the school’s reputation.

  2. Ofsted inspections will become rarer and may soon be limited to schools with Well below average or unchanging negative P8 scores and schools with significant drops in P8. This may well be intended. Ofsted inspections are expensive and reducing their number will make significant savings.

  3. These changes will gradually stop SLTs using Ofsted to frighten their staff and making unreasonable demands on them. The Ofsted School Inspection Handbook, paragraph 28, contains a remarkable list of 19 things that Ofsted does not require from schools (“does not” is in bold each time). They include such things as Ofsted requiring lesson plans, evidence of previous planning and assessments, and data recorded in a particular format or frequency. The Handbook calls these “myths” and they clearly have been included because SLTs still use them to pressure their staff to work in particular ways – https://www.gov.uk/government/publications/school-inspection-handbook-from-september-2015

    Moving to P8 should put an end to this, also to schools buying advice from former Ofsted inspectors and spending time and resources on mock inspections.

  4. Moving to Progress 8 will also put an end to the unfairness where Ofsted judges schools in middle-class areas as Outstanding much more often than schools in more deprived areas. The Education Policy Institute has found that secondary schools with up to 5 per cent of pupils eligible for free school meals (FSM) are over three times as likely to be rated ‘outstanding’ as schools with at least 23 per cent FSM. Secondary schools with the most FSM pupils are much more likely to be rated ‘inadequate’ than those with the fewest.

    The least deprived schools are also most likely to improve their Ofsted judgement and least likely to be down-graded. Secondary schools with the fewest pupils with low prior attainment are almost six times as likely to be rated ‘outstanding’ as schools with the most low prior attainers. At the other end of the scale, schools with the most low prior attainers are much more likely to be rated ‘inadequate’ than those with the fewest low prior attainers – https://epi.org.uk/report/school-inspection-in-england/

    This shows Ofsted inspectors’ bias towards schools in middle-class areas and the replacement of such judgements with a more objective system is long overdue.

  5. A move to Progress 8 as schools’ headline figure for accountability should in due course allow Ofsted to resume HM Inspectors’ original role of advising on good practice. At present Ofsted inspectors are forbidden to give advice on improving teaching and learning. The School Inspection Handbook, paragraph 28, begins:

    Inspectors must not advocate a particular method of planning, teaching or assessment. It is up to schools themselves to determine their practices and for leadership teams to justify these on their own merits.

    This shows Ofsted’s continuing lack of confidence in the professional judgement of its inspectors, probably for good reason in that most of its inspectors developed their experience under the now discredited National Strategies. But it causes the bizarre situation in which inspectors judge lessons but cannot advise on how to improve them. It means Ofsted can’t recommend to schools the excellent advice in its own publications such as Moving English forwardhttps://www.gov.uk/government/uploads/system/uploads/attachment_data/file/181204/110118.pdf It is a strange situation when the Government body charged with raising standards in schools is not allowed to advise schools how to do this. Perhaps the new Chief Inspector will recognise the opportunity given by Progress 8 to rectify this.

[1] Approx indicates a narrow range of scores where schools can be categorised as Above average or Average (or Average / Below average) depending on the confidence interval of the school’s score. This is a statistical calculation explained at Progress 8 and Attainment 8, pages 21/22 and Annex D.

Leave a Reply

Your email address will not be published. Required fields are marked *