Understanding the Limitations of Assessment

What does Ofsted mean by ‘understanding the limitations of assessment’?

The phrase ‘understand the limitations of assessment’ appears four times in the current Ofsted School Inspection Handbook, once as above and three further times in the descriptor for Good Quality of Education. Outstanding schools must also meet ‘all of the good criteria securely and consistently’, so must also understand the limitations of assessment.

This phrase is not defined in the School Inspection Handbook, however; and is mentioned only once in the Education Inspection Framework, but with no further explanation. The phrase was adapted from the Making Data Work report, published in 2018, and is one of the four principles for evaluating data use in schools defined by the Teacher Workload Advisory Group – that ‘the precision and limitations of data, and what can be inferred from it, are well understood.’

As can be seen from the wording in the Making Data Work report (extract below), some nuance was lost when the phrase was adapted by Ofsted.

Whilst the Making Data Work report built on both the earlier Eliminating unnecessary workload associated with data management report and the Final report of the Commission on Assessment without Levels (both of which are worth revisiting to consider what they have to say about assessment and data), the word ‘limitations’ does not appear in either report; the Making Data Work report is therefore the origin of the phrase which was adapted by Ofsted.

Why was the phrase introduced into the current framework?

To understand why the phrase was introduced into the current Ofsted framework, it is necessary to look at previous frameworks and the problems created by the wording which they contained.

Prior to 2015, Ofsted’s School Inspection Handbook contained the following wording:

It is clear that this framework assumed that schools could and should accurately assess pupils’ ‘progress and attainment levels’. This led to schools attempting to do just that; there was no suggestion that schools should understand the limitations of assessment in the way they are expected to do currently. If anything, schools were encouraged to make assumptions about assessment, attainment and progress which were not warranted, as explained by the Making Data Work report.

From 2015, Ofsted increased the pressure on schools to spend time and effort attempting to generate information which could be used to analyse what Ofsted called ‘performance information’, as well as introducing demands for schools to use this information to evaluate ‘differences between groups‘, by which they meant ‘between mean scores of groups of students‘ within a school.

This in turn caused so many concerns across the schools sector that Ofsted was forced to issue a series of ‘Myth Busting’ statements (see here and here for examples), many of which were included in the current Inspection Framework introduced in 2019. A small sample is shown below.

Schools had been encouraged to make assumptions about student performance which were not warranted and this had created unnecessary burdens for schools.

So what does the phrase ‘understand the limitations of assessment’ mean for those currently in school?

As the wording in the current Inspection framework states, those holding school to account expect that school leaders and other staff both understand the limitations of assessment and use assessment in a way that will avoid creating unnecessary burdens. The wording in the Making Data Work report gives a clear example what the ‘limitations of assessment’ means in practice.

This means that, currently at least, those working in school have a solid defence against any pressure to spend time and effort making unwarranted assumptions about the performance of those they are educating.