Grammar schools and access to universities: HEPI report not an accurate or complete picture

Lindsey Macmillan, Matt Dickson, Simon Burgess.

HEPI Occasional Paper out today claims that “grammar schools … play a significant role in supporting social mobility”. This is based on two statements in the paper: firstly, that a high proportion of disadvantaged pupils attend grammar schools, and secondly, that areas with selective systems are better at progressing children into elite universities.

The striking claim is in stark contrast to much of the rest of the evidence, including our own. On closer inspection, the report’s claim relies on faulty data and inappropriate statistical methods.

Faulty data

The claim made by the report’s author that ‘45% of pupils in grammar schools come from below median income households’ is based on a Department for Education (DfE) technical report that clearly states “The data and threshold used to define households as below the median income in this analysis should therefore be treated as provisional. Caution should be taken in drawing definitive conclusions from these findings until we have completed further work” (p. 12).

This DfE report presents analysis on the first attempt to match household income data records into the National Pupil Database. As is quite clear on reading the DfE report, the household income data that is matched in is incomplete, particularly for higher earners, meaning that 70% of pupils are defined as below median household income (£24,900) when using a threshold from a national survey.

Using data that dramatically over-estimates the number of below-average-income families is obviously central to how we should view a finding that there are more below-average-income families in grammar schools than we might have thought.

However, the HEPI report does not mention this severe drawback of the data, and makes no correction for it in the analysis. Rather this questionable finding is repeated numerous times throughout the report.

Our own published analysis of the background of pupils attending grammar schools clearly shows that these schools provide very limited opportunities for pupils from both disadvantaged and ‘ordinary working’ families.

Inappropriate statistical methods

The analysis on progression to higher education compares progression in areas with grammar schools (selective areas) to progression in all non-grammar areas (see Table 2, page 27).

This is a naïve comparison, given that we know that selective areas are not representative of the nation as a whole. The areas that chose to keep grammar schools have specific characteristics – they are, for example, generally more affluent and have a higher proportion of degree educated people. These are precisely the sorts of characteristics that support access to elite universities, and so we would naturally expect to see more pupils in those regions attending ‘highly selective higher education institutions’. As such, comparing grammar school areas with all other areas will upwardly bias the grammar school effect.

Other research has shown that even restricting analysis to grammar areas matched with similar looking non-grammar areas does not solve this bias problem, and so we should expect the naïve comparison in the report to contain a non-trivial upward selection bias. This inappropriate statistical approach means that the report’s conclusion is unlikely to be robust.

Incomplete picture

More fundamentally, conclusions about the impact of any policy on social mobility have to consider the impacts at the bottom of the distribution as well as the top. Even if grammar schools increase the chances of disadvantaged children who attend them going on to (elite) HE institutions, the aggregate impact on social mobility depends on whether there is any offsetting impact on the disadvantaged in these areas who do not attend the grammar school. Given that the vast majority of disadvantaged students in selective areas do not attend grammar schools, and the evidence suggests that these students suffer an education and wage penalty compared to their equivalents in comprehensive areas, the conclusion that grammar schools actually enhance aggregate social mobility is unlikely to hold true.

The merits or otherwise of selective versus comprehensive education is a fiercely debated area of education policy and has been reignited by recent government moves to expand the grammar school system. As with all policy analysis, it is very important to ensure that appropriate data and robust statistical methodologies are used before drawing conclusions and making policy recommendations. This is particularly the case when the actual aggregate impacts of a policy may in fact be the reverse of what is being claimed.

Dr Lindsey Macmillan is an Associate Professor (Reader) of Economics at UCL Institute of Education, Dr Matt Dickson is a Reader in Public Policy at Institute for Policy Research, University of Bath, and Professor Simon Burgess is a Professor of Economics at University of Bristol.

This piece was originally posted on the University of Bath’s IPR Blog.

Posted in Education policy, Evidence-based policy, Further higher and lifelong education
4 comments on “Grammar schools and access to universities: HEPI report not an accurate or complete picture
  1. The authors of this post are of course correct but there is much more, as set out in this article and its links.

    https://rogertitcombelearningmatters.wordpress.com/2016/08/19/bringing-back-grammar-schools-would-lower-the-national-iq/

    In the article I also make a different argument against academic selection in our school system.

    Academic selection at age 11 lowers our national IQ.

    Every new grammar school creates at least three similar sized secondary moderns. How can these schools still meet the GCSE grade performance thresholds imposed by the government?

    Only by abandoning any serious attempt to provide a cognitively demanding, broad and balanced education, through developmental teaching methods. Such empowering education will be replaced by training, and the teaching methods of behaviourism will dominate. This already happens in comprehensive schools that have an intake cognitive ability profile skewed towards lower CATs scores.

    These teaching methods do not result in cognitive development and will not make our school leavers cleverer or wiser, which is what is really needed.

    Even when the marketisation and competition model is finally abandoned along with the ‘Tyranny of testing‘ required to drive it, secondary modern schools will find it more difficult to provide a full, broad and balanced curriculum for all pupils.

    The first Principal of Mossbourne Academy and the former Chief Inspector of Schools, Sir Michael Wilshaw, was very clear about the superiority of all ability comprehensive schools compared to a mixture of grammar and secondary moderns. This is from a Guardian article of 14 December 2013.

    “In comments that put him on a collision course with the then education secretary Michael Gove, who has expressed support for grammar schools. Wilshaw said: “Grammar schools are stuffed full of middle-class kids. A tiny percentage are on free school meals: 3%. That is a nonsense. What we have to do is make sure all schools do well in the areas in which they are located.”

    But will the degradation and impoverishment of the education available to 11 plus failures be more than made up for by ‘grammar school excellence’ for the more able? The following questions are crucial.

    Is the quality of teaching better in grammar schools than in comprehensives?

    Do grammar schools support the learning of bright children from poorer homes better than comprehensives?

    There is no evidence that either is the case. As the former Head of OfSTED, Sir Michael Wilshaw should know.
    :

  2. John Mountford says:

    Interested by the apparent contradictory nature of the ‘evidence’ from research into the allegedly clear benefits of grammar school education carried out by HEPI and, as was claimed to be the case in your blog, the fact that two vital aspects of that research do not ‘stack-up’ in the face of detailed analysis, I took the opportunity to look up HEPI and to read the report for myself. As a reasonably intelligent and interested citizen, if I did not already have a clear view about the inappropriateness of the grammar school selective process and its mainly damaging outcomes, especially for late developers and the disadvantaged, I would be unable to make up my mind who to believe in this battle of the academic giants.

    Obviously now, curious to dig a little deeper in order to settle this battle over this really important issue, I decided to write to both yourselves and to Sir Ivor Crewe, Master of University College, Oxford and Chair of HEPI’s Trustees, in an attempt to expose the evidence in question to greater scrutiny. It is my hope that both parties will agree to engage.

    This is what the HEPI website says is its raison d’etre – “The Higher Education Policy Institute was established in 2002 to shape the higher education policy debate through evidence. We are UK-wide, independent and non-partisan. We are funded by organisations and universities that wish to see a vibrant higher education debate as well as through our own events.”

    In his introduction to the launch of the HEPI report, you can read Nick Clegg’s views at –
    https://www.hepi.ac.uk/2019/01/10/grammar-schools-progressive-rhetoric-can-lead-regressive-policies/

    As will be seen from reading Nick Clegg’s own words, the argument to retain the grammar school system is based on the assumption that ‘with all other things being equal’, it works.
    ” these state-funded selective schools and with all other things being equal, Oxbridge would have far fewer students with Black and Minority Ethnic backgrounds. In addition, other selective universities would have far fewer students from the bottom half of the income distribution.” This line of reasoning really does assume that in order to tackle the issue of inequality of opportunity, the only thing opponents of the present unfair system want to do is end selection. This would be a first and essential step in trying to create a fairer society, well beyond the school system.

    There are enormous improvements to teaching and learning that will only find their way into classrooms once we rid ourselves of the excessive testing and examination culture that currently impoverishes the education of most young people. Grammar schools contribute to the collective consciousness that promotes aggressive competition as a key outcome of the educative process.

  3. […] on it (bout of tonsillitis). If you didn’t, Lindsey Macmillan et al. have written one response. However, tonight I am better and from my quick read of the HEPI report it is immediately clear how […]

  4. […] is a fundamentally erroneous claim. Two responses from UCL will suffice to give an idea, one from Lindsey Macmillan and the other from Becky Allen, of where the HEPI report has gone astray, both for reasons of […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

UCL Institute of Education

This blog is written by academics at the UCL Institute of Education.

Our blog is for anyone interested in current issues in education and related social sciences.
IOE Tweets
Keep up with the latest IOE research

Enter your email address and we'll let you know when a new post is published

Join 34,154 other followers