Should we eat more fish or more ice-cream to boost PISA scores?

John Jerrim.

If anyone has ever read one of the international PISA reports or seen Andreas Schleicher present they will know that the OECD is rather fond of cross-national scatterplots. These illustrate the relationship between two variables measured at the country level.

Take, for instance, the chart below. This has been taken from one of Mr Schleicher’s blogposts, and illustrates the relationship between a country’s test scores and its rate of economic growth. It has been interpreted by the OECD as showing “that the quality of schooling in a country is a powerful predictor of the wealth that countries will produce in the long run”.

Source: Research by Hanushek and Woessmann, via the OECD Education and Skills Today blog

Sounds convincing, right?

The trouble is, correlation does not equal causation. And, despite the OECD’s obsession with such cross-country relationships, they can often deceive.

Should we eat more fish?

Of course, it is not only the OECD that produces such analyses. Academics do as well.

One of the more unusual examples I have found is this study by a group of epidemiologists. They look (I kid you not) at the relationship between average fish consumption and average PISA test scores.

Somehow, the study manages to find a strong positive correlation of around 0.6! It then goes on to conclude that a “linear association between fish consumption and PISA scores is likely” and that “it is plausible to assume that a higher level of fish consumption drives improvement at school level”.

As a nation of fish-and-chip lovers, this should be good news for our PISA scores…

But isn’t eating ice-cream just as important?

Getting kids to eat more fish is difficult. My little boy hates the stuff! He does, however, like ice-cream – and lots of it (who doesn’t?). If only eating ice-cream were as good for PISA scores as eating fish.

Well, apparently it is.

The chart below provides one of my all-time favourite PISA graphs. It plots the level of ice-cream consumption of a country against average PISA reading scores. There is actually quite a strong positive relationship, with around half the variation in PISA scores explained.

It also seems that the relationship is exponential (i.e. the line of best fit is curved upwards). Not only should we eat more ice-cream, but we should be eating lots of it.

The take-away message (no pun intended)

Hopefully, the point of this blogpost has become clear.

When the PISA results get released at the start of December, the international report and presentations given by the OECD are bound to include this kind of graph, along with stories about how ‘high-performing countries’ all do X, Y or Z.

Clearly, we should be treating any such interpretation of the PISA results with caution. There are likely to be hundreds, if not thousands, of reasons why some countries do well on PISA and others don’t. In reality, it is almost impossible to separate these competing reasons out.

What we do know is that overly simplistic “explanations” for the PISA results must be avoided. Organisations like the OECD have their own agenda, and it is just too easy for such groups to use the results to promote their own hobby-horses.

Other posts in this series can be found hereherehereherehereherehere and here.

Photos: fish by tkcrash123, ice cream by J Lippold via Creative Commons

Tagged with: ,
Posted in accountability and inspection, International comparisons
7 comments on “Should we eat more fish or more ice-cream to boost PISA scores?
  1. John, super post, as are all in this series!

    I am just intrigued to know that despite us eating so much fish, whether or not there is a negative correlation with chips….. so much research still to do (sigh!).

  2. “If anyone has ever read one of the international PISA reports or seen Andreas Schleicher present they will know that the OECD is rather fond of cross-national scatterplots. These illustrate the relationship between two variables measured at the country level.”

    This is true, but the really important link with national IQ is ignored. I did the statistics myself for PISA maths scores and you can find it here.

    https://rogertitcombelearningmatters.wordpress.com/2016/12/11/national-iqs-and-pisa-it-changes-everything/

    There is a correlation of 0.89, which is very similar to that which results when the GCSE scores of schools within an LA area are plotted against mean intake Cognitive Ability Test (CATs) scores. Whereas causation looks unlikely in the case of fish & ships and ice-cream this is not the case in respect of IQ and it is indeed clear from national CATs data going back decades that CATs scores are the strongest predictor of GCSE results. Indeed when CATs scores are taken into account social and economic disadvantage effects largely disappear.

    My study has been updated with better national IQ data and can be found here.

    https://rogertitcombelearningmatters.wordpress.com/2016/12/18/national-iqs-and-pisa-update/

    Surely there should be no great surprise that OECD’s plot of national economic growth against national IQs follows the same pattern as PISA scores.

    Virtually every article on my website hammers away at these links, the obvious point being that this being the case schools should be focused primarily on exploiting the proven plasticity of cognitive ability/IQ to prioritise its development over jumping the arbitrary GCSE school attainment thresholds need to drive school league tables and the marketisation ideology produced perverse incentives that are actually making our students dimmer rather than brighter.

  3. africanschoolclapham says:

    Great article thanks!
    I wrote a paper on PISA while at the IoE. I visited the offices of the OECD in Paris and had to plead and go through 3 levels of security just to go to the bookshop pick up the 20 pound PISA report.
    Also what Schleicher et al put forward from the report is a mini-sample from the research. If you read more deeply you will see a large part of the findings contradict what the OECD is promoting. So for ex the OECD promotes the publication of student results in league tables but its own research shows that countries that don’t do this have the highest scores (and generally best regarded educational systems). Hmmmm

  4. […] form of relationship. However, as Edward Tufte has famously illustrated, a a recent article from John Jerrim at University College London beautifully illustrates, correlations can be somewhat absurd. As a warning about the upcoming release of PISA scores, […]

  5. […] Other posts in this series can be found here, here, here, here, here, here, here, here, here and here. […]

  6. […] Other posts in this series can be found here,  here, here, here, here, here, here, here, here, here and here. […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

UCL Institute of Education

This blog is written by academics at the UCL Institute of Education.

Our blog is for anyone interested in current issues in education and related social sciences.
Keep up with the latest IOE research
IOE Tweets

Enter your email address and we'll let you know when a new post is published

Join 41,862 other followers