Who are the world’s biggest bullshitters? A light-hearted look at PISA data

John Jerrim.

The PISA study is usually used to measure educational achievement across countries.

However, as well as taking a two hour test covering reading, science and mathematics, young people are also asked to complete a background questionnaire.

This covers a wide range of topics, such as self-confidence, life satisfaction, expectations for the future and out-of-school activities. There is hence a whole host of things – outside of educational achievement – that PISA allows us to compare across the world. 

This includes, for the first time ever, how big bullshitters teenagers are [PDF].

Let me explain…

How is bullshit measured in PISA?

In the 2012 edition of PISA, 15-year-olds across the world were asked how well they understood a set of 16 mathematical constructs. They were asked to indicate a response on a five-point scale, from 1 (“never heard of it”) through to 5 (“know it well, understand the concept”).

These constructs were:

1. Exponential function
2. Divisor
3. Quadratic function
4. Proper number
5. Linear equation
6. Vectors
7. Complex number
8. Rational number
9. Radicals
10. Subjunctive scaling
11. Polygon
12. Declarative fraction
13. Congruent figure
14. Cosine
15. Arithmetic mean
16. Probability

There is, however, a catch. The three constructs in italics above are completely made up! There is no such thing as a ‘proper number’, ‘subjunctive scaling’ or a ‘declarative fraction’ – yet some teenagers said that they knew such constructs well. I hence use how teenagers responded to these three fake constructs to create a ‘bullshit scale’. 

The chart below compares average scores on this scale across English-speaking countries. Results have been standardised, so positive figures indicate being above the international average on the bullshit scale and negative figures below the international average. The magnitude of results are reported as effect sizes (standard deviation differences).

North American teenagers are found to be the biggest bullshitters in the English-speaking world – Canada and the United States take first and second place. In the middle of the pack are England, Australia and New Zealand. Whereas it is the Irish, Northern Irish and Scottish who are the most straight-talking.

The paper also explores differences between demographic groups, as illustrated in the chart below.

Young men were found to be much bigger bullshitters than young women, while rich kids were much more likely to tell porkies than the poor.

So, if on the way home tonight you meet a charming, posh, male Canadian, perhaps its best you don’t believe everything that they say…

Implications

This work was, of course, meant to be light-hearted and a bit of fun. But it does also have a serious side.

Everyone knows a bullshitter. Yet there has been surprising little research done into this widespread social phenomenon. Resources such as PISA, when used in imaginative ways, can tell us a lot more about the world in which we live than just educational achievement alone. 

Other posts in this series can be found here and here.

‹ PISA: England’s schools segregate by ability more than almost every other country in the world

Oracy: children’s skills are skewed by deprivation and privilege. How can schools bridge the gap? ›

Tagged with: PISA, teenagers
Posted in Social sciences and social policy, young people