Who are the world’s biggest bullshitters? A light-hearted look at PISA data

John Jerrim.

The PISA study is usually used to measure educational achievement across countries.

However, as well as taking a two hour test covering reading, science and mathematics, young people are also asked to complete a background questionnaire.

This covers a wide range of topics, such as self-confidence, life satisfaction, expectations for the future and out-of-school activities. There is hence a whole host of things – outside of educational achievement – that PISA allows us to compare across the world. 

This includes, for the first time ever, how big bullshitters teenagers are [PDF].

Let me explain…

How is bullshit measured in PISA?

In the 2012 edition of PISA, 15-year-olds across the world were asked how well they understood a set of 16 mathematical constructs. They were asked to indicate a response on a five-point scale, from 1 (“never heard of it”) through to 5 (“know it well, understand the concept”).

These constructs were:

  1. Exponential function
  2. Divisor
  3. Quadratic function
  4. Proper number
  5. Linear equation
  6. Vectors
  7. Complex number
  8. Rational number
  9. Radicals
  10. Subjunctive scaling
  11. Polygon
  12. Declarative fraction
  13. Congruent figure
  14. Cosine
  15. Arithmetic mean
  16. Probability

There is, however, a catch. The three constructs in italics above are completely made up! There is no such thing as a ‘proper number’, ‘subjunctive scaling’ or a ‘declarative fraction’ – yet some teenagers said that they knew such constructs well. I hence use how teenagers responded to these three fake constructs to create a ‘bullshit scale’. 

The chart below compares average scores on this scale across English-speaking countries. Results have been standardised, so positive figures indicate being above the international average on the bullshit scale and negative figures below the international average. The magnitude of results are reported as effect sizes (standard deviation differences).

North American teenagers are found to be the biggest bullshitters in the English-speaking world – Canada and the United States take first and second place. In the middle of the pack are England, Australia and New Zealand. Whereas it is the Irish, Northern Irish and Scottish who are the most straight-talking.

The paper also explores differences between demographic groups, as illustrated in the chart below.

Young men were found to be much bigger bullshitters than young women, while rich kids were much more likely to tell porkies than the poor.

So, if on the way home tonight you meet a charming, posh, male Canadian, perhaps its best you don’t believe everything that they say…

Implications

This work was, of course, meant to be light-hearted and a bit of fun. But it does also have a serious side.

Everyone knows a bullshitter. Yet there has been surprising little research done into this widespread social phenomenon. Resources such as PISA, when used in imaginative ways, can tell us a lot more about the world in which we live than just educational achievement alone. 

Other posts in this series can be found here and here.

Tagged with: ,
Posted in Social sciences and social policy, young people
2 comments on “Who are the world’s biggest bullshitters? A light-hearted look at PISA data
  1. Hi John Great article, but on subject of PISA I would be interested in your response to my analysis of PISA scores as mitigated by cognitive ability/IQ data. You could do something similar on the upcoming round. Best wishes Roger Titcombe

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

UCL Institute of Education

This blog is written by academics at the UCL Institute of Education.

Our blog is for anyone interested in current issues in education and related social sciences.
Keep up with the latest IOE research
IOE Tweets

Enter your email address and we'll let you know when a new post is published

Join 41,040 other followers