Email This Post Email This Post
Home » Blog

How Does PISA Put the World at Risk (Part 1): Romanticizing Misery

9 March 2014 31,735 17 Comments

PISA, the OECD’s triennial international assessment of 15 year olds in math, reading, and science, has become one of the most destructive forces in education today. It creates illusory models of excellence, romanticizes misery, glorifies educational authoritarianism, and most serious, directs the world’s attention to the past instead of pointing to the future. In the coming weeks, I will publish five blog posts detailing each of my “charges,” adapted from parts of my book Who’s Afraid of the Big Bad Dragon: Why China has the Best (and Worst) Education.

Part One: Romanticizing Misery

Andreas Schleicher has on many occasions promoted the idea that Chinese students take responsibilities for their own learning, while in “many countries, students were quick to blame everyone but themselves.” France is his prime example: “More than three-quarters of the students in France … said the course material was simply too hard, two-thirds said the teacher did not get students interested in the material, and half said their teacher did not explain the concepts well or they were just unlucky.” Students in Shanghai felt just the opposite, believing that “they will succeed if they try hard and they trust their teachers to help them succeed.” Schleicher maintains that this difference in attitude contributed to the gap between Shanghai, ranked first, and France, ranked 25th. “And guess which of these two countries keeps improving and which is [sic] not? The fact that students in some countries consistently believe that achievement is mainly a product of hard work, rather than inherited intelligence, suggests that education and its social context can make a difference in instilling the values that foster success in education”[1]

Self-condemnation Does Not Lead to High Scores

Schleicher got the numbers right, but his interpretation is questionable. There are plenty of countries that have higher PISA rankings than France, yet report similar attitudes. For example[2], more students in number eight Liechtenstein and number nine Switzerland (over 54%, in contrast to 51% in France) said their teachers did not explain the concept well. The percentage of students attributing their math failure to “bad luck” was almost identical across the three countries: 48.6% in Liechtenstein, 48.5% in Switzerland, and 48.1% in France. The difference in percentage of students claiming the course material was too hard wasn’t that significant: 62.2% in Liechtenstein, 69.9% in Switzerland, and 77.1% in France. Neither was the difference in the percentage of students saying that “the teachers did not get students interested in the material:” 61.8% in Liechtenstein, 61.1% in Switzerland, and 65.2% in France.

Moreover, the PISA report seems to contradict Schleicher’s reasoning because it finds that students with lower scores tend to take more responsibility:

Overall, the groups of students who tend to perform more poorly in mathematics—girls and socio-economically disadvantaged students—feel more responsible for failing mathematics tests than students who generally perform at higher levels [1, p. 62].

The degree to which students take responsibility for failing in math or blaming outside factors does not have much to do with their PISA performance. The percentage of students who attribute their failing in math to teachers: countries with low percentages of students saying “my teacher did explain the concepts well this week” or “my teacher did not get students interested in the material” do not necessarily have the best ranking. Conversely, countries where students are more likely to blame teachers are not necessarily poor performers.

Using Shanghai as the cutoff, the countries with the lowest percentage (below 35%) of students blaming their teachers for failing to explain the concepts well are Korea, Kazakhstan, Japan, Singapore, Malaysia, Russian Federation, Chinese Taipei, Albania, Vietnam, and Shanghai-China. An almost identical list of countries has the lowest percentage (below 41%) of students blaming their teachers for not interesting students in the material: Kazakhstan, Japan, Albania, Singapore, Thailand, Malaysia, Russian Federation, Montenegro, and Shanghai-China. Combining responses to both questions are combined, we have the top 10 countries where students are least and most likely to blame their teachers (Tables 1 and 2).

Table 1: Top 10 Countries Where Students Are Least Likely to Blame Their Teachers for Failing Math and Their Respective 2012 PISA Ranking

Country PISA Ranking (Math)
Kazakhstan 49
Japan 7
Albania 57
Singapore 2
South Korea 5
Malaysia 52
Russian Federation 34
Chinese Taipei 4
Shanghai-China 1
Viet Nam 17

Table 2: Top 10 Countries Where Students Are Most Likely to Blame Their Teachers for Failing Math and Their Respective 2012 PISA Ranking

Country PISA Ranking (Math)
Norway 30
Italy 32
Germany 16
Slovenia 20
France 25
Austria 18
Czech Republic 24
Sweden 38
Liechtenstein 8
Switzerland 9

As can be seen in the two tables, among the countries whose students are least likely to blame teachers are some of the best (Shanghai, Japan, Korea, Singapore, Taipei, and Viet Nam), worst (Kazakhstan, Albania, Malaysia), and average (Russian Federation) PISA performers. Students who are most likely to blame teachers come from countries that earn the top PISA scores (Liechtenstein, Switzerland, and Germany) and middle-level PISA scores (Norway, Sweden, Italy, Slovenia, France, Austria, and the Czech Republic).

Self-condemnation as Result of Authoritarian Education

What’s intriguing is that the countries whose students are least likely to blame their teachers all have a more authoritarian cultural tradition than the countries whose students are most likely to blame their teachers. On the first list, Singapore, Korea, Chinese Taipei, Shanghai-China, Japan, and Viet Nam share the Confucian cultural tradition. And although Japan and Korea are now considered full democracies, the rest of the countries on the list are not[3]. In contrast, the list of countries with the highest percentage of students blaming their teachers for their failures ranked much higher in the democracy index. Norway ranked first; Sweden ranked second, and Switzerland was number seven. With the single exception of Italy, all 10 countries where students were most likely to blame their teachers ranked above 30 on the Democracy Index (and Italy ranked 32nd).

One conclusion to draw from this analysis: students in more authoritarian education systems are more likely to blame themselves and less likely to question the authority—the teacher—than students in more democratic educational systems. An authoritarian educational system demands obedience and does not tolerate questioning of authority. Just like authoritarian parents [2], authoritarian education systems have externally defined high expectations that are not necessarily accepted by students intrinsically but require mandatory conformity through rigid rules and sever punishment for noncompliance. More important, they work very hard to convince children to blame themselves for failing to meet the expectations. As a result, they produce students with low confidence and low self-esteem.

On the PISA survey of students’ self-concept in math, students in Japan, Chinese Taipei, Korea, Viet Nam, Macao-China, Hong Kong-China, and Shanghai-China had the lowest self-concepts in the world, despite their high PISA math scores[4]. A high proportion of students in these educational systems worried that they “will get poor grades in mathematics.” More than 70% of students in Korea, Chinese Taipei, Singapore, Viet Nam, Shanghai-China, and Hong Kong-China—in contrast to less than 50% in Austria, United States, Germany, Denmark, Sweden, and the Netherlands—“agreed” or “strongly agreed” that they worry about getting poor grades in math[5].

Emperors’ Ploy to Deny Responsibility

In other words, what Schleicher has been praising as Shanghai’s secret to educational excellence is simply the outcome of an authoritarian education.

As discussed previously, Chinese education has been notoriously authoritarian for thousands of years. In an authoritarian system, the ruler and the ruling class (previously the emperors; today, the government) have much to gain when people believe it is their own effort, and nothing more, that makes them successful. No difference in innate abilities or social circumstances matters as long as they work hard. If they cannot succeed, they only have themselves to blame. This is an excellent and convenient way for the authorities to deny any responsibility for social equity and justice, and to avoid accommodating differently talented people. It is a great ploy that helped the emperors convince people to accept the inequalities they were born into and obey the rules. It was also designed to give people a sense of hope, no matter how slim, that they can change their own fate by being indoctrinated through the exams.

Adapted from Who’s Afraid of the Big Bad Dragon: Why China has the Best (and Worst) Education (Chapter 8)

References:

1.         OECD, Ready to Learn: Students Engagement, Drive, and Self-beliefs. 2013, OECD: Paris.

2.         Baumrind, D., Effects of Authoritative Parental Control on Child Behavior. Child Development, 1966. 37(4): p. 887-907.


[1] http://oecdeducationtoday.blogspot.com/2013/12/are-chinese-cheating-in-pisa-or-are-we.html

[2] All data from (OECD, 2013)

[3] according to Democracy Index 2012, an annual ranking of 165 countries’ state of democracy produced by international research group The Economist Intelligence Unit (The Economist Intelligence Unit, 2013).

[4] Based on aggregated data from (OECD, 2013, p. 304).

[5] Based on aggregated data from (OECD, 2013, p. 310).

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading ... Loading ...

17 Comments »

  • How Does PISA Put the World at Risk (Part 1): Romanticizing Misery | Educational Policy Information said:

    [...] YongZhao PISA, the OECD’s triennial international assessment of 15 year olds in math, reading, and [...]

  • James Harvey said:

    A very compelling post, Yong Zhao. Many thanks. I’d like to add a couple of comments.

    There are some very real questions about how appropriate many of these country comparisons are. To state the most obvious: From the quote in your blog, Mr. Schleicher seems to be under the impression that Shanghai is China. He of course knows better. And he knows Shanghai is the wealthiest city in China. He might as well assess Scarsdale, New York and pass it off as the United States.

    Or take Liechtenstein and Kazakhstan, cited in your post. The former is a fabulously beautiful and quite wealthy enclave in the Alps. It has a total population of 35,000 people, men women, and children. Its entire student population could probably fit into one large American high school. The latter: a kleptocracy presided over by the same man since it declared its independence from the former Soviet Union in 1990. He was last re-elected in 2005 with more than 90% of the popular vote. (Khazakstan’s neighbor, Azerbaijan, recently demonstrated an equally touching devotion to democratic principles. In a triumph of efficiency last October, election authorities released final election results a full day before the polls opened.Link below.)

    http://www.washingtonpost.com/blogs/worldviews/wp/2013/10/09/oops-azerbaijan-released-election-results-before-voting-had-even-started/

    Why is the United States being compared with nations such as this as though the comparison was in some way informative and meaningful?

    A more direct point is the following: Earlier this month, after years of denying that PISA’s assessment of students in Shanghai ignored hundreds of thousands of the children of Chinese migrant laborers, Mr. Schleicher earlier this month acknowledged before a committee of the British House of Commons that PISA had failed to account for more than one quarter of the 15-year-olds in the city. (Link below to the Times (of London) Education Supplement.)

    More than a quarter of Shanghai pupils missed by international Pisa rankings
    http://news.tes.co.uk/b/news/2014/03/05/more-than-a-quarter-of-shanghai-pupils-missed-by-pisa.aspx

    This belated acknowledgment calls into question all of the lessons we are supposed to learn from Shanghai’s educational system. It calls into question also the competence with which the PISA assessment is conducted and the integrity of the assessment itself.

  • Michael Maser said:

    Thanks again Yong. PISA testing, and the media that accompanies it, also serves to reinforce the illusion that standardized testing, à la PISA, is valid way to rank children and young people.
    In an era when mass education has come to be defined (hijacked is more correct) by politically-driven initiatives like NCLB, Race to the Top, PISA, etc., the main outcome to my perception as an award-winning educator, has been harm to young learners and the education system.
    The main beneficiaries have been and continue to be principals of the testing industry (now a multi-billion $ lobby force) and political advocates riding in this parade.
    - Michael Maser
    SelfDesign Learning (BC Canada)

  • Alan said:

    You are concerned about the maturation of young adults?
    200 years ago 14 year old boys were leading groups of men into battle (until a common agreement was reached between the major European nations that “it was more proper” to have a minimum age of 16 – which has since stood for many years).
    I know it myself personally that many young people even now-a-days grow up in areas of high danger and extreme hardship – the “stress” of failing a test is laughable!
    Next you’ll be wanting to wrap 30 years olds in diapers feeding them candy flow and lolly pops!

  • Dianne Khan said:

    It just goes to show that any two data points of statistical facts can be drawn out of the set to make any argument.

    Andreas Schleicher knows this, but is the master of the sound bite and these two random data points serve his purpose well.

    However, to start to unpick information in any meaningful way the whole of the data must be considered (alongside the realisation that it may or may not be wholly reliable data in the first place). And that is what we should expect from someone in Schleicher’s position – not simplistic statements, nor smoke and mirrors.

  • Yong Zhao Explains What Is Wrong with PISA | Diane Ravitch's blog said:

    [...] In this post, Zhao demonstrates one of the misleading claims made by Andreas Schleicher, who runs PISA. [...]

  • Education in the Age of Globalization » Blog Archive » How Does PISA Put the World at Risk Part 1: Romanticizing Misery | The Sharing Tree said:

    [...] Education in the Age of Globalization » Blog Archive » How Does PISA Put the World at Risk Part 1:…. Share this:ShareLike this:Like [...]

  • Yong Zhao Explains What Is Wrong with PISA | Georgia SchoolWatch said:

    [...] In this post, Zhao demonstrates one of the misleading claims made by Andreas Schleicher, who runs PISA. [...]

  • Yong Zhao Explains What Is Wrong with PISA | Educational Policy Information said:

    [...] In this post, Zhao demonstrates one of the misleading claims made by Andreas Schleicher, who runs PISA. [...]

  • Yong Zhao--US needs to lead, not follow - I wonder if... said:

    [...] How Does PISA Put the World at Risk (Part 1): Romanticizing Misery [...]

  • Sridhar Rajagopalan said:

    I believe PISA has a number of merits and strengths and one needs to look at data and evidence in education too, just like in the other areas.

    The article starts with the claim that “PISA… has become one of the most destructive forces in education today” (A)

    But it goes on to only argue (and provide some good evidence) that “what Schleicher has been praising as Shanghai’s secret to educational excellence is simply the outcome of an authoritarian education”. (B)

    (B) is probably correct. But far, far removed from (A).

  • Läraren ÄR magiker! | goranwestling said:

    [...] Det finns nämligen inget samband mellan huruvida eleverna “skyller på lärarna” eller inte och resultaten i PISA-undersökningen! (Vill du veta var jag fått detta ifrån kan du klicka här!) [...]

  • Circles of Innovation » Educating Creative and Entrepreneurial Students said:

    [...] is a link to his blog on the topic, and here is Dr. Zhao speaking on the topic. (He also has a 2011 TED Talk you might [...]

  • Stop PISA! ? ¡Paren PISA! | Psicologia en Morelia said:

    [...] “How Does PISA Put the World at Risk”, Dr. Yong Zhao » Lo que oculta el informe Pisa, Carlos Manuel Sánchez, XL Semanal, 27 abril [...]

  • PISA ¿para qué? | Psicologia en Morelia said:

    [...] » ¿Déficit democrático en los informes PISA?, Josep M. Vallès, El País, 22 abril 2014 » “How Does PISA Put the World at Risk”, Dr. Yong Zhao . » Ítems liberados de pruebas de evaluación y otros recursos, INEE, España [...]

  • PISA ¿para qué? El Ecuador en PISA | Psicologia en Morelia said:

    [...] » ¿Déficit democrático en los informes PISA?, Josep M. Vallès, El País, 22 abril 2014 » “How Does PISA Put the World at Risk”, Dr. Yong Zhao . » Ítems liberados de pruebas de evaluación y otros recursos, INEE, España [...]

  • Articles of Interest: March 14, 2014 « National Creativity Network said:

    [...] How Does PISA Put the World at Risk (Part 1): Romanticizing Misery Education in the Age of Globalization [...]

Feel free to comment:

The views expressed on this site are entirely my own. They do not represent my employer or any other organization/institution. All comments are subject to approval.

Add your comment below, or trackback from your own site. You can also subscribe to these comments via RSS.

Be nice. Keep it clean. Stay on topic. No spam.

You can use these tags:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

This is a Gravatar-enabled weblog. To get your own globally-recognized-avatar, please register at Gravatar.