How Does PISA Put the World at Risk: Part 1 Part 2 Part 3 Part 4

Part 5

If I were a conspiracy theorist, I would suggest that PISA is a secrete plan of Western powers to derail China’s education reforms.  China has been working hard to introduce significant reforms since the 1990s to overcome the apparent shortcomings of its education system in order to cultivate a more diverse, creative, and entrepreneur citizenry. Such a citizenry is urgently needed for China’s successful transition from a labor-intensive economy to one that relies on innovation, a transition China must make for its future development. The Chinese exam-oriented education has long been recognized as the culprit for limiting China’s capacity for producing creative and diverse talents. Just as China’s education reforms began to touch the core of its traditional education—the gaokao or College Entrance Exam and the wide use of testing at all levels of education, PISA announced that the Chinese education is the best in the world. And the exam system, including the gaokao, is glorified as a major contributor to China’s success, making it difficult for the Chinese to continue the battle against testing. Even Marc Tucker, president and CEO of the National Center on Education and the Economy and one of the most prominent PISA proponents who has on many occasions expressed unequivocal admiration of China’s education, admits[1]:

… many people in China are upset about the success of Shanghai on the PISA league tables, because they think that success will blunt the edge of their fight to dethrone the Gaokao from its premier position as the sole determinant of advancement in Chinese society. They see the Gaokao as enforcing an outdated ideal of education, one that rewards memorization and rote learning over understanding and the ability to apply mastery of complex skills to real world problems, particularly problems requiring innovation and creativity.

If I expanded the conspiracy theory, I could say that PISA is a plot to disrupt all Eastern Asian countries’ serious efforts to develop an education system that cultivates confident, creative, diverse, and happy students. For example, PISA “played a role in the decision to reverse, at least in part, the yutori reform launched at the beginning of the decade,” writes a 2011 OECD document[2]. Yutori kyoiku (roughly translated “relaxed education” or education with some freedom) was a major education reform movement started in the 1980s in Japan. “The yutori reform was based on an emerging consensus that the school system was too rigid and that a new approach was needed to encourage creativity,” observes the OECD document[3]. The major changes included reduction in school days and a 30% cut in the school curriculum. “In addition, the government relaxed grading practices and introduced “integrated learning classes” without textbooks in an effort to help students think independently and reduce the importance of rote learning” [4]. The changes were announced in 1998 and implemented in 2002. “The ultimate desire was to instill in students ‘a zest for learning.’”[5]

In 2003, Japan’s PISA rankings fell, resulting in a public panic over Japan’s decline in international academic standing. Opponents of the yutori reform seized the moment and blamed the reform for the decline. In response, Japan decided to water down the previous reforms with increase in required topics in standard academic subjects, increase time devoted to these subjects, and introducing national standardized testing in math and Japanese for the first time in 2007.

Putting someone on a pedestal is an effective way to ensure he does not veer far from his previous behaviors because any deviation could tarnish the bestowed honor. The Chinese call such actions pengsha or “killing with flattery.” Pengsha derives from a story recorded almost 2,000 years ago: A nobleman rides on a beautiful horse and wins great praises from admiring onlookers. Enjoying the flattery, the nobleman keeps on riding till the horse dies from exhaustion.

PISA has certainly successfully put a number of East Asian education systems on a pedestal and thus constrained their ability and desire to make drastic changes. But they need drastic changes if they wish to truly cultivate the kind of talents needed to become innovative societies that rival the West because the authoritarian East Asian education model leaves little room for creative and unorthodox individuals to pursue their passion, question the authority, and develop their strengths, although it is extremely effective in homogenizing individuals, enforcing compliance, and hence producing great test scores.

PISA’s claims about progress East Asian education systems have made over the years can further convince them to keep riding their horses. It gives them the illusion that they are moving forward, in the right direction, because their PISA rankings keep going up. But in reality, East Asian education systems have never “risen,” as PISA often claims. They have always been great test takers. Singapore, Korea, Japan, and Hong Kong scored extremely well on international tests succ as TIMSS prior to the birth of PISA. Shanghai did not participate in these studies but if it did, it would have scored well.

PISA’s problem-solving assessment outcomes can also give these countries the illusion that their students are creative because their students scored well on the tests. But the fact is that solving existing problems is very different from posing or finding problems worth solving. Today, coming up with problems worth solving is far more important than solving existing problems, at least the kind of problems PISA put forth. As important is the motivation and other non-cognitive qualities required for posing and solving worthwhile problems, which PISA has nothing to say about.

Seeing the damaging effects of PISA on Western education systems debunks my conspiracy theory. PISA, rather than an evil ploy hashed out by Western powers to keep East Asian countries from being innovative, is an ironic tragedy of the 21^st Century born out of ignorance. The genuine anxiety over their low rankings, the sincere admirations (or envy) of Shanghai’s status, the eager actions to borrow from top performers, and the authentic efforts to emulate Asian education are all evidence that political leaders of many Western nations, particularly the U.S., England, and Australia did not work together to use PISA to derail education reforms in Asia. They are truly concerned about improving their own education, but they have been misguided by PISA.

There is no question that education in the West, and for that matter everywhere in the world, needs transformational changes, in the face of transformative changes in the world. Education in the U.S., for example, is far from adequate to prepare citizens for the future. It is obsolete, broken, and must be replaced with a new paradigm. However PISA is not offering such a paradigm. The top performers of PISA are simply better implementation of the old paradigm—the Prussian industrial model of education, which many Western education systems, including the U.S. system, are based on.

In other words, the PISA does not only have the effect of discouraging East Asian systems from abandoning their old paradigm, but also luring Western countries to fix the old paradigm by shaming them for not having been as perfectly obsolete as their Asian counterparts. It keeps them fixated on things (e.g., test scores) that matter little for the future, while neglecting the work they should be doing—inventing a new paradigm.

By associating test-scores with a nation’s capacity for future competitiveness, PISA scares political leaders into actions to raise test scores. The sad truth is that even if children in America, Australia, or Britain scored as high as those in China or Korea, they would not be as competitive because their scores cost a lot more. To compete with skills and knowledge that take much less resource to acquire, it is necessary to develop talents, abilities, and skills that are qualitatively, not quantitatively, different. More important, in a globalized world with over seven billion people, global collaboration is a lot more productive for all than competition. Global collaboration requires differentiated skills and talents.

By misleading countries to believe scores are the only worthy outcomes of education and discounting family spending on education in some countries, PISA puts shame to nations that spend more on education as inefficient. In response, instead of capitalizing on these resources to produce outcomes not measured by PISA and unattainable without the resources, these countries may reduce investment or waste their resources on what can be indeed achieved cheaply: test scores.

By directing the attention to fixing elements of traditional educational arrangements—curriculum, teachers, and or testing–in traditional configurations such as classrooms and schools, PISA squanders opportunities for true innovations brought about by technology and globalization. Instead of working on exploiting these potentials to create a new paradigm that delivers education experiences beyond the physical classroom that are globally connected, product-oriented, and personalized, PISA sends governments to waste time on prescribing curriculum, developing tests, and fixing teachers.

By attracting poor, developing countries into a senseless academic race, PISA wastes precious resources of these countries. While the 182,000 euros (about US$250,000) participation fee[6] and millions of dollars implementation costs may not be much for developed nations, it can be a huge burden for developing countries. More important, the money can buy a lot more meaningful education resources—pencils, for example—than humiliating PISA rankings or policy advice that cannot be implemented.

“PISA is a good servant but a bad master,” wrote Finnish education scholar Pasi Sahlberg, author of the Finnish Lesson: What Can the World Learn from Educational Change in Finland. Pasi is, as always, wise and generous, but in my mind, PISA is a servant that has turned into a bad master, perhaps by design. As it commands the world to race to fix the old paradigm and forgo opportunities to invent a new one, it puts the entire world at risk.

Part 1: Romanticizing Misery

Part 2: Glorifying Educational Authoritarianism

Part 3: Illusory Models of Excellence

Part 4: Misleading the World

Adapted from my book Who’s Afraid of the Big Bad Dragon: Why China has the Best (and worst) Education in the World.

[2] Jones, R. S. (2011), “Education Reform in Japan”, OECD Economics Department Working Papers, No. 888, OECD Publishing. http://dx.doi.org/10.1787/5kg58z7g95np-en. P. 22

[3] Jones, R. S. (2011), “Education Reform in Japan”, OECD Economics Department Working Papers, No. 888, OECD Publishing. http://dx.doi.org/10.1787/5kg58z7g95np-en p. 13

[4] Jones, R. S. (2011), “Education Reform in Japan”, OECD Economics Department Working Papers, No. 888, OECD Publishing. http://dx.doi.org/10.1787/5kg58z7g95np-en p. 13

[5] http://asiasociety.org/education/learning-world/japan-recent-trends-education-reform

[6] It is very difficult to find public data about the total cost of PISA around the world, but there are some clues. First, PISA requires a substantial contribution from each participant to cover “the international overhead costs”. The 2015 PISA requires 182,000 euros (about US$250,000) for each new participant.  I cannot find information about the amount for existing ones, but the UK contributed 200,000 pounds per year to OECD for the three year cycle of the 2012 PISA, according to information provided by UK Department of Education in response to a public information request. If the UK contribution were typical and to be continued, that would be 600,000 pounds (about one million US dollars).

Participating entities have to cover all costs related to the implementation, “including staff costs related to the review of items, sampling activities, training of the test instructors, data coding, analysis of the national results and material costs such as printing test, travel expenses for the National Project Manager,10 etc.,” according to an OECD document. “Countries also must bear the costs for translation of items and questionnaires… It is important to note that national costs far surpass the international costs.” It is hard to find information about how much PISA cost each participating entity, but the direct cost to Canada was about $2.4 million a year for the most recent cycle. The UK spent about 575,000 pounds (nearly $1.0 million) for the latest cycle. Mark Schneider, vice president at the American Institute for Research and former commissioner of the National Center for Educational Statistics estimates that “The cost to test that number of students [about 1,500 as required by PISA] is around $700,000 for PISA and, based on the experience of Massachusetts,” which participated in the latest PISA as a state. The cost of Peru’s implementation of the 2,000 PISA was $380,000.