How Does PISA Put the World at Risk (Part 4): Misleading the World

Statistik Penting	Angka
Bahagian Pasaran Mega888 di Malaysia	Melebihi 90%
Pemain dari Malaysia, Singapura, dan Thailand	80%
Jumlah Muat Turun Aplikasi Mega888	Lebih 1 juta

Statistik Mega888	Nilai
Bahagian Pasaran Permainan Slot di Malaysia	Lebih 90%
Pemain dari Malaysia, Singapura, dan Thailand	80%
Jumlah Muat Turun Aplikasi Mega888	Lebih 1 Juta

Jenis Bonus	Perincian	Syarat Tuntutan
Bonus Selamat Datang	Bonus sehingga RM1,000 untuk pemain baru	Daftar akaun baru dan buat deposit pertama
Bonus Deposit	Bonus sehingga 150% daripada jumlah deposit	Buat deposit ke dalam akaun dan penuhi syarat taruhan
Promosi Bermusim	Bonus dan ganjaran istimewa seperti Birthday Month Bonus	Ikuti syarat dan tempoh promosi yang ditetapkan

Ciri-Ciri	Touch 'n Go	GrabPay	Boost
Had Transaksi	Tinggi	Fleksibel	Munasabah
Kelajuan Pemprosesan	Pantas	Pantas	Pantas
Keselamatan	Baik	Baik	Baik
Populariti di Mega888	Tinggi	Sederhana	Rendah

Kaedah Sokongan	Waktu Operasi	Tindak Balas Purata
Telefon	24 jam sehari, 7 hari seminggu	20 saat
E-mel	24 jam sehari, 7 hari seminggu	6 jam
Borang Hubungi dalam talian	24 jam sehari, 7 hari seminggu	2 jam

Kaedah Pengeluaran	Tempoh Pemprosesan
E-wallet	1 hari bekerja
Perbankan dalam talian	1 hari bekerja

Creative, Entrepreneurial, and Global: 21st Century Education

Email This Post

Home » Blogs, China/Chinese, Education Reforms, Globalization

How Does PISA Put the World at Risk (Part 4): Misleading the World

29 March 2014 76,032 7 Comments

How Does PISA Put the World at Risk: Part 1 Part 2 Part 3

Part 4

China’s poorest beat our best pupils—The Telegraph (UK), 2-17-2014
Children of Shanghai cleaners better at math than kids of Israeli lawyers—Haaretz (Israel), 2-19-2014
Cleaners’ children in China beat kids of US, UK professionals at maths: study—NDTV (India), 2-18-2014
Children of Chinese janitors outscore wealthy Canadians in global exams—The Globe and Mail (Canada), 2-19- 2014

These are some of the most recent sensational headlines generated by PISA with a 4-page report entitled Do parents’ occupations have an impact on student performance released in February 2014. These headlines exemplify the secret of PISA’s great success as a masterful illusionist: effective misdirection of attention by exploiting human instinct for competition.

From the start, the entire PISA enterprise has been designed to capitalize on the intense nationalistic concern for global competitiveness by inducing strong emotional responses from the unsuspecting public, gullible politicians, and sensation-seeking media. Virtually all PISA products, particularly its signature product—the league tables, are intended to show winners and losers, in not only educational policies and practices of the past, but more important, in capacity for global competition in the future. While this approach has made PISA an extremely successful global enterprise, it has misled the world down a path of self-destruction, resulting in irrational policies and practices that are more likely to squander precious resources and opportunities than enhancing capacity for future prosperity.

Take the report on the relationship between parents’ occupations and student achievement as an example, instead of highlighting its main finding—the well-known fact that family background has a strong impact on student achievement—to call attention to the issue of social-economic inequity in the world, the report chose to dramatize the finding that students of parents in “elementary occupations” in Shanghai and Singapore outperform children of professionals in the U.K. and U.S. in mathematics. In so doing, the report commits a basic logic fallacy called misleading vividness: “a small number of dramatic and vivid events are taken to outweigh a significant amount of statistical evidence.”[1] The result is the illogical and incoherent “bottom line” given by PISA:[2]

While there is a strong relationship between parents’ occupations and student performance, the fact that students in some education systems, regardless of what their parents do for a living, outperform children of professionals in other countries shows that it is possible to provide children of factory workers the same high-quality education opportunities that children of lawyers and doctors enjoy.

Unlike what the report claims, the fact that “students in some education systems, regardless of what their parents do for a living, outperform children of professionals in other countries” cannot be used to show “that it is possible to provide children of factory workers the same high-quality education opportunities that children of lawyers and doctors enjoy.” It only shows that the children of factory workers in some education systems received higher PISA scores than children of doctors and lawyers in other systems. Since higher PISA scores do not necessarily mean higher quality education opportunities, the causal link cannot be established. More important, even such a link were empirically verified, the fact does not say anything about whether children of factor workers receive the same high quality education as their peers in the same education system, belying the conclusion that “it is possible to provide children of factory workers the same high-quality education opportunities that children of lawyers and doctors enjoy.”

The data actually show the contrary. In both China and Singapore, children of manual labor workers scored way below children of professionals and managers. The gap in PISA scores between students whose parents in professional occupations and those in manual occupations is actually bigger in Shanghai than in the U.K. and U.S.. The gap in math scores between children of professionals and children of agricultural workers is well over 100 points in Shanghai but below 100 points in the U.K. and U.S., according to PISA’s own analysis[3]. The same is true for Singapore.

Worse yet, the data are not even reliable.

Over a quarter of students in Shanghai were excluded

Tom Loveless of Brookings Institute put forth a powerful challenge to the Shanghai sample, questioning the inclusion, or rather exclusion of children of migrant workers in the PISA sample. PISA, after initial denial, finally admitted that more than a quarter of Shanghai students were not covered by PISA. This is much higher than what PISA’s sampling standards permit, which is “up to a total of 5% of the relevant population.” As a result, Shanghai should be excluded from any analysis, as it did to several other countries such as Austria for sampling errors.

So many managers, so few cleaners

As a result of the sampling problem, the Shanghai sample is incredibly distorted. The distribution of parents’ professions of students who participated in PISA in no way represents the reality. According to data released by PISA, a total of 5177 students in Shanghai took the test. A total of 9031 parental jobs were reported, indicating that most students have two working parents. What immediately stands out is the contrast between the percentages of parents who are “managers” and parents who are in “elementary occupations:” 16% vs. 11% (Figure 1). In other words, more students have parents working in “manager” jobs than holding “elementary occupations” such as cleaners. By the way, only 165 out of the 9013 parents in Shanghai are cleaners and helpers. That is about 80 out of the over 5,000 students who took the PISA test.

If the sample reflects the reality, this would mean more people in Shanghai hold “manager” jobs than “elementary occupations” or managers produce more children than those in elementary occupations. Both are untrue.

According to China’s census data, managers only made up about 5% of all jobs in 2010 in Shanghai (Figure 2). The Census data did not use the same categorizations as PISA so did not have the “elementary occupation” group. But over 60% of people worked in manual labor jobs such as services/sales, plant/machine operators and drivers, and agricultural workers. Adding the 13% clerical workers, about 80% of people in Shanghai were in jobs not with high social and economical status. This group is significantly underrepresented in the PISA sample, which included a lot more students whose parents hold jobs with high social economic status.

Figure 1: Distribution of PISA Parents’ Occupations: Shanghai (Source: PISA)

Figure 2: Distribution of Occupations in Shanghai (Source: China 6th Census)

It is unlikely that managers and professionals produce more children either because of China’s one-child policy. If anything, people in manual labor jobs may be more likely to have more than one child because they could pay a fine to get away with it, while managers and professionals would lose their jobs for having an additional child.

Distorted samples produce distorted results. Another country where students with parents in “elementary occupations” beat children of professionals in the U.S. and the U.K., Singapore, also has a high proportion of managers, professionals, and technicians (48%) and low percentage (18%) in “elementary occupations” (Figure 3). The U.S. and U.K. samples have 30% managers and professionals and 20% (UK) and 24% (US) “elementary occupation” holders (Figures 4 & 5).

Figure 3. Distribution of PISA Parents’ Occupations: Singapore (Source: PISA)

Figure 4. Distribution of PISA Parents’ Occupations: U.K. (Source: PISA)

Figure 5. Distribution of PISA Parents’ Occupations: U.S. (Source: PISA)

The smart children of some cleaners

The small number of children of manual laborers included in the PISA sample are either lucky or good at taking tests. Given PISA’s two-staged stratified sampling design, which first selects 150 schools randomly from “a comprehensive national [in the case of Shanghai, municipal] list all PISA-eligible schools”[4] and then randomly selects 35 students, the small proportion of students with parents in manual labor occupations and the large percentage of managers suggest three possibilities. First, the sampled schools have a small number of students with parents in manual labor occupations and large number in managerial jobs, assuming all sampled schools have similar student population in terms of parental occupations. Second, the sampled schools included more schools that primarily serve children of managers, professionals, and technicians than schools with a high concentration of children with parents in manual labor occupations, assuming that schools serve different types of children in terms of their family background. The third possibility is a combination of both possibilities, which is perhaps what happened.

The PISA data show that the 11% of children whose parents with “elementary occupations” come from 152 of the 155 sampled schools and the 1% of students whose parents with agricultural jobs are from 44 schools. This means children of elementary occupation parents are more integrated in schools with children whose parents are professionals and managers, hence likely better schools. They could be in these schools in three ways. First, they happen to reside in the district (very likely). Second, their parents have the wealth or social connections to enroll them (quite unlikely). Third, they were very good at taking tests and had been recruited by the schools (quite possible). The third group would be the one to score very well on PISA and may raise the average scores of this category of students. It is of course, also possible, that because these schools are so excellent that they all got great education. But the data seem to show otherwise. The variation in math scores among the children of parents with “elementary occupations” is much bigger than those with professional and manager parents[5]. In other words, a portion of the children of parents with elementary occupations may have been selected for their academic achievement in the first place and thus boosting the PISA scores, while the others remain undereducated, even in the Chinese way.

So what if they scored higher

Even if PISA had done everything properly and indeed children of factory workers in Shanghai scored better than children of lawyers in the U.K. and the U.S., it does not necessarily mean they are better educated or prepared for the modern society, considering the limitation of PISA test scores as I discussed in Part 3 of this series. It could mean something entirely different: while PISA scores can be achieved with little resources and intense repetition of narrowly defined, uniformly prescribed content and skills, what truly matters—talent diversity, creativity, and entrepreneurialism—cannot. The multiplication table can be learned with a piece of paper, but it would be difficult to force anyone to play the piano without a piano. Everyone can be forced to memorize the Hamlet, but it is unlikely to force anyone to invent the iPhone.

As I wrote in The Difference between a $10,000 Education and a $10 Education three years ago:

…America’s rich kids’ poor performance in math and science and relatively good performance in reading is at least partially the result of choice—choice made by students, parents, schools, and communities. Rightly or wrongly, Americans may be more likely than others to believe that the academic abilities measured by PISA is not the only path to success in life. American schools also create more choices through a broader curriculum. America also has a more diverse set of higher education institutions that, unlike colleges in other countries, use a more diverse set of indicators beside test scores to admit students. There are also many Americans who believe they can succeed without going to college. In America there are also very successful entrepreneurs, entertainers, athletes, and other celebrities who did not necessarily have a great academic background, which can inspire many to pursue non-academic enterprises. And until very recently, there are no centralized national curriculum,standards, and high-stakes testing to push American schools, teachers, and students to narrow the options to what PISA measures.

…

The availability of choices in America is an envy of many countries. Having the option to choose different paths is not cheap or easily available. It is an advantage, a privilege, not a disadvantage or liability. A $10,000 education can and should buy much more than test scores while a $10 education can only buy test scores. Why do we want to turn a $10,000 education into one that can be achieved with $10?

The bottom line: Until OECD-PISA became the only employer in the world with PISA scores as the only qualification, I would not suggest lawyers and doctors in the U.S., U.K., or any nation to replace your children’s activities in music, arts, sports, dancing, debates, and field trips with math tutoring. For the same reason, it is not time yet for schools in developed countries to close your swimming pools, burn your musical instruments, end museums visits, or fire your art teachers.

Part 1: Romanticizing Misery

Part 2: Glorifying Educational Authoritarianism

Part 3: Illusory Models of Excellence

Adapted from my book Who’s Afraid of the Big Bad Dragon: Why China has the Best (and worst) Education in the World.

[1] http://www.logicallyfallacious.com/index.php/logical-fallacies/126-misleading-vividness

[2] http://www.oecd.org/pisa/pisaproducts/pisainfocus/PISA-in-Focus-N36-(eng)-FINAL.pdf

[3] http://beta.icm.edu.pl/PISAoccupations2012/OccupationsPISA2012.pdf

[4] http://www.oecd.org/pisa/pisaproducts/50036771.pdf, p. 58.

[5] http://beta.icm.edu.pl/PISAoccupations2012/OccupationsPISA2012.pdf

Share

(No Ratings Yet)

Loading...

7 Comments »

How Does PISA Put the World at Risk (Part 4): Misleading the World | Educational Policy Information said:

[…] YongZhao China’s poorest beat our best pupils—The Telegraph (UK), 2-17-2014 Children of Shanghai […]

# 29 March 2014 at 3:13 pm

Some Released PISA Questions | GFBrandenburg's Blog said:

[…] Zhao and some other commentators have been criticising PISA for a number of reasons, one being that its sample populations are at times ‘gamed’ by two cities (Shanghai and […]

# 8 April 2014 at 11:09 pm

Yong Zhao: How PISA Misleads the World | Diane Ravitch's blog said:

[…] this fourth installment in his series of posts criticizing PISA, Yong Zhao examines the claim that low-income children in China […]

# 9 April 2014 at 4:00 am

Yong Zhao: How PISA Misleads the World | Educational Policy Information said:

[…] this fourth installment in his series of posts criticizing PISA, Yong Zhao examines the claim that low-income children in China […]

# 9 April 2014 at 8:00 am

Florida’s Testing Freaks | Educational Policy Information said:

[…] THEN THERE’S PISA… Yong Zhao tells us how the international testing competition race to the bottom has pushed China off the deep end. His […]

# 9 April 2014 at 9:01 am

#Education Week in Review: PISA-OECD, Standardized Testing | Longacre said:

[…] Does PISA Put the World at Risk.” Last week he released Part 4 of the series, “Misleading the World.” (His argument is long.) Essentially, he says that students in the U.S. are learning very […]

# 9 April 2014 at 2:38 pm

Articles of Interest: April 4th, 2014 « National Creativity Network said:

[…] How Does PISA Put the World at Risk (Part 4): Misleading the World Education in the Age of Globalization […]

# 3 July 2014 at 3:59 pm

Feel free to comment:

The views expressed on this site are entirely my own. They do not represent my employer or any other organization/institution. All comments are subject to approval.

Silverliningforlearning

Most Commented

Most Viewed

A Discussion on China’s Recent Education Reform on CCTV 9 - 596,025 views
New article: Build back better: Avoid the learning loss trap - 417,948 views
You must be joking, Professor Chua: An open letter to the Chinese Tiger Mom - 302,365 views
Common Sense Vs. Common Core: How to Minimize the Damages of the Common Core - 283,231 views
Five Questions to Ask about the Common Core* - 266,566 views

Mega888 Online Casino Malaysia Duitnow Slot Ewallet Casino TNG

Intisari Utama

Pengenalan Kepada Platform Mega888 Malaysia

Sejarah Mega888 di Malaysia

Ciri-ciri Utama Platform

Lesen dan Keselamatan

Cara Mendaftar Akaun Mega888

Sistem Pembayaran DuitNow di Mega888

Kelebihan Menggunakan DuitNow

Langkah-langkah Transaksi DuitNow

slot ewallet mega888 TNG

Permainan Popular di Mega888

Slot Games Terbaik

Live Casino Games

Panduan Deposit Menggunakan E-wallet

Kelebihan Menggunakan E-wallet untuk Deposit

Langkah-langkah Deposit Menggunakan E-wallet

Keselamatan Transaksi di Mega888

Teknologi Enkripsi

Sistem Verifikasi

Bonus dan Promosi Mega888

Panduan Bermain Slot Online

Tips Menang

Strategi Pengurusan Modal

Aplikasi Mega888 Mobile

Ciri-ciri Utama Aplikasi Mega888 Mobile

Perbandingan E-wallet di Mega888

Touch 'n Go

GrabPay

Boost

Sokongan Pelanggan 24/7

Tips Keselamatan Akaun

Trend Terkini Casino Online Malaysia

Perkembangan Teknologi

Perubahan Industri

Panduan Pengeluaran Wang

Langkah-langkah Pengeluaran Wang

Had Pengeluaran

Tempoh Pemprosesan

Kesimpulan

FAQ

Apakah Mega888 dan bagaimana ia beroperasi di Malaysia?

Bagaimana saya boleh mendaftar akaun Mega888?

Apakah kelebihan menggunakan DuitNow sebagai kaedah pembayaran di Mega888?

Bagaimana saya boleh menggunakan e-wallet TNG untuk bermain slot di Mega888?

Apakah permainan slot dan kasino langsung yang paling popular di Mega888?

Bagaimana saya boleh membuat deposit di Mega888 menggunakan e-wallet?

Bagaimana Mega888 melindungi keselamatan transaksi pengguna?

Apakah bonus dan promosi yang ditawarkan oleh Mega888?

Bagaimana saya boleh bermain slot online dengan selamat dan berkesan di Mega888?

Apakah kelebihan menggunakan aplikasi Mega888 berbanding platform web?

How Does PISA Put the World at Risk (Part 4): Misleading the World

7 Comments »

Feel free to comment:

Archive

Tag Cloud

Silverliningforlearning

Most Commented

Most Viewed