Not an Anomaly: Systemic Ills Caused by Test-based Accountability Policies

Secretary Duncan is not the only who tries to minimize the scale of the problem and reduce it to a technical issue. Chester E. Finn, a senior fellow at Stanford’s Hoover Institution and president of the Thomas B. Fordham Institute. He is a former assistant secretary at the U.S. Department of Education under President Ronald Reagan, tries to do same. In his essay entitled Don’t ditch testing after Atlanta cheating, boost test security and published as a CNN special on July 13, argues that cheating is:

A problem, indeed, and one worth solving, but not an argument against testing as a key element in learning about how students are doing and holding educators and schools accountable.

Finn attempts to make cheating in schools an issue of human nature and minimize its scale to a few isolated cases saying that: “Regrettably, this is about human nature, not about the immediate example of test-score cheating in a few hundred of our nearly 100,000 schools.” He likens cheating on tests to “tax cheating, Medicare fraud, pleading innocent when one is guilty; professors plagiarizing and medical researchers falsifying their data; and on and on.”

Essentially, testing proponents would do anything but acknowledge the fact that the testing-driven education policy is the root cause of cheating and its consequent damages to children. They try to make it an anomaly, a small problem caused by a few unethical individuals, and a technical issue that can be addressed with simple solutions. However, evidence suggests just opposite.

Evidence of Blatant Cheating Practices

We may never know exactly how many schools or educators cheat on standardized tests simply because we cannot afford to audit all schools and those that cheat are unlikely to report or confess. But publically available reports unambiguously reveal that cheating is not an anomaly in our schools—it is not isolated to Atlanta and it is not only “a few hundred of our nearly 100,000 schools.”

Just recent media reports of test-score cheating by adults make the number of schools way more than a few hundred. In a March, 2011 post entitled Testing Anomalies Found in Many States on the U.S. News and World Report website by Jason Koebler reports that hundreds of schools in Washington DC, Georgia, Arizona, Detroit, Baltimore, and several other states under investigation for testing irregularities. Also in March, 2011, a USA Today investigation found “1,610 examples of anomalies in which public school classes — a school’s entire fifth grade, for example — boasted what analysts regard as statistically rare, perhaps suspect, gains on state tests” in DC and each of the six states they looked at– Arizona, California, Colorado, Florida, Michigan and Ohio. Last week, the Notebook reported that 225 schools were flagged in the 2009 for “erasure analysis” in Pennsylvania.

In their 2007 book Collateral Damage: How High-stakes testing Corrupts America’s Schools, Sharon Nichols and David Berliner report that 9% of teachers survey in Tennessee said they had witnessed test impropriety on the state’s tests and nationally, a survey found:

• 10% of teachers admitted to providing answers during tests
• 10% of teachers pointed out mismarked items by students
• 15% of teachers gave students more time to finish the test than allowed
• 5% gave instructions during the test

The New York Times reported in May, 2011 that “an unusually large number of students have obtained exactly the minimum score needed to pass the exams, which are required for graduation and are often graded by students’ own teachers.” An investigation by the New York Times found that students attending New York City’s public high schools “had been roughly five times as likely to score 65, the passing grade, or slightly above it, than to score just below it… But even on the algebra exam, in which there are no essays, 8,451 students got grades of exactly 65, while a combined 7,145 students ended up with a score of 61, 62, 63 or 64. Statisticians say that such a difference is out of line with the smooth scoring curve that should normally result.”

I don’t know what percentage would make cheating not an anomaly in the minds of testing proponents, but “it seems readily obvious that teachers and administrators are often engaged in test-related impropriety,” write Nichols and Berliner in their book.

Evidence of Softer, More Acceptable Forms of Cheating

While telling students the correct answers, changing student test scores, and directly changing student answers are forms of blatant, direct, hardcore cheating practices that are considered unethical and illegal, there are other forms of practices that may not be viewed as cheating on the surface but in reality they are. They cheat students out of a real valuable education and cause as much damage, if not more, to our children as the behaviors we label cheating.

Teaching to the tests and test preparation

Many schools in the U.S. have turned into test preparation institutions. They only teach what is on the high-stakes tests. A study by the Center on Education Policy (CEP) published in 2007 found that five years after the implementation of NCLB, over 60% of school districts reported that they have increased instructional time for math and English language arts, while 44% reported that they have reduced time for other subjects or activities such as social studies, science, art and music, physical education, lunch and or recess.  The study also found that most school districts have narrowed their English language arts and math curricula to what is covered on the state tests. The study found that 84% of districts reported that they have changed their curriculum “somewhat” or “to a great extent” to put greater emphasis on tested content in elementary level reading; 79% in middle school, and 76% in high school. A similar pattern was found in math: 81% of districts have changed their curriculum at the elementary and middle school level to emphasize tested content and skills, and 78% in high school math. Classroom instruction has also been transformed into test preparation. Linda Valli and her colleagues found that since the implementation of NCLB, teachers have lost curriculum and pedagogical autonomy to standards and testing. “Teachers felt compelled to match closely what they taught to what would be tested and worried about how well aligned the district curriculum was with state test’s content, language, and format” (Valli & Buese, 2007, p. 531). A more recent study by CEP of the impact of federal and state accountability polices on curriculum and instruction in three states, Rhode Island, Illinois, and Washington, found classroom instruction to be focused on test preparation and that teachers generally focus their instruction on test-related content.

Cheating by Schools and States

It is well known that schools and states manipulate student performance data in a number of ways. For schools, excluding certain students from testing or even discouraging certain students from attending the school have been reported in various places. A 2005 article by Lisa Snell reports that in 2004, a high school in Florida boosted its test scores from an F to a D after “purging” 126 low-performing students from its attendance rolls. In the same year, “some 160 Florida schools assigned students to new schools just before standardized testing in a shell game to raise school grades.” In a third of Houston’s 30 high schools, scores on standardized exams have risen as enrollment has shrunk. In 2011, a school principal in DeKalb County, GA sent a letter to the parents of 13 students advising “that they would be withdrawn due to poor attendance, which would cause the school not to make AYP.” A 2007 Time magazine article recounts how one top-performing school, i.e. a school with high standardized test scores, forced or discouraged disadvantaged students to leave the school in order to close the achievement gap. In this story, an African-American student was pushed out after multiple “disciplinary suspensions” but the story tells that was perhaps merely a way to push certain group of students out to help the school retain its top-performing reputation without being labeled “failing” under NCLB.

At the state level, the manipulation of test results has happened frequently since the implementation of NCLB. One of the activities can be adjusting the “cut-scores” on standardized tests used to define different proficiency levels. Because NCLB holds schools and states accountable for increasing the percentage of students achieving a level of proficiency, states have been found to change their cut scores and lower their standards. A federal study in 2009 found that “nearly a third of the states lowered their academic proficiency standards in recent years, a step that helps schools stay ahead of sanctions under” NCLB.

Discrepancy in Achievements—Collateral Evidence

Since NCLB, test scores on state high-stake tests have been reported to rise, sometimes, dramatically, but such gains have not been generalized to other assessments, according to a recent report by the nation’s education experts commissioned by the National Research Council. The panel of the nation’s leading experts found that “When the [test-based accountability] systems are evaluated—not using the high-stakes tests subject to inflation, but using instead outside comparison tests, such as the NAEP—student achievement gains dwindle to about .08 of a standard deviation on average, mostly clustered in elementary-grade mathematics.”  This discrepancy suggests high-stakes test scores are inflated and do not indicate true education gains. Such inflation can come from multiple sources, most of which are not sound, honest, and valuable education practices that help improve the children’s learning.

To summarize, cheating in American schools is not isolated as testing proponents suggest. We cannot simply blame it on a few unethical educators. Instead, we must acknowledge the fact that it is widespread, in multiple forms, in the nation’s schools and it is a direct result of the test-based accountability policies so we can begin to change course and ditch testing.

In the next post, I will discuss why cheating is not a simple issue that can be fixed with technical measures, but rather it is a cultural and psychological issue that can only be fixed by removing test-based accountability from our schools to extinguish the motivation for cheating or the “trigger” of the “cheating gene.”

Part 1

Part 3

Part 4

Part 5