Ed Deform Follies: Yes, You Get What You Measure, or Be Careful What You Wish For

Orwell_video_game_posterApril 30, 2019

Today, Diane Ravitch posted two  important pieces on her blog. One is on Ken Robinson’s book about the egregious consequences of high-stakes standardized testing. The other is about the Stalinesque workings of the “justice” system in a city in Georgia where innocent people are arrested and given the choice of a) pleading guilty to the trumped-up charges and getting a fine and a slap on the wrist or b) pleading not guilty and risking being jailed for decades under false charges.
These two phenomena are related, and I would like to explain how. I hope you will stick with me here, because I’m going to present an argument, and while it’s not a difficult or complex argument, it will take some attending to.
Ed Deformers based their “data driven accountability” systems on a key idea, which we might call the Fundamental Axiom of Ed Deform:
You get what you measure.

The unelected monarch of Ed Deform in the United States is Bill Gates. Find a shill or shell organization (or a piece of federal legislation) that promotes high-stakes standardized testing, evaluating teachers and schools based on their test scores, or promoting “common standards” to make it easier to prepare and sell tests and educational software “at scale,” and the probability is quite high that the Gates Foundation will be its primary funder or, at least, one of its primary funders. All these Ed Deforms just happen, by the way, to involve buying and using a lot of computers and computer software–taking standardized tests on computers, using depersonalized education software, vast databases of cradle-to-grave information about students and citizens. Gates has spent many, many millions promoting this stuff. For instance, he paid to have a guy with no relevant experience, David Coleman, create the Common [sic] Core [sic], on which the high-stakes standardized computer tests are based.

So, Ed Deform is all about computers, and Gates just happens to be a computer guy. Such coincidence! Such synchronicity!

As a computer guy (and an avid reader of serious books), Gates is doubtless familiar with the triple bar (≡) from symbolic logic, which means “if and only if” and is the symbol for material equivalence. What experience has shown is that foundational principle on which Ed Deform is based ought to be rewritten as

Outcome ≡ measurement

In other words, in high-stakes, data-managed systems,

You get what you measure AND ONLY WHAT YOU MEASURE.

The last part of this (“and only what you measure”) is REALLY important because IT CHANGES EVERYTHING.

The Common [sic] Core [sic] State [sic] Standards [sic] (CCSS) make lip service to students reading substantive works of literature. But the standards [sic] themselves consist entirely of statements of “skills,” and these AND ONLY THESE are  measured on the high-stakes standardized tests.

Furthermore, they are measured in a particular way: students are given a snippet of random text and are asked a question that requires them to “apply the standard” to the text. So, that’s what you get: you get the devolution of education in ELA into depersonalized learning software and textbooks and worksheets, print or online, that present students with random snippets of text and ask them to apply the skills standards to these. In other words, you get this vast distorting or devolution of ELA pedagogy and curricula. Today, shortly after I drafted this post, I heard from yet another friend in educational publishing that she was quitting because she was sick of this Ed Deform-driven dumbing down of curricula.

Gates occasionally gives breezy interviews in which he talks about the “experiment” he is conducting with his “higher standards” and how he will know in a decade or two whether they will be successful. And this is what people who don’t know what they are doing but are incredibly sure of themselves do and say. They rush in, for their own reasons, where angels fear to tread, and they make a mess of things. Meanwhile, every educational publisher in the United States now starts the planning of every project by making a list of the Gates/Coleman standards [sic] in the far-left column of an Excel spreadsheet and, in the next column over, a list of the places in the product where the standard [sic] will be “covered.” In other words, the standards [sic] have become the default curriculum. They, and only they, are what is taught and what matters. In the past, when states had competing standards, the publishers worked in this way instead: they created a coherent product–a survey text in British literature for 12th graders, for example, and then “correlated” it with all those competing standards. The correlations were often quite fanciful, but the product was coherent and and unified and substantive. But now, it has been narrowed to teach the standards and only the standards, and bright folks like my friend, who care deeply about kids and the teaching of English, quit their jobs in disgust. So, Mr. Gates’s actions have these very real consequences–they narrow the curriculum and destroy careers that would have been spent doing good in the world.

In real life, when you read Orwell’s 1984 or the Constitution of the United States, you don’t read it primarily to apply a skill from Lord Coleman’s list to it. You read it because the experience is so moving as to be unforgettable and because you are interested in the important things that Orwell or the founders had to say and because you want to join in the ongoing cultural discussion of these. So, the approach forced upon us by Gates and Coleman is utterly unnatural. It leaves all that out. Here’s a little experiment you can try: read Melinda Gates’s new book, The Moment of Lift, and then write a review of it. However, limit your review to discussing how, in the book, Ms. Gates’s choice of particular words affected the tone and mood of her piece and to giving examples of this (CCSS.ELA-LITERACY-RI.8.4). Then see if you care at all to write such a piece or if anyone would care at all to read it.

In short, the Gates/Coleman approach, which I call New Criticism Lite, leads to completely unnatural InstaWriting and InstaThinking and reduces the process of reading and responding to literary and other written works to a Procrustean, highly constrained exercise IN TRIVIALITIES. The whole reason for reading and writing—the commerce in ideas and experiences–is devalued or lost. It’s not going to be on the test.

But there is another modification we need to make in the Fundamental Axiom of Ed Deform to reflect how it plays out in practice. Data-based accountability, Ed Deform style, depends upon the high stakes. You don’t get what you measure and only what you measure in the absence of high stakes accorded that measurement. In other words, you need violence and the threat of violence: give me what I am measuring (and only what I am measuring), or you will be fired, your school will be closed and replaced by a charter, or your student will not graduate or be advanced to the fourth grade. So, the fundamental tenet of Ed Deform, revised:

You get what you measure and only what you measure if you threaten state violence if you don’t get it.

So, what does the Fundamental Axiom of Ed Deform have to do with justice as practiced in that small town in Georgia? Well, the same axiom is at work. Increasingly in the United States, the criminal justice system is data-driven, and what is measured are guilty pleas and convictions, not whether justice was served. And so, under threat of state violence, you get what you measure and only what you measure—more guilty pleas and more convictions. That’s why the “land of liberty” now has a higher percentage of its citizens under penal supervision (in jail, in prison, or on parole) than does any other country in the world (almost 3 percent of the adult population). Think of the sickest, most repressive regime on the planet. We in “the land of liberty” imprison people at a rate greater than it does. There are, today, more black men under penal supervision in the US than there were black men who were slaves in 1858. We are becoming one nation, accountable to data, like something out of Orwell or Kafka. Glory be to data in the highest. In data we trust. Out of many, data. We hold this truth to be self-evident, that all people are reducible to data.

So, what sounds on the surface of it like a good idea (“We need real data-based accountability!”) ends up having these horrific consequences in the lives of actual people. Little Yolanda hates reading because she thinks it’s a guessing game in which you find which of the four tortured sentences in the multiple-choice question about standard CCSS. ELA-LITERACY.RI.666 is “most correct,” and then she grows up to be unable to get a job when she’s 30 because she was caught when she was 18 with her boyfriend’s roach clip in her pocket. She’s held accountable, forever. When she was 18, she went to a party, and she met the wrong guy, and so, of course, her life should be ruined going forward.

The “data,” ofc, from those standardized tests is garbage. The tests don’t measure what they purport to measure. They are invalid and unreliable. You can’t draw from them the conclusions that the Ed Deformers draw, and you certainly can’t rationally base important educational decisions on them. But none of this matters. It doesn’t matter that all this data-based decision making is just so much numerology. It sells computers and computer systems, and it’s an effective means of command and control. And that was the point of it.

So, Ed Deform has a lot of similarities to the Eugenics Movement of the early 20th century. It’s a pseudoscience, based on bad data and poor statistical reasoning, from which false inferences are drawn that lead to horrific social consequences. A great book could, in fact, be written on the parallels between these two periods of madness in our history–the Era of Eugenics and the Era of Ed Deform.

Those who don’t learn from history are forced to repeat it.

“If you want a vision of the future, imagine a boot stamping on a human face – forever.” –George Orwell, 1984

Art: Orwell video game artwork/poster, by Osmotic Studios, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=52585671


For more pieces on Ed Deform by Bob Shepherd, go here: https://bobshepherdonline.wordpress.com/category/ed-reform/


About Bob Shepherd

interests: curriculum design, educational technology, learning, linguistics, hermeneutics, rhetoric, philosophy (Continental philosophy, Existentialism, metaphysics, philosophy of language, philosophy of mind, epistemology, ethics), classical and jazz guitar, poetry, the short story, archaeology and cultural anthropology, history of religion, prehistory, veganism, sustainability, Anglo-Saxon literature and language, systems for emergent quality control, heuristics for innovation
This entry was posted in Ed Reform, Uncategorized. Bookmark the permalink.

4 Responses to Ed Deform Follies: Yes, You Get What You Measure, or Be Careful What You Wish For

  1. “You get what you measure AND ONLY WHAT YOU MEASURE.”

    And if you are measuring NOTHING?

    Which is what standardized testing “measures”.

    The most misleading concept/term in education is “measuring student achievement” or “measuring student learning”. The concept has been misleading educators into deluding themselves that the teaching and learning process can be analyzed/assessed using “scientific” methods which are actually pseudo-scientific at best and at worst a complete bastardization of rationo-logical thinking and language usage.

    There never has been and never will be any “measuring” of the teaching and learning process and what each individual student learns in their schooling. There is and always has been assessing, evaluating, judging of what students learn but never a true “measuring” of it.

    But, but, but, you’re trying to tell me that the supposedly august and venerable APA, AERA and/or the NCME have been wrong for more than the last 50 years, disseminating falsehoods and chimeras??

    Who are you to question the authorities in testing???

    Yes, they have been wrong and I (and many others, Wilson, Hoffman etc. . . ) question those authorities and challenge them (or any of you other advocates of the malpractices that are standards and testing) to answer to the following onto-epistemological analysis:

    The TESTS MEASURE NOTHING, quite literally when you realize what is actually happening with them. Richard Phelps, a staunch standardized test proponent (he has written at least two books defending the standardized testing malpractices) in the introduction to “Correcting Fallacies About Educational and Psychological Testing” unwittingly lets the cat out of the bag with this statement:
    “Physical tests, such as those conducted by engineers, can be standardized, of course [why of course of course], but in this volume , we focus on the measurement of latent (i.e., nonobservable) mental, and not physical, traits.” [my addition]

    Notice how he is trying to assert by proximity that educational standardized testing and the testing done by engineers are basically the same, in other words a “truly scientific endeavor”. The same by proximity is not a good rhetorical/debating technique.

    Since there is no agreement on a standard unit of learning, there is no exemplar of that standard unit and there is no measuring device calibrated against said non-existent standard unit, how is it possible to “measure the nonobservable”?

    THE TESTS MEASURE NOTHING for how is it possible to “measure” the nonobservable with a non-existing measuring device that is not calibrated against a non-existing standard unit of learning?????


    The basic fallacy of this is the confusing and conflating metrological (metrology is the scientific study of measurement) measuring and measuring that connotes assessing, evaluating and judging. The two meanings are not the same and confusing and conflating them is a very easy way to make it appear that standards and standardized testing are “scientific endeavors”-objective and not subjective like assessing, evaluating and judging.

    Those supposedly objective results are used to justify discrimination against many students for their life circumstances and inherent intellectual traits.

    Liked by 1 person

  2. “So, Ed Deform has a lot of similarities to the Eugenics Movement of the early 20th century. It’s a pseudoscience, based on bad data and poor statistical reasoning, from which false inferences are drawn that lead to horrific social consequences. A great book could, in fact, be written on the parallels between these two periods of madness in our history–the Era of Eugenics and the Era of Ed Deform.”

    Excellent comparison!

    Liked by 1 person

  3. Roy Turrentine says:

    After a time I re-read this essay, and I focused on Little Yolanda. The girl who ruined her life with a roach clip. I grew up in an unintentionally forgiving society. There were few people willing to forgive the fast driver, or the unwise user of a hammer. A friend of mine was swinging his baseball bat at a game, warming up. Like a kid, he did not look around himself, and so beaned a passerby, or at least as much as a little boy can get that done. Next day there was nothing but talk about how his father had removed his belt and beaten the child. I hear he made a good marine.

    As unforgiving as the people were, the environmental was very forgiving. A young driver could be insane in his path down tiny dirt roads, and the consequences of his idiocy would barely ever surface. Usually, you were the only car. Unlike a circular saw, a handsaw rarely hurt the unwise user. If you threw something on the ground, you usually could depend on its being absorbed unnoticed into it naturally. If you went slightly outside of the law, as teens are wont to do, very little would happen. Kids will be kids.

    Even those who went far outside the law could leave and disappear into the big country. If Hugo’s Jean Valjean had lived in America, he would have showed up in California and never been heard from again by his vindictive pursuer.

    But data has changed all that. They can keep you in the sights. A nineteen year old caught with a seventeen year old gets on the sex registry. Drug convictions hang onto a person so much that even the republicans are trying to roll,back some of the earlier three strikes laws.

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s