CT News Junkie | OP-ED | Statistics and Liars

Social Networks We Use

Connecticut Network


Our Partners

OP-ED | Statistics and Liars

by | Jun 24, 2014 2:30pm
() Comments | Commenting has expired
Posted to: Education, Opinion


California’s teacher tenure laws were invalidated recently by Superior Court Judge Rolf Treu because the “statutes made it so onerous to fire bad teachers that they all but guaranteed needy kids would be stuck in classrooms with incompetent instructors.”

We’ve all heard “bad teacher” stories before. Critics often use the “bad teacher” scenario as Exhibit 1 in the Case Against Public Schools. Unfortunately, this focus on “bad teachers” and the tenure laws that supposedly prevent their dismissal have become a smokescreen for an even bigger problem: the blind worship of educational statistics.

Treu, for example, reinforced his decision by quoting the expert-witness account of Arizona State University professor David Berliner who testified that “1 to 3 percent of California teachers are grossly ineffective. Given that the evidence showed roughly 275,000 active teachers in the state, the extrapolated number of grossly ineffective teachers ranges from 2,750 to 8,250.”

Sounds horrific. It also sounds imprecise. What’s the source of these statistics?

“I pulled that out of thin air,” said Berliner. “There’s no data on that. Just a ballpark estimate, based on my visiting lots and lots of classrooms.”

Ballpark estimate?

Lots and lots of classrooms?

Welcome to the world of educational statistics.

Berliner is actually a critic of another notorious statistical contrivance in education, value-added models (VAMs): “These models require data that track individual students’ academic growth over several years and different subjects in order to estimate the contributions that teachers make to that growth.”

“[R]esearch suggests that teachers don’t really control much of how their pupils perform on exams,” writes Berliner. “According to the American Statistical Association (ASA), they influence anywhere between 1 percent and 14 percent of the variation in students’ scores. As result, teachers often don’t deliver the same results year after year.”

ASA’s official statement warns school districts that VAMs are “generally based on standardized test scores, and do not directly measure potential teacher contributions toward other student outcomes.” That is, “VAMs typically measure correlation, not causation: Effects — positive or negative — attributed to a teacher may actually be caused by other factors that are not captured in the model.”

Despite such criticism, value-added models are used in teacher-evaluation models because they provide a seemingly exact measurement of teacher performance. What’s not to like about specific numbers attached to specific teachers in order to categorize them as “good,” “bad,” or “grossly ineffective”?

I’ve explored this topic before, noting how the current fetish for educational statistics has its roots in “scientific management,” the system of workplace efficiency established by Frederick Winslow Taylor in the early 20th century through “time-and-motion studies.”

Taylor’s ideas spread beyond the industrial workplace and gained a sort of “worldview status” by which devotees could neatly measure, label, and evaluate anything and everything.

Neil Postman explained in his book, Technopoly: The Surrender of Culture to Technology, that Taylorism frames the world around assumptions: “These include the beliefs that the primary, if not only, goal of human labor and thought is efficiency; that technical calculation is in all respects superior to human judgment; that in fact human judgment cannot be trusted, because it is plagued by laxity, ambiguity, and unnecessary complexity; [and] that subjectivity is an obstacle to clear thinking.”

In short, Taylorism suggests “what cannot be measured either does not exist or is of no value; and that the affairs of citizens are best guided and conducted by experts,” according to Postman.

If teachers cannot be measured, therefore, they are of no value. Thus, we must somehow measure them to demarcate “valuable teachers” and “bad teachers.”

My own worldview is not so orderly. Where others see numbers and percentages attached to teacher classifications, I see ambiguity and manipulation. We’re talking about human beings, after all — not robots.

In my classroom, I often use music to enhance lessons. Billy Joel’s “Shades of Grey” seems fitting here:

“Now with the wisdom of years, I try to reason things out,
And the only people I fear are those who never have doubts.
Save us all from arrogant men, and all the causes they’re for.
I won’t be righteous again.
I’m not that sure anymore.”

Perhaps Judge Rolf Treu is confident enough with statistics “pulled out of thin air” to help him make a decision affecting millions of people.

Me? I’m not that sure anymore.

Barth Keck is an English teacher and assistant football coach who also teaches courses in journalism and media literacy at Haddam-Killingworth High School. .(JavaScript must be enabled to view this email address)

Tags: , , , , ,

Share this story with others.

Share | |


(3) Archived Comments

posted by: ocoandasoc | June 24, 2014  5:09pm

I have a few problems with this. First, the precise validity of the statistics in Treu’s written opinion do not affect the gist of the decision. The principle would be the same if the number of grossly ineffective teachers was a fraction of one percent. Second, it’s too bad that the teachers unions couldn’t refute Berliner’s “seat of the pants” statistics during the court arguments, but then they don’t admit to their being even ONE grossly ineffective teacher in the State, and furthermore, their position was that it wouldn’t make any difference in the constitutionality of teacher tenure laws anyway (and that opinion may very well hold on appeal). The teachers unions like to throw a huge smokescreen over the entire issue of teacher effectiveness. While almost every other organization of professionals has tests, standards, or measurements of effectiveness, the teachers unions consider anyone who pays their dues as equally qualified. (Though they’re not so sure about those non-union charter and parochial school instuctors!) Finally, I think scientific management has come a long way from the days of Frederick Winslow Taylor (who was a better tennis player than he was a writer – even his own professional association refused to publish his book!), since he’ll be dead a hundred years in 2015. Everything can and should be measured, be it quantitatively or qualitatively, and if the teachers themselves won’t agree on how their performance should be measured they should not be shocked to discover that others who find the diminishing returns on their educational investments troubling will try and do it themselves.

posted by: art vandelay | June 25, 2014  8:13pm

art vandelay

I think it all boils down to the fact that no person is entitled to a job for life. A teacher with tenure is the closest thing to it.

posted by: Barth Keck | June 26, 2014  8:39am

As always, I appreciate the comments. To coandasoc, I agree that Treu’s decision was about more than just statistics. That said, I disagree strongly that “everything can and should be measured.” Further, your point that “almost every other organization of professionals has tests, standards, or measurements of effectiveness” may be true, such standards were never used in my annual review during my six years in business—my first career choice out of college before I turned to teaching. In fact, my performance evaluations were about as subjective and statistically-blind as possible.

To Art Vandelay, I respond with this comment from, yes, a union president (David Cicarella of the New Haven Federation of Teachers): “The reason for the misunderstanding is because at the college level, tenure does provide guarantees of employment for professors.  Public school teachers through grade 12 enjoy no such protection. The term tenure is unfortunately shared and has created the ridiculous notion that teachers in Connecticut have some mythical and absolute job security.” (For some common-sense suggestions for improving teacher tenure laws, read the entire article here: http://www.newhavenindependent.org/index.php/archives/entry/what_do_you_mean_tenure/)

Teaching, actually, is a profession that is largely self-monitored, as 46% or more of new teachers quit the profession within five years. (See more in this Forbes article: http://www.forbes.com/sites/erikkain/2011/03/08/high-teacher-turnover-rates-are-a-big-problem-for-americas-public-schools/) Perhaps if California’s tenure law was changed so that tenure was not earned until the first day in the classroom after four years’ experience—as in Connecticut—the situation might improve.

Again, I appreciate the feedback and discussion.