How to Avoid Misunderstanding, Misinterpreting and Distorting Data
"... should be required reading in teacher preparation and administrative leadership programs."
"Education suffers from an overabundance of well-meaning B.S. and a worrisome shortage of useful B.S. detectors. In this accessible volume, through case studies and clear prose, Steve Rees and Jill Wynns provide leaders a better way to gauge how schools are doing."
"While it may be frightening to learn that the data being used and frequently misinterpreted to determine the quality of a school is flawed, take heart. Rees and Wynns have provided a path to improvement, if not enlightenment."
Why mismeasurement matters
So many numbers in so many hands for so many years … Making sense of it all has led to many mistakes. With this book, we have turned those errors in human judgment into opportunities to learn.
When we misjudge schools’ vital signs, someone gets hurt
District leaders
Reputations may get damaged. Leaders get blamed or praised for the wrong reasons. Bad policies cause harm no one sees.
District leaders
Principals
Schools’ annual plans set the wrong priorities. Principals misread instructional strengths. Students get misidentified.
Principals
School board trustees
Budgets shift resources in the wrong direction. Superintendents are evaluated unfairly. Weak strategic plans get approved.
School board trustees
Students
Likely to be misidentified when their test results are misunderstood. May be denied help that they really need, or get help they don't need.
Students
Parents
May choose schools for their kids based on faulty evidence. May take accountability ratings too literally.
Parents
Advocates
May misinterpret the evidence of discipline gaps, while missing the evidence of individual bias in grading.
Advocates
“Educators and parents are awash in data, but how can they correctly use it? This thoughtful book points out ways in which education data are misused and offers concrete guidance to do better. Any program preparing teachers or school leaders … could benefit from reading this candid, approachable book.”
Morgan Polikoff, Associate Professor of Education at University of Southern California
What the book covers
Mismeasuring Schools’ Vital Signs
Table of Contents
- Chapter 1: What is mismeasurement and how does it occur?
- Chapter 2: Twisting test results and missing sound evidence of learning
- Chapter 3: The hidden hazards of interpreting graduation rates
- Chapter 4: Gaps mismeasured, misattributed and misunderstood
- Chapter 5: Logic errors when identifying and evaluating English learners
- Chapter 6: Mistaken ways of measuring money, buildings and people
- Chapter 7: Reducing barriers to progress at every level
“This book should be read by every school board member, superintendent, school principal, education reporter, and advocate. Rees and Wynns have spent decades wrestling with the measurement of school quality, and they understand the mistakes that have so often misled us. In clear and concise prose … they identify smart measurement methods that will help our state, district, and school leaders avoid those all-too-common errors.”
David Osborne, Author, Reinventing America’s Schools and Reinventing Government
Why we wrote this book
For the several decades the two of us have been in the world of schooling, we’ve seen schools’ and districts’ vital signs misunderstood far too often. These misperceptions are rarely the result of deception. Rarely are they the result of accountability pressure. Rather, they are the result of errors in judgment. We think it’s time to dive into those mistakes, untangle their causes, and learn from them. This book, then, is decidedly not another cookbook of best practices. It is a collection of stories and case studies of human errors, which we believe are rich opportunities to learn.
“A timely book about the importance of the use of data, facts, and evidence in education. It raised thoughtful questions about how data should be used for what purposes and the ensuing interpretations.”
Ellen Mandinach, Senior Research Scientist at WestEd, author
