Chapter 1

What is mismeasurement? It’s not an everyday term in the school world, after all. But it is commonly used in other fields and professions. When we use the term “mismeasurement,” we mean an error that occurs when gauging the quantity or quality of behavior, attitudes or other human events. Mismeasurement includes misunderstanding the size or meaning of something, which in turn may lead to errors in judgment. Given how much data educators have been collecting over the past several decades, it shouldn’t be surprising that some of them have been measured incorrectly. But where in the process have those errors occurred? Who has made them? How consequential are they? We don’t expect errors when an event is recorded in a database, or when simple student-level events like attendance are assembled into a body of evidence (attendance statistics). We expect those to be routine, free of bias and free of errors. But accidents occur in the act of observation, in the recording of events and in the act of human judgment. All three steps are full of human effort, including human bias, and are susceptible to error. Our definition of mismeasurement also includes misunderstanding the meaning of something. We expect people to disagree about the interpretation of measures of organizational vital signs, especially when the reputation of a school or district depends on it. We also expect differing interpretations to emerge from the same body of data. Attendance statistics, for instance, used to be viewed as simply the percentage of days that students attend school. Now you’ll often see attendance data viewed from the vantage point of chronic absentee rates—the percentage of students who are absent more than 10 percent of school days. Other analysts reframe attendance data to isolate those days missed that are adjacent to holiday breaks, or to exclude excused absences. New meaning may emerge from each reframed point of view. So, mismeasurement as we use it is more than a recording error or the use of the wrong measuring tool. Mismeasurement also includes misinterpretation. It may be making a mountain out of a proverbial molehill. Or it may be ignoring the mountain altogether. We consider mismeasurement to include errors of human judgment. In between the act of counting something and reporting that result is a lot of reasoning. To begin examining the soundness of that reasoning, let’s create a typology of mismeasurement….