How to Avoid Misunderstanding, Misinterpreting and Distorting Data
Menu
Chapter 3
The hidden hazards of interpreting graduation rates
Table of Contents
Chapter 3 Overview
This chapter looks at the factors that make comparing graduation rates precarious. States set graduation requirement minimums. Then districts layer their own requirements on top of their state’s. Some schools do the same. The results make comparability elusive. Credit recovery programs in some districts give struggling students a boost. But in too many districts, credit recovery has become a way to falsify student learning. Dropouts are at times hard to identify. Finally, continuation high schools’ vital signs remain invisible. Success for their students can’t be reduced to obtaining a diploma in four years.
Chapter 3 Excerpt
We’d like to find one aspect of a school’s vital signs that is unambiguous, easy to count and likely to be interpreted correctly. I wish I could tell you that high school graduation rates fit these three criteria. But graduation rates are noisy, messy and filled with ambiguity. However, by becoming aware of where the noise in the signal is hiding, by seeing what’s messy and ambiguous, you can become a smarter consumer of this vital sign. And if you’re a district or school leader, you can boost your ability to see your students’ progress more clearly and take steps to pave their many paths to graduation.
The results you get when you’re figuring out graduation and dropout rates is just a function of what goes up top in the numerator and what goes below in the denominator. Pay attention to who’s counted in each. When you are asked about your graduation rate, you’d do well to back up and explain what goes into it before you explain what comes out of it.
Let’s start with denominators. Today, the U.S. Dept. of Education requires that all states use ninth-grade enrollment for a graduating class as the denominator. That makes the graduation rate a measure of a group of students who move more or less together toward graduation. This is also called a cohort. But note that it’s not a perfectly static one. Students transfer out to other schools. Other students transfer in. Some students leave school altogether, what we call dropouts. This migration results in a less than perfect grad rate measure. That’s why this measure of a group of graduating seniors, who are not exactly the same students as the group of ninth graders four years prior, is called by researchers a “quasi-longitudinal” measure. If they were exactly the same students, the measure could be called a truly “longitudinal” measure.
Let’s take note of the impact of student mobility on a mid-size high school whose 2020 graduating class cohort started with 300 students in 2016–2017. If the attrition rate for this hypothetical Class of 2020 above is just 15 percent, and if newly arriving students arrive at exactly the same pace as students transfer out of the school, by the end of their senior year, only 61 percent of the students in that class will have started four years prior….
Chapter 3 References
Chiatovich, Tara and Elizabeth Rivera Rodas, “Developing an Early Warning Indicator System in a High-Poverty Urban Context,” 2016. SDP Fellowship Capstone Report.
Larsen, Matthew F., “High-School Exit Exams Are Tough on Crime,” Education Next, summer 2020 (vol. 20, no. 3).
Malkus, Nat, “Practice outpacing policy? Credit recovery in American school districts,” American Enterprise Institute, November 21, 2019.
Mathews, Jay, Class Struggle: What’s Wrong (and Right) with America’s Best Public High Schools, Three Rivers Press (1999), 320 pages.
Ruiz de Velasco, Jorge and Daisy Gonzales, “Accountability for Alternative Schools in California,” Policy Analysis for California Education (February 2017).
Rumberger, Russell W., Dropping Out: Why Students Drop Out of High School and What Can Be Done About It, Harvard University Press (2011), 400 pages.
U.S. Department of Education, “Every Student Succeeds Act High School Graduation Rate Non-Regulatory Guidance,” January 2017, 32 pages.
“Mismeasurement of Schools’ Vital Signs by Rees and Wynns is a rewardingly accessible and compelling analysis of the misuses and misunderstandings of K-12 educational data. The authors document the problem and identify ways we might overcome our tendency to make those all too frequent, yet avoidable misjudgments that result in costly, negative, life-long impacts on our nation’s students. The examples and explanations are relatable and easy-to-follow. Each chapter’s ‘questions to spark conversation’ invite meaningful examination of educational effectiveness data by educators, leaders, and governing boards. This book is a bold and forthright consideration of the bumpy terrain of K-12 educational assessment. At a minimum it should be required reading in teacher preparation and administrative leadership programs across the county. Parents, teachers, school board members, educational leaders, commentators, and policy makers will discover it to be a refreshingly frank and constructive resource worthy of serious attention.”
Carol Ann Gittens – Dean, Kalmanovitz School of Education at Saint Mary’s College of California
