Chapter 7

Reducing barriers to progress at all levels

Table of Contents

Chapter 7 Overview

This chapter suggests contributions to the reduction of mismeasurement in schools that can originate from six levels: schools of education, administrative credential programs, state departments of education, assessment firms, districts and school boards. Schools of education aren’t teaching data analysis skills. Administrative credential programs could stop issuing credentials to those who lack assessment knowledge and aren’t yet data literate. Assessment firms and state departments of education could turn to design and communication firms to elevate the quality of reports. Districts could add analytic responsibilities to their assessment directors’ job, and lift some of the assessment interpretation responsibility off the shoulders of teachers. School boards could insist on better evidence when plans, budgets and policies are on the table, and ask leadership to explain more fully how they reached their conclusion about the meaning of test results.

Chapter 7 Excerpt

After spending more than 20 years helping school and district leaders make sense of their numbers, I have become more impatient and yet more hopeful. My impatience comes from my strong belief that some mismeasurement is correctable. That’s the part that’s due to bureaucratic ineptitude, slothfulness and innocent human errors. But I’m not naïve enough to think it’s all correctable without friction among competing parties. Some mismeasurement has evolved because it has served both institutional and professional interests. To take one example of institutional interests, everyone wants students to graduate, so there’s an incentive for districts to make it increasingly easy for students to earn a diploma. The bar gets lowered so far in some districts that the value of the diplomas they award become questionable. To take an example of professional interests, consider the state of teacher evaluations. Principals need the cooperation of their teachers. So, principals by and large evaluate teachers without providing critical feedback, believing this wins them a more cooperative staff. The long tradition of teacher compensation that rewards time-in-service and education adds to the ethos of treating all teachers equally, or as “widgets,” the key word in the title of the report of the National Center for Teaching Quality. The result: almost no information captured in teacher evaluations about the quality of teaching. My hopefulness comes from meeting and working with many school and district leaders who are hard at work bringing evidence and empirical methods to the table. My hope also comes from the fact that you’re reading this book. If I’m not falling under the spell of premature exuberance, then I have a hunch you have already joined (or are about to join) the small but growing legion of principals, superintendents, school board members, teacher leaders and citizens who are ready to insist that evidence and reason must justify decisions. Our small band of empiricists is already suspicious when school leaders justify their decisions with “experience” alone. When someone high up justifies her opinion with “tradition” or “common sense,” we reply, “Show me the evidence!” Among the “common sense” crowd, most fall into three camps: those who are puzzled or overwhelmed by too much data; those who distrust measurement altogether; and those who believe that wisdom comes solely from experience. Each deserves attention of a somewhat different sort. Each has a reason to undervalue or disfavor empirical paths to knowledge. Persuading colleagues begins with understanding their viewpoint….

Chapter 7 Resources

Hattie, John, Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement,Routledge (2009), 374 pages.

Mandinach, Ellen and Gummer, Edith S., Data Literacy for Educators: Making It Count in Teacher Preparation and Practice, Teachers College Press (2016), 176 pages.

Rankin, Jenny Grant, Designing Data Reports that Work: A Guide for Creating Data Systems in Schools and Districts, Routledge (2016), 217 pages.

Statewide Longitudinal Data Systems Grant Program, SLDS Data Use Standards: Knowledge, Skills, and Professional Behaviors for Effective Data Use, Version 2. U.S. Department of Education. (2015) Washington, DC: National Center for Education Statistics. Available online at: https://slds.ed.gov/#program/data-use-standards

Photo of Ellen Mandinach
“A timely book about the importance of the use of data, facts, and evidence in education. It raised thoughtful questions about how data should be used for what purposes and the ensuing interpretations.”

Ellen Mandinach – Associate director for research of EDC’s Center for Children and Technology and director for research of the Northeast and Islands Regional Education Laboratory (NEIREL)