The Problem With Benchmarking
Schools Seeking the Transient Safety of the Herd
Last month offered the chance to spend some quality time with a hardworking independent school governing board, in one of my favorite places— the 50th state. The ostensible purpose was to do an institutional health check of sorts. We dove into metrics of all kinds, from compensation to philanthropy to admissions to facilities, and the list extends. But throughout our sessions I worried about context, about the “compared to what” dimension.
Surely, though, there’s value in looking at any school by the numbers. A particularly cantankerous CFO once hit me with the assertion that “if it can’t be measured, it can’t be improved.” As much as that dictum struck me as too minimalist, it stuck as one of those sayings that echoes across the decades thereafter. Forming a dashboard, then reviewing the data it contains with disciplined regularity, will invariably produce insights, combining a healthy blend of discovery and affirmation connected to institutional trajectory.
Here’s a shameless plug: a while back, our little schoolhouse in Nashville accepted an invite to join the Independent School Data Exchange (INDEX), a not-for-profit consortium that shares heaps of (often behind the curtain) information about spending, revenue, outcomes and inputs, generating voluminous annual reports that tell the unvarnished truth on myriad topics. Our singularly sharp CFO used those numbers to frame reports that then helped move mountains.
Charting our place in the INDEX constellation of schools proved to be mesmerizing during Board retreats, admin team meetings, and budget-setting town hall sessions with families and faculty and staff. What people found most compelling were the outlier data points— places where our numbers differed from the broader averages of that esteemed group of peer schools. And in most instances, those points were the result of specific decisions we’d made, e.g. limiting tuition increases and facilities expansion in order to ultimately steer more toward teacher salaries while increasing class sizes incrementally. We aimed, as a maxim of sorts, to charge in the bottom half of our cohort, pay in the top quartile, and do so with an endowment that labored to rise from the 5th to the 4th quintile. Lather, rinse, repeat.
Those benchmarking exercises reinforced the zero-sum nature of the work at hand and underscored the importance of setting priorities that could hold up over time. I recall a conversation with an idealistic founder of a little faith-based school in Music City who told me he wanted to keep tuition low, offer lots of financial aid, and pay people really well. My response was that doing one of those things would be impressive, and accomplishing two of those things miraculous, and finessing all three would require the suspension of fundamental laws of physics. Dashboards keep you humble, but they can keep you grinding. That’s all in the best case scenario, assuming that there’s a persistent hunger to use metrics in a strategy to improve, to beat the odds of a given set of resource-based realities.
As we were working through that data set on Kaua’i a few weeks ago, I kept wondering what the take-home lessons would be for those trustees. How were they making sense of the absolute numbers, in light of the relative numbers from all 1400 or so NAIS member schools, and what did they think about their outliers, some of which were stunningly impressive btw.
My worry was/is that they would feel some pressure to adjust in ways that put the school more in the mainstream, thinking there must be some wisdom in the crowd. But what if there’s not? What if a slow, constant, never fully considered drift just pushed everyone toward an outcome that in hindsight they never would have chosen?
And what if the inertia/momentum was such that it just kind of happened, and before we knew it, while consumer prices doubled, wages and salaries tripled, and tuitions quadrupled, so that 5% annual increases started to really ratchet up? Then no amount of spending on shiny consultants to talk about markets and customers and value propositions could put education within reach for growing swaths of the population that once (but no longer) could see themselves signing on. And then no one can see a path back? Hello, 2026.
In that kind of environment, of course schools would seek the safety of the herd, absent any viable alternative here in the age of the affordability conundrum. And then, benchmarking would be understood to confirm the futility of efforts to do otherwise. Independent schools would be left to convince their communities that household spending needs a proportionate shift to education, away from food, clothing, shelter, and leisure. And maybe it should, but more likely instead is a shift toward extremely high income families as the core demographic, from docs and profs and lawyers to hedge fund and private equity hotshots— a more finite cohort.
What’s another way forward, you might well ask? What if K-12 private institutions affirmatively decided not to follow private colleges off the cliff that seems to be up around the bend, having already claimed 500+ (per WSJ) of them in the past decade? What if some enterprising schools decided which metrics they’d engineer to be outliers? What if they saw outlier status as something to be celebrated, pushing back against regression to the independent school mean, concluding that maybe the collective means were actually the problem and not the safety zone?
Nothing about this would be easy, but postponing efforts in that direction will only make the hill steeper to climb. My question is who’s out there doing this work already, and who is telling their story? Who’s doing a different style of benchmarking, pushing back against the tendency toward a full payer/financial aid recipient student body barbell configuration— and what were essential first steps?
Getting there would require identifying the current norms as a call to action— but maybe folks just don’t see it that way, or maybe in the midst of the rest of the daily struggle it’s just too hard to pick one’s head up far enough to look toward the horizon. No one has experienced the years since COVID as a cakewalk, to be sure. But starting with two or three initiatives, challenging the status quo in ways that acknowledged the dizzying AI-infused change in the air, challenging an educational model that would have been familiar to Rip Van Winkle, sure would be worth at least the thought exercise and a good faith effort.
So find the right benchmarking cohort, peg the trends, consider the context— regional, cultural, historical— and start building some scenarios. Nothing but good can come of that.
Ever eager to hear reports from the field,
Vince
One of the parallel purposes of putting these thoughts out into the ether is to open a window for a response that's better than the original post. Case in point- your reply- many thx Dominic- you put a z axis onto the x/y that limited what I generated. YES- we need to work on measuring the hard to measure- don't we? And we need to acknowledge the biases inherent in any dashboard creation. One corollary to the whole benchmark importance argument is that whatever is measured will almost certainly be over-responded to by most audiences. And that which is not measured will more easily be ignored. Should we work on this mon ami?
Thank you for this intriguing post.
So, I have many thoughts about this, Vince. I will try to be succinct.
I worked in a school where we were instructed to develop metrics for the board to review at each board meeting. So we collected and found data that we possessed but had not put into a dashboard. Over the years, we collected more data and refined the dashboard. After a while, it was clear that the trouble was that the data and the metrics we had collected and analyzed did not align with the idealistic aims of the school. There were no metrics for critical thinking, developing character strengths, or feeling an emergent sense of purpose. All the things we truly cared about were not reflected in the data we had collected so religiously. As a result of this, I spent the next two decades with others trying to understand these ineffable and idealistic outcomes that we were not currently collecting or measuring. There was an interesting tests in critical thinking that we developed with ETS. There was a non-profit, the Character Lab, where we investigated how to develop and measure character strengths. We played around with portfolio assessment. The same problems always emerged. Firstly, it took an immense effort, both in time and financially, to try and develop alternative benchmarks. Secondly, they were often very difficult to measure. Thirdly, it was incredibly difficult to abandon the existing metrics in favor of new ones.
I agree with what you wrote about “outliers” and how they may lead the way in thinking differently about outcomes and how we track our progress both individually and collectively towards them; however, we also need a significant shift, nationally and globally, about the purposes of school and education to be less about checking boxes, compliance and supposed achievement and for the actual aims to be about developing better thinking, better moral decision-making and healthier young people who have collective well-being predominately in their purposes rather than self-aggrandizement and getting ahead. This societal significant shift has to happen alongside developing significantly different benchmarks and measures. Our systems for education and learning are under significant stress and many parts of our systems are broken, and yet the status quo remains resilient. We need a revolution.
Thanks for getting us thinking about this important issue. It is imperative that this becomes a priority for governments and institutions rather than a collection of small initiatives and theories. We need there to be an imperative for wholesale change rather than tinkering at the borders. As a final note, I am glad to see countries like Estonia considering these very issues in a more holistic national way, especially as AI disrupts education (https://tihupe.ee/en/).