Ranking The Unrankable: A Veterinary Education Series. Part 5: The Formula Fallacy (#593)

Can Objectivity Ever Be Truly Neutral?

Faced with the shortcomings of reputation and research-based rankings, the instinct is almost irresistible:

Let’s make it objective.

No opinions. No surveys. No prestige bias.

Just numbers.

Transparent.

Measurable.

Defensible.

And so, proposals have emerged, most notably a formula-based model that seeks to rank veterinary schools using quantifiable variables such as:

Teaching hospital case load,

Faculty numbers,

NAVLE pass rates,

Research expenditures, and

Tuition and cost of living.

At first glance, this feels like a breakthrough.

At last, something rigorous.

The Appeal of the Formula

The attraction is obvious.

A formula promises:

Consistency.

Transparency.

Reproducibility.

Every school is evaluated using the same criteria.

No ambiguity. No subjectivity. Just a score.

And in a world uneasy with uncertainty, that is deeply comforting.

What Feels Objective Often Isn’t

But there is a quiet problem embedded within every formula.

Before the numbers are calculated, decisions must be made:

Which variables matter?

How are they measured?

What weight is assigned to each?

For example:

Should hospital case load carry more weight than student well-being?

Should research funding outweigh teaching quality?

How much importance should be given to cost?

These are not technical decisions.

They are value judgments.

The Hidden Hand of Values

The proposed formula assigns weights to different parameters.

Some more important than others.

But who decides those weights?

In one model, they were derived from a survey of faculty within a single institution.

Reasonable? Perhaps.

Universal? Hardly.

Because what one group values:

Clinical exposure.

Research output.

Infrastructure.

another might value differently:

Mentorship.

Student support.

Community engagement.

The formula does not eliminate subjectivity.

It simply hides it inside the weights.

The Reduction Problem

There is a deeper issue.

Even if we agreed on the variables and their weights, we are still attempting something fundamentally difficult:

Reducing a complex educational ecosystem to a single number.

Consider what is lost in that reduction:

The quality of a mentor who changes a student’s life.

The confidence gained during a difficult clinical case.

The culture of a program.

The subtle interplay between faculty, students, and community.

These are not peripheral details.

They are the essence of education.

The Illusion of Fairness

Formulas feel fair because they treat everyone the same.

But equal treatment is not the same as equitable understanding.

A school serving:

Rural communities.

Underserved populations.

Resource-limited environments.

may not score as highly on:

Research funding.

Infrastructure.

Citation metrics.

And yet, its contribution to the profession may be profound.

A formula may penalize what is, in fact, deeply valuable.

What Gets Measured Expands

There is another predictable consequence.

Once a formula is introduced, institutions begin to optimize for it.

Increase case load.

Expand faculty numbers.

Invest in measurable outputs.

Not necessarily because these improve education.

But because they improve score.

We have seen this before.

Metrics shape behavior.

The Comfort of a Number

Why, then, do we keep returning to formulas?

Because they offer certainty.

A number feels definitive.

It allows us to say:

This school is better than that one.

But certainty, in this context, is often an illusion.

A carefully constructed one - but an illusion nonetheless.

A Different Kind of Question

Instead of asking:

Can we build a perfect ranking formula?

Perhaps we should ask:

What are we unwilling to lose in the process of measuring?

Because every act of measurement is also an act of exclusion.

And what we exclude matters.

Final Thought

The formula-based approach is an improvement over pure reputation.

It is more transparent.

More defensible.

More thoughtful.

But it cannot escape a fundamental truth:

Objectivity is not the absence of values.

It is the expression of values, carefully arranged in numbers.

And until we confront those values directly, no formula will give us what we are really seeking.

Coming Next

Part 6: Competence Without Competition: Do We Need Rankings at All?

#bunyipsbooksbeyond