The axiom of measurement, and academic institutions

The axiom of measurement is that the system being measured does not respond in a self-serving way to the act of being measured.

Thus, if you have a stick and you use calipers to measure its diameter at several points along its length, the stick does not dynamically redistribute its mass to form bumps wherever you place your calipers.

But when you use similar measurement-based quantitative approaches to monitor individuals, then those individuals respond in self-serving ways. CEOs may watch the next quarter or next year while compromising the next decade. Employees may behave in ways that make their supervisors give them good assessments (leading to bonuses or promotions) while compromising the goals of the company. Professors may write more research papers rather than better ones. Universities that measure their worth in terms of numbers of PhDs "produced" may indirectly incentivize the thesis supervisor to aim for minimum acceptable standards (which is hardly the ideal goal). And so on.

Imagine that you run a factory that makes steel balls for bearings. You carefully monitor the quality of the balls produced, e.g., by measuring asphericity. Any ball whose asphericity exceeds your tolerance goes into the scrap heap. And then, one day, your factory produces a ball whose asphericity is essentially zero. The ball is accurate to the diameter of one atom.

Will you celebrate? Have a small party, mention this amazing event in your company newsletter, or tell your customers?

You won't. In fact, you won't even notice. That's not where you are looking. You are watching out for bad ones, not good ones.

Now suppose that same factory management philosophy is adopted by the administration of an academic institution. The problem is that universities make their reputation not by the lack of horribleness in their worst faculty members, but by the excellence of their best ones. The "watch the worst" approach from factories will prompt you to try to improve quality by setting minimum numerical standards. Guide at least so many PhD students, write at least so many papers, get at least so much funding.

And your faculty members will self-servingly respond to whatever you are measuring. Your system will violate the axiom of measurement.

The only way to do it, it seems to me, is to arrange things so that responding costs more than the benefits therefrom.

I think there are two main ways to do this. One is to make the possibilities high dimensional. Instead of setting just a few criteria (e.g., PhDs, papers, money) and demanding numerical excellence on all of them, you can allow several criteria (e.g., teaching quality and volume, outreach activities, consulting, book writing, lecturing in other institutions, hosting excellent academic visitors, development of products and IP, starting successful companies, and maybe others). Of those, require excellence on just a few that the faculty member is free to select. And the second way is in how you deal with failure to achieve excellence. If excellence is not achieved, delay promotions slightly but do not deny promotions altogether.

This will mean that you will have some people who will not work to their full capacity. That is the cost of doing business. But you will also have some people who will work to their full capacity because they are not being scrutinized by bean counters. And those people, the ones who do work to their full capacity, unlike the balls in the factory, will be the ones that are noticed.

Will our academic institutions adopt such a gentle policy with faith in the future, betting that internal motivation beats external policing? I am not optimistic.