The Lost Art of Evaluation

One of my clients is a senior middle manager in a very complex organization. He had been telling me for sometime about his supervisor, a member of the C-suite, who was completely non-communicative, and micromanaged him via email. He was certain that she hated him—or at least, had no interest in him. We spoke about her on occasion over the course of months, and I assumed that she was equally at poor at her work as she was at communication and management. I was wrong.

When I finally got around to asking about her performance, I was amazed to discover that she was incredibly productive. So much so that nobody could come close to her numbers. It explained a lot.

The organization in question had a robust data management structure. They measured everything. But they had no serious qualitative performance evaluation model. And the culture reflected it. Managers screamed at subordinates, peers called each other names and negative gossip was rife. But there was no means to discover any of that.

Get Me a Tape Measure!

Organizations love to measure. They dutifully collect data on everything from sales and costs to customer satisfaction and market share. But an over-reliance on measurement can blind them to information that is easily observable. Measurement and evaluation are not the same thing.

In today’s business world, data is kind. And so, we rely on metrics to illuminate every aspect of both business and the people in it. But it only captures part of the story.

Evaluation is distinct. It is the human process of assessing behavior, outcomes and processes in context. Those judgments are rarely quantifiable, but they are no less critical for that. Using our ability to appraise and evaluate enables us to make important judgments about effectiveness and performance.

You Manage What You Measure

When Jack Welch ran GE, he zealously pushed metrics like Six Sigma. His mantra was "What can be measured can be improved." Welch's forced ranking system meant that the bottom 10% of employees (based on the numbers) were fired every year! Of course, that led to high turnover.

While financials improved, many believe it came at a human cost. Plus, it stifled innovativeness. Divisions that were pressured into competition with other divisions were so beholden to production that they stopped creating. There was no room for failure—and failure is integral to experimentation. The unmeasurable costs of poor morale, attrition of tacit knowledge, and weakened culture was costly over the long run. But it wasn’t obvious until later, seen in hindsight.

Bring Me The Scalped Heads

Robert McNamara was the Secretary of Defense in the Kennedy administration. He oversaw the progress of the Vietnam War using a dat-driven approach, rigorously measuring factors like enemy casualties, territorial control, and South Vietnamese troop levels.

But, as seems obvious now, the “positive” metrics failed to capture the complexity on the ground. And McNamara’s reliance and faith in his metrics blinded him to those unquantifiable factors; factors that predicted the horrible eventual outcome. Measurement was a profoundly weak instrument through which to view the reality of the endless conflict.

Measuring the Evaluation-Measurement Gap

Intuitively, it seems like indicators of high sales volumes and customer retention would imply that employee performance is equally strong. But, according to research, it doesn’t. In a well-known study, Ridgway found that there was very poor correlation between performance metrics like sales volume, costs, and profits were poorly and how managers evaluated employees within their teams.

That wasn’t the only study on this topic. Similar observational and experimental studies have continued to reinforce this finding. There is a significant gap between measurements and holistic assessment. More recently, Adam Grant and others showed that key determinants of top performers like relationship-building, communication, and empathy were critical aspects of employee performance, satisfaction and retention. But, they don’t show up in most metrics. True evaluation requires observing these unquantifiable "soft skills" in action through techniques like interviews, focus groups, and immersive observation.

Putting it Together

Knowing all of this leads us to several conclusions. First, that leaders must remain actively engaged with their teams. Without their observations, conversations and insights, the human and cultural territory of the enterprise is obscured. While dashboards are a piece of the performance management puzzle, it is far from sufficient.

There are also tools and structures we can use to gain greater insight into diverse stakeholder perspectives. The goal is to to see more than the measurable production—but also to see assess performance, strategy, and culture. Surveys, interviews, focus groups, and observation provide subjective but crucial context. Does morale reflect the metrics? Is turnover high despite meeting targets? Do employees feel empowered or suppressed?

Observation reveals bottlenecks, team dynamics, and innovation in action. It takes time and skill. But statistics never give the full picture.

This is a challenge, especially for current startups that pride themselves on their terabytes of data. New technologies like AI and big data analytics could easily be mistaken as potential replacements for observational evaluation. That would be risk over-indexing on the quantifiable at the expense of the unmeasurable.

Avoiding these potentialities takes deliberate effort and ongoing engagement. Leadership must carve out time for qualitative evaluation and signal its importance. Management by walking around (assuming workers are periodically collocated) facilitates ongoing observation and involvement with the actual people executing real processes. With care and intention, organizations can strike the right balance between measurement and evaluation

Do you lead a team —especially a technology team—and hunger for more leadership, strategy and negotiation skill? The Leaders Lab mastermind is designed to provide that! Get a small taste of it by enrolling in this FREE e-mail course on leadership. Click here to join for free.