Measurements “The Good”, “The Bad”, and “The Ugly” and how to design measurements that improve performance metrics when evaluating Engineers!

How to measure the performance of an Engineer is a critical business leadership task that can have major impacts on the business, customers, suppliers, and the individual engineer and engineering department within a business. The difference between motivation and innovation and de-motivation and lackluster innovation circles around having the right measurements. 

Ugly Measurement:

In the old days, one measurement of productivity was to quote the “Lines of Code” in a program. The business would boast and brag about how many lines of code were in their program and how larger and complex their software application was to prove to the customer the worth of the software. This is a clear example of an “Ugly” Measurement. The fact that the program/application took 12,000 lines of code to create is not only bloated by today’s standards of Micro Services it downright Ugly. 

It tells the customer that it took your engineers 12,000 lines to perform an application task. Meaning that if there a bug anywhere in that 12,000 lines of code it could take weeks, months, and in some cases years to find, fix, test, and deploy. This means unhappy customers, lost revenue, and increased costs. To adding to this anytime a new feature is proposed and merges with the code that contributes to the code bloat and now the code base is 15,000 lines of code. Operationally hosting that bloated codebase takes larger Storage, more Memory, and more CPU. This measurement promotes code bloat since the engineers are measured on how many lines of code have been written into the codebase. Thankfully this measurement went away with the business typewriter. 

Bad Measurement:

When I think of a bad measurement, it is a measurement that has the direct opposite impact than what was expected. I had a Senior Executive that determined that what software Engineers do is write code and “Check-In” that code to the Source Control. This is correct engineers do write code and then they check in that code to the company code repository. This senior manager also likes the passive verse the active measurement collection. Meaning instead of actively gathering data points have a system collects attributes and then automatically counts those attributes to produce a measurement. 

This Senior Executive determined that if an engineer had high levels and more frequent “check-in” of code this was a productive engineer. Those engineers who had a less frequent and small number of check-ins of code were less productive. A report was created on a daily, weekly, monthly, and quarterly totals of the number of Check-ins an engineer produced. These daily, weekly, monthly, and quarterly numbers were used to evaluate the highest producing engineer and the lowest producing engineer. Since the Senior Executive developed this measurement he was quite in love with this single measurement. This measurement was applied to Engineering Managers, Directors, and Vice Presidents in the Senior Executives Organization. Each Manager, Director, and Vice President were evaluated on the roll-up totals of check-ins within their area of responsibility. Senior Executive measured each level of the organization on this measurement of productivity. 

These senior executive-based promotions and bonuses and was used to reduce staff in times of layoffs. Once the Engineers, Managers, Directors, and Vice Presidents understood the direct impact of this single measurement many things happen immediately:

1) The number of daily and weekly check-ins increased by more than 300%.

2) The codebase grew more than doubled in size.

3) The software projects were being delayed and timelines were being missed. 

4) The best and brightest engineers were contemplating leaving the company.

This single productivity measurement had the direct opposite effect the senior executive thought it would have and caused multiple problems. The engineers being math inclined determined if they were being judged on the number of check-ins then they would write a small script and run a batch to check-out the code and place a Space or a comment or a date in the code thus changing the code and then the script would check-in the code. At first, these scripts were run during the day. But the Engineers expanded the time from just normal work hours to nights and weekends. The idea here was the engineers would show they were so productive that they would spend nights and weekends writing and checking in code. In truth, the scripts ran all day and every day, and it looks like these were highly productive. It was not changed until revenue numbers were seriously impacted the Vice Presidents had a one on one with this senior executive. 

Good Measurement:

The best measurements are not a single measurement, but a collection of measurements all tied to the revenue of the business. The list of measurements was based on the building blocks of an Application and the speed and effectiveness of the application. This promoted and tied revenue goals to software components and software components to applications and applications to revenue. This promoted the use of Micro Services and the reuse of the Micro Service in multiple Applications. Micro Service is a small component the does one thing well and only one thing and can be reused multiple times. This created a tighter code base, faster development times, shorter QA cycles, fewer software bugs, and a faster too market of new features and services creating higher revenue. 

The measurement rule in carpentry is to measure twice and only cut once. This is a good rule of thumb when it comes to the evaluation of an Engineer's productivity. By tying all measurements to revenue impact there is a direct productivity number in dollars. For example, if a new feature will produce X revenue dollars as a service to customers, and that feature takes 6 components and one component is assigned to 6 engineers each, each component is assigned revenue value 1/6 of the total. The impact is the engineers work as a team to create those 6 components and achieve the revenue goals for that new feature.

The Greenfield Groves Leadership

Coming together to reimagine health and wellness – because we too are consumers, and

we all deserve personalization, convenience, and quality we can trust.