Very very interesting find: most devs using LLM tools for 6+ months see productivity increases, but most eng leaders where their teams use these tools do not see team-level increases. Why?
As context, this finding comes from this week's The Pragmatic Engineer deepdive where with Elin Nilsson we analyzed 216 responses from software engineers and engineering leaders on their usage of LLM tools for software engineering.
My theories on why this disreptancy:
1. Productivity translates to better work-life balance.
People could well be more productive, but the gains are consumed in other tasks. If GenAI helps save a few minutes off a longer task, there’s more time for a walk, a coffee, go home earlier, etc.
2. GenAI mostly aids simpler work.
GenAI helps with coding tasks in large organizations, but coding is often the smallest of all work! Alignment, meetings, and talking with others often takes up more time. GenAI won’t change this.
3. Maybe it's just hard to measure?
As an engineering leader: consider that GenAI can have a positive impact without the impact being measured! We have ample evidence via other studies (e.g. this peer reviewed one: https://lnkd.in/eCgXfyRw) that GenAI tools are genuinely helpful for software engineers. And while most engineering leaders have the perception that these tools don’t make a big difference at the team level: I’d urge all engineering leaders to look closer before declaring that there really is no difference!
The full article where the image is from, with more findings: https://lnkd.in/eBkrxDJA
Experienced CRO, NED & GTM Advisor
8moJude Jarvis