What’s a good developer survey participation rate?

What’s a good developer survey participation rate?

This is the latest issue of Engineering Enablement, a weekly newsletter covering the data behind world-class engineering organizations. To get articles like this in your inbox every week, subscribe here.


Upcoming webinar: Laura Tacho and I are hosting a webinar next week covering new industry benchmarks for measures of speed, effectiveness, quality, and impact. Register for the discussion here.


In recent years, partly due to the SPACE and DevEx frameworks, more companies have started using developer surveys to better understand developer productivity and the factors that affect it. At DX, one of our products is a developer survey platform, so a question we often get asked about is what survey participation rates other companies are getting. 

To answer this, we analyzed the average survey participation rates from a sample set of 200 companies. We only looked at companies that are DX customers, which is important because DX customers have significantly higher participation rates than others in the industry.

Here’s what we found from the data. 

The companies we analyzed achieved an average 96% participation rate in their surveys from their past three quarterly surveys. Again, this is markedly higher than others in the industry: companies like Microsoft, Google, Peloton, and Shopify hover between 25% - 50% participation. Many of the companies in this dataset went from having participation rates around 15-20% before using DX up to 80-100%. For more on how DX achieves this, register for a free product demo.

We also explored whether participation rates vary by organization size since, across the industry, we’ve heard of lower participation rates at larger companies. In our data, we can see an incremental decrease in participation rates the larger an organization gets. Companies with under 100 engineers saw an average 98% participation rate, while those with 100 - 500 engineers saw 95% participation, and larger organizations saw 91-92% participation.

So what survey participation rate should you be aiming for? If you’re just trying to collect some qualitative feedback comments, participation rate isn’t necessarily a critical concern. However, for organization-wide measurement and benchmarking, 80%+ participation rate is a must. Here’s why: 

  • Credibility - Tech execs don't believe in the data without strong participation.
  • Team-level data - Team-level data is important, otherwise data can’t be used for team action and improvement. Managers can't use the data if only half of their team responds.
  • Reliability - Measurements and benchmarking aren’t reliable if they’re based on a sparse sample. 

For more information on DX’s approach to developer surveys, read this article.


Who’s hiring right now

Here’s a roundup of Developer Productivity job openings. Find more open roles here.


That’s it for this week. Thanks for reading.

-Abi

Bill Havanki

Principal Software Engineer at Segment, Author

1mo

I've been consistently amazed by our DX participation rate - our last one was well over 90%, which is kind of nuts. I can think of two reasons: the questions are relevant to engineers' lives, and our organization follows up and takes action on the findings. We've had other surveys from other companies offered as well, and participation for those is way lower. Even when offered at the same time (ouch), DX participation is still through the roof.

Like
Reply
Nico Westerdale

Technical & Product Leader, Advisor, Mentor, Public Speaker - Accelerating startups 🔥

1mo

Do you have any metrics you can share about how people engage with your engagement metrics content?

Like
Reply
Oren Reshef

Test Strategist and Quality lead, Developer Experience Tribe at SEB

1mo

What does statistics says about good participation rate? With population/confidence level/margin of error you can get a good estimation to how many you need. For 1000 employees and 5% error you’ll need 285 responses, but 96% is obviously better 😀

Like
Reply
Dan Grøndahl

Platform Owner - CCoE @ VELUX

1mo

Interesting. 🙂 Could high participation rates in your data be self-reinforcing from companies already prioritizing developer experience? If I’m a developer in an organization where my feedback leads to actionable change, I’m much more likely to engage with these surveys. Do you see a correlation between survey participation and the organizational commitment to DevEx? And can you actually measure whether there’s an inherent bias, given that your customers may already be inclined toward improving DevEx?

John Cutler

Product Stuff @Dotwork ex-{Company Name}

1mo

Goes without saying but IME you start seeing an effect where eng. managers more than subtly start "requiring" participation. High participation rates can also be a sign of pathology, and the same forces that get the high rate, are also what inhibit improvement.

To view or add a comment, sign in

Insights from the community

Others also viewed

Explore topics