What’s a good developer survey participation rate?
This is the latest issue of Engineering Enablement, a weekly newsletter covering the data behind world-class engineering organizations. To get articles like this in your inbox every week, subscribe here.
Upcoming webinar: Laura Tacho and I are hosting a webinar next week covering new industry benchmarks for measures of speed, effectiveness, quality, and impact. Register for the discussion here.
In recent years, partly due to the SPACE and DevEx frameworks, more companies have started using developer surveys to better understand developer productivity and the factors that affect it. At DX, one of our products is a developer survey platform, so a question we often get asked about is what survey participation rates other companies are getting.
To answer this, we analyzed the average survey participation rates from a sample set of 200 companies. We only looked at companies that are DX customers, which is important because DX customers have significantly higher participation rates than others in the industry.
Here’s what we found from the data.
The companies we analyzed achieved an average 96% participation rate in their surveys from their past three quarterly surveys. Again, this is markedly higher than others in the industry: companies like Microsoft, Google, Peloton, and Shopify hover between 25% - 50% participation. Many of the companies in this dataset went from having participation rates around 15-20% before using DX up to 80-100%. For more on how DX achieves this, register for a free product demo.
We also explored whether participation rates vary by organization size since, across the industry, we’ve heard of lower participation rates at larger companies. In our data, we can see an incremental decrease in participation rates the larger an organization gets. Companies with under 100 engineers saw an average 98% participation rate, while those with 100 - 500 engineers saw 95% participation, and larger organizations saw 91-92% participation.
Recommended by LinkedIn
So what survey participation rate should you be aiming for? If you’re just trying to collect some qualitative feedback comments, participation rate isn’t necessarily a critical concern. However, for organization-wide measurement and benchmarking, 80%+ participation rate is a must. Here’s why:
For more information on DX’s approach to developer surveys, read this article.
Who’s hiring right now
Here’s a roundup of Developer Productivity job openings. Find more open roles here.
That’s it for this week. Thanks for reading.
-Abi
Principal Software Engineer at Segment, Author
2moI've been consistently amazed by our DX participation rate - our last one was well over 90%, which is kind of nuts. I can think of two reasons: the questions are relevant to engineers' lives, and our organization follows up and takes action on the findings. We've had other surveys from other companies offered as well, and participation for those is way lower. Even when offered at the same time (ouch), DX participation is still through the roof.
Technical & Product Leader, Advisor, Mentor, Public Speaker - Accelerating startups 🔥
2moDo you have any metrics you can share about how people engage with your engagement metrics content?
Test Strategist and Quality lead, Developer Experience Tribe at SEB
2moWhat does statistics says about good participation rate? With population/confidence level/margin of error you can get a good estimation to how many you need. For 1000 employees and 5% error you’ll need 285 responses, but 96% is obviously better 😀
Platform Owner - CCoE @ VELUX
2moInteresting. 🙂 Could high participation rates in your data be self-reinforcing from companies already prioritizing developer experience? If I’m a developer in an organization where my feedback leads to actionable change, I’m much more likely to engage with these surveys. Do you see a correlation between survey participation and the organizational commitment to DevEx? And can you actually measure whether there’s an inherent bias, given that your customers may already be inclined toward improving DevEx?
Head of Product @Dotwork ex-{Company Name}
2moGoes without saying but IME you start seeing an effect where eng. managers more than subtly start "requiring" participation. High participation rates can also be a sign of pathology, and the same forces that get the high rate, are also what inhibit improvement.