Quantifying Quality: How to Effectively Measure Developer Experience
Dear reader,
Welcome back to the second issue of "DevEx Nuggets", your weekly digest delving into the world of Developer Experience and its role in creating and maintaining high-performing teams and organizations.
Last week, we defined Developer Experience and explored why it is essential. This week, we're taking the next step: How do we measure developer experience?
Quantifying something as multifaceted as Developer Experience can seem challenging, but it's not impossible. By identifying the right tools, approaches and metrics, we can assess the experience of engineers and make informed decisions to enhance it.
Here are a few metrics and techniques to consider:
Surveys: Regularly survey your team on their work, tools, processes, and environment. Tools like eNPS (Employee Net Promoter Score) can provide valuable insights and often get higher responses than lengthy surveys. The drawback of the eNPS is that it can show that there is a problem but not what the problem is.
Retrospectives: Post-project retrospectives can offer detailed feedback on the developer experience during a particular project or sprint.
Developer Productivity Metrics: While not the sole measure, productivity metrics, like bug resolution time or time taken to deploy a feature, can hint towards the developer experience. It is important to note that these metrics often are trailing indicators of developer experience. Use them with care!
Turnover Rates: High turnover rates often indicate an unfavorable developer experience. This is a serious alarm signal.
Feedback Channels: Establish anonymous feedback channels where developers can express concerns or share suggestions.
I hope this has given you a starting point for measuring the developer experience in your team or organization. Measuring and understanding the developer experience often is the first step toward identifying the most significant improvements and their value for the company.
A word on 1-on-1s
When I talk with people about how they measure developer experience and productivity, they often mention 1-on-1s. While 1-on-1s can bring great insights and give the flexibility to dive deeper into certain topics when needed, they have a couple of risks:
You only get to hear what people are comfortable telling you. This can mean that the *real* issues go unnoticed.
You only get feedback from the people you are talking to. This can mean that the results you get are only from a certain group of people and do not cover the full space.
This is, why my DevEx Assessment is a combination of 1-on-1 interviews and customized surveys. Here I am using the 1-on-1 interviews to inform the creation of the surveys. The surveys then surface deep and broad insights across the organization.
Additional Reading
Today I would like to recommend the following three articles in my blog as additional reading on the topic of measuring and assessing developer experience:
Developer Experience at Risk: Red Flags for Engineering Leaders – on clear warning signs that indicate issues with developer experience, regardless of what metrics tell you.
How to Measure Developer Experience: The Danger of Metrics — on the issues that can arise when you focus on metrics alone.
11 Tips to Prevent Survey Fatigue in Your Organization — on how to create surveys that get more responses.
Tobi's Top Reads of the Week
Now, over to my top reads of the week:
The SPACE of Developer Productivity by Nicole Forsgren, Margaret-Anne Storey and others has been out since 2021 and still is a valuable resource for understanding developer productivity and experience. Thus it fits well into today's newsletter.
How Not to Use the DORA Metrics to Measure DevOps Performance by David Rant — One of the common pitfalls I see with teams using DORA (or other) metrics is the slippery slope between measuring improvements and making the metric a target. This article nicely shows potential misuse of the DORA metrics.
Question of the Week
How do you currently measure developer experience within your organization?
I'd love to hear about the tools and metrics you've found most effective.
As always, I invite you to share your thoughts, experiences, and questions in the comments or via direct message. Your insights enrich this space.
If Developer Experience is an important topic for you, get in touch!
Until next week,
Tobi