How to evaluate the Validity of Evidence - Product Thoughts #127

The days in which we had to collect proof for why an idea might be worth investing time, money, and resources are gone. These days, your company (ideally) embraces a mindset of gathering evidence to increase the certainty that "becoming serious" about an idea (aka building an MVP). Or, as my buddy Timo recently said: He wants to feel secure, instead of trying to be secure. Spoiler: You will never objectively be secure.

Gathering evidence can (and has to) happen during all three primary levels of product development: Product Strategy, Product Discovery, and Product Delivery. But behold: Not all evidence is to be treated equally. Instead, you should be aware of its origin.

My primary measure of validity for evidence is simple: How far away from the user has it been? Is it an idea your CEO had based on a recent article she read about trends in China? Was it a story told from a sales rep who overheard customers say something over lunch? Did you sit down with a customer yourself? Or is it derived from quantitative and qualitative feedback from a product which is life?

I'm not a friend of over-scoring every idea prioritization process, but I would argue that any of these scenarios mentioned above should be weighed differently when it comes to influencing the decision to pursue an idea. Ideally, you combine these sources for a holistic perspective on prioritizing initiatives.

In a way, you might want to treat gathering evidence for a new idea like first principle thinking. First principles thinking is the act of boiling a process down to the fundamental parts that you know are true and building up from there. A first principle is a basic assumption that cannot be deduced any further. So, insights you have been able to gather without a proxy in between (whether it's a market research institute, a co-worker or the bias of your analytics manager).

I might share one of my own mistakes of not following through on first principle evidence gathering and the consequences this brought to a product I've worked on.

How is your company treating the different degrees of validity for evidence around a new idea?

Have a great week and take care,

Tim

What I've been reading

Product adoption: how to get customers to embrace your product

The sad truth is your potential customers are probably more comfortable staying where they are rather than taking a chance on something new. Luckily, there are ways to convince them to make a switch and increase your product adoption. Whether you’re releasing your first product or getting ready to announce your fiftieth feature, here are a few tips we’ve learned to help you overcome customer inertia and help users see the value of what you have to offer.

Empathy and Product Management How important is empathy in product management?

Much has been written about empathy when it comes to product design, user experience, and understanding customers. If you’re building something for someone else, you’ll be more successful if you can identify with their needs first. But empathy is also essential to leading a team, especially for PMs who tend to have responsibility without authority and must earn the respect and support of others. If you can see the world through your engineer’s eyes, or from the perspective of your VP of sales, you can understand what motivates them, what frustrates them, and what you can do to help.

The Power of Thoughtful Research by Prakriti Parijat

It is tempting to ask customers what they want, but you won’t get a good answer. Because what they really want is to be liked. We may say “there is no such thing as a dumb question,” but in research, this isn’t true. Leading questions don’t provide good insights – the participants want to answer “correctly” and end up giving the responses you want rather than the real information you need.

Guide to a remote product manager’s daily routines: tools, processes and tips

What is it like to work as a remote product manager? A product manager’s day is rarely uniform, especially if they’re working in a different location from their team. If you’re transitioning from a co-located product manager role to a remote or distributed environment (or you just want to learn how other product managers handle working with a distributed team), you’re in the right place.

What’s Wrong with Dot Voting Exercises

If I’m being honest, part of my frustration is that I see really great tools — such as dot voting — being wielded poorly, with lackluster results. This eventually leads to faulty conclusions about the value of such tools. This is true of tools as well as the entire practice of design. In the democratization of design methods and mindsets, I see a lot of adolescent fumbling with new things, by non-skilled practitioners. This is good and to be expected. But, this calls for those who are skilled to step into coaching roles, and model best practices. Or, as is the intention of this article, to be more explicit about things might have come naturally and never have been articulated.

A great product starts with a great vision

What unique opinions does your product have about the world? What do you believe about your problem space that no one else does? These answers form the basis of what your company and product will stand for, and who is likely want to work there or use it.

Internal Product Development by Lucia Adams

Change theory can be broken down into three key stages – Endings, The Neutral Zone, and New Beginnings. Lucia uncovered an important realization about herself as an influencer of change: her natural tendency was to start at the ‘New Beginnings’ stage. However, this “neglected the needs” of the news people she had been working with: they wanted some form of ending to familiar old ways first. Having learned this lesson, Lucia has implemented considerations of all three change stages into her coaching methods ever since.

Product Thoughts is a weekly newsletter/digest I send out to over a thousand leaders in product, UX, and business every week. To learn more about previous editions and sign up for it, go to herbig.co/newsletter.