About feedback reliability in early-stage startups

Introduction

Feedback is fuel for growth. Neither startup could disagree on that, and specially the ones at early stage that are still validating their value proposal and growth strategy. Feedback allows learning and with learning comes better served products. Feedback also cleans the mist inside the "innovation black box", letting us see more than only assumptions and begin to rely on verified facts. Actually, early strategy and product should be purely based on feedback while only vision remains steady.

But feedback by itself is just an empty word if we don't dive into the gathered data. Provided that we do not care about feedback reliability and quality, we are essentially wasting time. Thus, it is not about accumulating copious quantities of customer opinions. We really want to make sure that feedback is relevant to us. At an early stage, we initially aim to extract valuable feedback from early adopters. But we must not enter in the pitfall of sacrificing feedback quality in order to increase the number of given opinions when we are lacking of enough data to comprehend our market.

In this short article, I will review feedback reliability according to the relationship between the founder and customer. Also, I will evaluate this relationship in different product context (free usage or must-paid). All these observations are in general terms, so there could be exceptions.

In order to visualize what I will write about, there are some useful graphs to understand better how feedback reliability could vary according to the sentimental bonds with the user.

For free products

For paid products:

Parents as customers

Whether a product is free or paid, parents' feedback may not drive the most trustful insight anyway. Naturally, parents tend to be more compliant and kind about product performance. I can not blame them. Logically, they would love to see their child's product achieve success. Their feedback is more inclined on positive reviews and encouragement words rather than an objective and neutral analysis. I believe its feedback has a high level of bias and we should consider it. Being said that, I would not suggest to stick closely to their opinions because it could not be a good measure.

Now, let's suppose you have a paid product and your parents have to pay for testing it. Their opinion would not be far distant than the free option. In fact, a mandatory payment could bias even more their review in favour of our "wellness". If the product subscription or payment is the primary (or unique) source of revenue, parents' feedback will be less reliable given the correlation between success and revenue perceived. In other words, they are more exposed to the main cause of our product growth, or at least, as it seems to be. If that payment is what allows product's growth, they will try to give their kindest review, weighing up the brightest side of your product and being aloof in telling you the flaws and drawbacks of your product. Parents will try to justify the payment despite product utility because they think your product has to sell well in order to reach success.

Friends as customers

On the other hand, friends can give you your first trustworthy insights about how the product is performing. But it is fundamental to not spread the word along all your friends. Instead of doing that, pick a couple of friends that may engage with the product in sake of relevancy. Friends' opinions tend to be less biased than parents', being more suitable to consider and execute. They want the best of us so they will point out good or bad things to make sustainable progress.

In this case, I think there is no strong differences between free or paid products. They will not bring you back more exhaustive reviews only if they have to pay for using your product. In both cases, free or paid, friends will give you useful feedback. Nonetheless, in front of a paid product they will act slighty as your parents did. Perhaps they could feel the pressure of being more supportive and would try to prove its price despite product performance could not fit what they expect from it.

Acquaintances (or friends of friends) as customers.

They are amidst between strangers and friends. Their degree of bias in feedback is even smaller than parents and friends. In this case, if the product is free our acquaintances may not be as considered and caring as our friends were. They will give you a neutral opinion about your product, but do not expect extensive analysis, at least if the person who you are targeting is very engaged with your product.

But if we turn into paid products, we could begin to appreciate a shift in the free option pattern. I think this is the stage were feedback reliability in paid products for lesser-known people begins to contrast with what we have observed in the free option. I mean, feedback reliability now keeps increasing instead decreasing, and in fact, acquaintances can give us more trustworthy opinions than friends. Why?

Putting it simple: the less we know a person, the more engaged will be in giving us product feedback when money is involved. Their spent is not aimed to help us raise our business (as friends or family might do), instead their consumption is driven by their own needs uniquely. They expect to see their money worth. Opinions become more neutral than ever. If they notice that the monetary investment does not pay off, they will let you know so. Regarding to the free option, as there is no monetary effort involved, people will be more lazy about telling you the product's flaws, because they do not expect ROI positive.

Strangers as customers

Finally, most commonly your set of customers will consist in people you have never met. A word of caution: they scarcely know who you are, and you are starting to learn from them. In most cases, they not even know what they need with accuracy. A free product would attract more customers easily than a paid one. But their feedback may lack of quality. As I said, most strangers will not engage as your friends did in providing detailed feedback. Your gain here is in feedback net quantity. Here, the key is to know which opinions should pick. We must filter languid opinions (or at least, work with them). A good way to preventing this happen is to control who is given access to your product test instead spreading your product around a marketplace. Although we do not them well, we should bring our product closer to early adopters.

As I pointed out above (acquaintances section), paid products foster more reliable feedback in our customers. With this I am not saying that free products aimed to the target audience yield lesser-quality feedback, but it is more likely to encounter users that are not willing to leave a meaningful review. The mandatory payment works as an incentive to speak out. Nonetheless, we should take into consideration that there are already many users willing to leave a complete and sincere feedback with or without payment. These are highly engaged users, the ones we always expect to serve. But in early-stage startups, we often pass for many types of customers until we reach the desired one.

Conclusion

We are used to read about feedback a lot. Articles highlighting its importance in startup's growth. Certainly, the best way to combat an extreme uncertain enviroment is to begin learning and that's why constant communication with our potential users is so important.

But I almost never read about the quality of the feedback we receive. I think we should take care of feedback reliability as well as we strive to get as many feedback as possible. The so-called Quantity vs Quality dilemma is also present here. We can not undermine quality for quantity. We should go out to get feedback, but it does not imply to consider all the feedback that is given in the same way. We have to be aware about how our relationship with different customers could distort the reliability of the feedback.

There are other factors that can create an exception to this observation. For instance, if your parents are truly neutral people and they are actually experts on the subject, in that case their feedback would be very credible to think about. In this analysis, I tried to be general instead of covering all the exceptions and diving into details.

In order to visualize what I wroted, there are some useful graphs to understand better how feedback reliability could vary according to the sentimental bonds with the user.IFor free products:

I hope you have enjoyed the reading! I am keen to hear about your thoughts, and keep learning with you while sharing different perspectives.