Core Tenet 2: Chase Perfection

There are certain principles or ways of working that have served me rather well over the years. I’ve talked about some earlier:

• Patterns for Problem Solving
• Anti-patterns for Problem Solving
• Core Tenet 1: Trust the user 

Today, I’ll talk about one more pattern/tenet, that has helped me drive dramatic impact.

Chase Perfection

Given a problem, start by asking ‘what will it take to be perfect?’ Alternatively, ‘why can’t we be perfect’?

This might sound impractical to you.

We can never achieve perfection. It’s easy to talk about perfection. Are you crazy?

No, I’m not.

Or perhaps you feel this is a trivial question. What’s the big deal with asking this question?

Of course, we can never achieve perfection. That’s not what this is about. The magic is in asking the question. Initially, it generates scepticism or laughter. But once people realise you’re serious, it’s as if they experience a Eureka moment.

Why does it do that?

It changes the perspective completely – it forces us to think about the fundamental core issues, as opposed to the immediate impediment. It yanks one out of the day-to-day ground level view, and asks us to zoom out and look at the 10,000 foot view.

It liberates us – gets us out of our thought prison, breaks the mental blocks we have applied on ourselves.

It resets expectations, and leads us to re-examine our assumptions of what’s normal.

And just by doing that, it enables us to discover and solve for dramatic impact.

Let me share a few examples from my experience – they are spread across businesses, and cover different areas. Try using this as a technique in your work, and let me know how it goes!

Example 1: Billing Issues at Ola

At Ola cabs, we used to have a lot of complaints (from both riders and drivers) that the final bill was very different from what they were initially shown. Drivers, specially, claimed that they drove more kms than what the bill showed.

Working with the billing team, we quickly realized that this was due to GPS issues- often, there would be gaps in the GPS signals we received from the driver app, due to network or GPS issues. For such situations, we had a simple algorithm for estimating the actual path taken. (GPS is the tech that allows your phone to track your location)

We started by asking ourselves the question- how do we know our algorithm is doing a good job?

Interestingly, we didn’t know. We had not developed a metric to measure how well we were doing. We didn't have a point of view.

So the first thing to do was to fix that. For the most part, when we were getting regular GPS signals, there was no problem. With regular signals, the distance travelled between two GPS points would be, say, 5 meters. And there would typically be only one path on the road between two points so close together.

The problem was when the gap was higher – say, 100 meters. That’s when estimation was required. Now, there were two possibilities – if there was only one possible path between the two points, our estimated path would still be 100% correct. But there will be situations where there was more than one possible path between these two points. That’s really where we could go wrong!

When we started measuring our ‘billing confidence levels’ based on this principle, we realized we weren’t doing very well. That was a start! At least we now had a measure for a user problem!

But once we got here, the ideas and solutions coming out from the team were very incremental, would take a lot of effort to implement, or were too complex. Mostly, people were stumped.

Then, we asked the perfection question:

What will it take to get the confidence level to 100?

When people pushed back, claiming this was absurd, we talked about billing confidence in other businesses – how often do we see billing errors in, say, retail? Why should our customers expect anything different for cabs?

Once people warmed up to this 0% billing error concept, the quality of discussion and solutioning completely changed! Fundamentally different approaches and solutions were thrown up and discussed.

Going through the solutions will need a separate post, so won’t attempt it here. Having said that, we talked about:

• ways to get the missing signals, (like using the rider’s app to fill the gaps from driver app), and
• using new signals and data points to be smarter about finding the most likely path between two GPS points far apart.

Very quickly, we found and implemented solutions that moved our confidence levels up dramatically. (I don’t remember the numbers now, but we might have moved from 85% to the mid 90s. Earlier, we were struggling to make even a one percentage point improvement.)

Example 2: CSAT 100

Both at Ola and at MakeMyTrip (MMT), when talking about customer satisfaction in the support function, we were in the mid 20’s to early 30s in CSAT scores. Initial efforts were incremental, with huge efforts spent hardly moving the needle.

But once we asked ‘why are we not at 100?’, things changed. I challenged the team to target 100% CSAT score. And that helped us to quickly leapfrog to the mid-60s levels.

I’ve talked about CSAT 100 in detail earlier, so won’t repeat it here. You can take a look here... CSAT 100.

Example 3: Fare/availability failures at MMT

When users search for a flight at MMT (or any other online travel agent (OTA), for that matter), they are served results from a locally stored copy (cache) of various flights and fares. (This allows OTAs to serve results fast – else they would have to call each airline and GDS system for every user, which will take a while. Also, each API call to these systems has significant cost. So for best user experience, as well as efficiency, OTAs cache results.)

But this also means that sometimes the information is out of date – availability or fares might have changed in reality. And the users discover this once they get to the payment stage (or sometimes after making the payment!)

MMT had a dominant share of the domestic flight business (at 16% plus), and broadly provided a great experience and support. But this was one factor that was not great- cache failures were high – in double digits. When the air product and tech team started looking at this seriously, they spent time optimizing caching strategies – a lot of effort got spent there. We did achieve significant improvements by doing that, but there was quite a bit of ongoing effort involved.

And then someone in the leadership team raised the question:

“Why should failures not be zero? Specially given our scale!”

Once this was asked, things started moving in a different manner. And very quickly, we put two things together to reach a perfect solution.

GDS systems/airlines weren’t too keen on the number of API calls we were making- they were being forced to invest in increasing capacity and throughput on their systems. They wanted us to reduce the hits. (And we actually had been increasing the hits even more to reduce failures on our side.)

So we had a customer experience issue, we had a cost issue, and we had unhappy partners.

Someone then proposed the radical thought – what if the airlines were to push availability and fare information to us whenever there was a change? (Rather then us asking them every so often). If that was to happen, we would be guaranteed to always have up-to-date information, and airlines could reduce their IT spends dramatically.

This was a win-win approach, and this strategy started getting implemented for some airlines.

Note that there was no reason this couldn’t have been done earlier. Airlines, in fact, were pushing us to reduce API calls! The only reason this got done was because the perfection question was asked, and the resulting change in perspective. 

Example 4: Driver Impersonation at Ola

One of the safety related features we started in 2018 was driver selfie check – drivers would be asked, at random times, to click a selfie. This would be compared against the photo we had on record to check if the person driving the car was the person we had onboarded as a driver.

For a few months, the team was struggling to improve things much. We were at about 20% impersonation rate, with a daily coverage of only ~6% of drivers. This was bad. But despite the team’s best efforts, we were making very little progress!

And then we asked, yes, the perfection question – what will it take to be at 0% impersonation rate with 100% daily coverage?

With this question, the approach changed completely. People reviewed their core assumptions. Radical thoughts came up. We stepped outside our comfort zones. Things people had thought that supply team would never agree to were voiced. Fundamental issues were brought up.

In a matter of a few months, we reached 0.1% impersonation with 80%+ daily coverage!

The story of how we got there deserves a separate post, but here’s a summary:

• We realized drivers weren’t even aware that two+ drivers could share a car officially. The sign-up process for that was not easy to discover and execute. That got fixed.
• In the past, governments required cab drivers to have a commercial driving license. A lot of drivers only had private driving license, and therefore couldn’t sign up officially. However, the government had relaxed the regulations- people were simply not aware. We just needed to market that better.
• We did targeted work with fleet operators – they could fix this rather quickly. Same with Ola’s own fleet.
• To push this along faster, we put in place strong, staggered penalizing mechanisms. And kept tightening them iteratively. 

Example 5: Instant Refunds at MakeMyTrip

Refunds due to booking cancellations was a major support and experience issue at MMT. For obvious reasons – travel is expensive, so the rupee amounts are large.

The support, product, and tech teams had been working on this for many quarters. Over multiple quarters, based on the improvements, they tightened the metric itself - they changed the core metric from %refunds processed in 3 days, to %processed in 1 day. But even then, we weren’t covering ourselves with glory. We were at about 70% in 1 day, and 97% in 3 days.

This was frustrating to everyone – the leadership, the customer, and even the team that was working on refunds – no one likes to have spent many months on a problem, to be told ‘not good enough’.

At this point, we asked ourselves – if payment is instant, why not refunds? Why does it have to take even minutes, forget hours or days?

What will it take to process 100% refunds instantly?

There was initial scepticism and pushback – instant refunds is  impossible. What’s the point, anyway? Customers can take a refund to wallet if they want instant refund. Etc.

But once people got beyond the mental block, things changed rapidly.

Within a couple of quarters, instant refunds went up from 0% to 75%. And 1-day refunds went from 70% to ~95%.

What we were left with were some edge cases where airlines had not automated cancellations. And some edge cases where payment partners had not automated refunds for some payment modes. (And teams were working actively with the partners to get this fixed too)

BTW – we discovered and fixed lots of other issues in the process of doing this. Automation coverage of both cancellation and refunds increased significantly. In addition, we uncovered quite a few issues in our asynchronous processes – there were always cases that would fell through the cracks. That stopped once we made everything synchronous and instant.

Customer support calls for refunds went down dramatically.

Its not like people were not trying to do the various automation pieces earlier. They were, but progress was always slow. This time, things moved rapidly. Why? Because the 100% instant refunds, by acting as a clear, north star goal, pulled everyone together into a clear mission. Across the org, it acted like a forcing function.

Example 6: Zero cancellations at Ola

Cab cancellations at Ola were a significant source of discontent for both riders and drivers. Depending on the city, cancellation rates were 28 to 35%. For an on-demand service, this was ridiculous!

Again, just like the pattern we’ve seen in other stories above, teams had been struggling to move this metric. Despite multiple experiments, including deploying machine learning models, the metric didn’t move at all.

Over time, people started considering this as a fact of life. Some even claimed this was a feature, not a bug!

I have talked about this in detail here (Fixing Cancellations at Ola), so won’t repeat myself today. But the summary remains the same – when we postulated that cancellations need to get to zero, (A 100% reduction, not 20/35/50%) everything changed. The teams found fundamental issues, and fixed many of them. Overall cancellations quickly went down 25%, and driver-at-cause cancellations went down 50%.

In addition, root causes for another 30-40% of cancellations were identified and solutions were ready to be executed. While we were still very far from getting to zero, we had made dramatic reductions. And that happened only because we asked ourselves the perfection question.

Other Examples

There are quite a few more, but I’ll briefly mention just two more here:

• Booking Denials by hotels at MakeMyTrip. Hotels would sometimes refuse to honor a booking. This was called ‘stuck’ booking, and was suffering the same fate of incremental improvements. Once we targeted zero stuck bookings, we achieved dramatic reductions.
• Disk Utilisation at Madhouse. Madhouse was a subscription based DVD rental business. We had quickly realized that our biggest cost was the DVD cost, and therefore increasing DVD utilization of popular titles was critical to profitability. When we targeted zero idle time of a DVD, whether in-store or at a customer location, we achieved very significant improvements. Total turnaround time of a popular disk (time between rentals) went from an average of 6+ days to ~2.5 days.

Have you seen the perfection question in action? Would be great to hear about your experience. If not – try it out!