Applied Design Thinking Part 3

Ok so you've got testable ideas now what?

"When you talk to customers please remember you have four ears and half a mouth." I often say this to Planners, Product Managers, Engineers and Researchers before visiting customers. My intention: I want the team to learn more from listening than their own monologues. Deep listening affords the interviewers opportunities to both test hypotheses, and catch the unknown -- arguably the vanguard of innovation. I remember interviewing an electronics manufacturer and by deeply listening we discovered how they rigorously used leasing to ensure being up to date on technology. As they continued their dialogue they revealed a problem related to moving data from old to new arrays. Essentially, it was cumbersome, manual and error prone -- not a surprise to many and this problem is still an industry work in progress. During our discussion they indeed asked for help moving data between storage systems at the end of a lease. This very simple ask had a huge impact on my product's design at that time. It took the team directions we hadn't imagined possible, and overall enabled new business opportunities. However, that is a story for later as we must first talk about how to get to your target audience.

From the previous article I related how to create testable hypotheses and associated questions. So let's assume they're available to use in an interview process. So, what's next? Quite simply, the interview schedule! To start with there can be significant variation in customer interactions depending on the circumstance and that will dictate how to approach the schedule and logistics. Here are the types of interactions I've used:

1. Direct one-on-one in person interviews.
2. Direct one-on-one teleconferences.
3. "Speed dating" at an event like an Advisory Board. Interviewers and interviewees sit down and discuss specific topics for 30-60min. At the end of the time period interviewers switch locations and engage the next interviewee.
4. Surveys with online tools like digital communities, Google Forms or Survey Monkey to test ideas in advance of 1, 2, or 3 above.
5. As related to structured usability testing, engaging users in your target audience to test an existing system or functional prototype.

As you can imagine in each of these scenarios recruitment approaches, execution techniques and overall style vary. However, I've focused on interaction types 1 and 2, so for the purposes of this article I'll mostly focus on those two. As to why I focus on 1 and 2, well I find these two to provide the most utility because:

• You can remove customer concerns about revealing insights to their competitors.
• It affords the interview/design team a chance to gel and build simpatico.
• Customers/Users are more comfortable on their own turf rather than an artificial location.
• When possible, sales teams can learn more about how to use what they "have on the truck" than they knew before.
• You can build lasting bonds with customers over time, especially if you repeat visit.
• It actually costs less than setting up an advisory board.
• To develop a real business you have to do it anyway because in these early stages innovation spans not just the technology but the path to the customer too.

However, to execute one of these methods you have to recruit your target audiences, schedule the time and then execute your plan...

Section Summary

1. As much as possible set up face-to-face one-on-one interviews with customers in the field.
2. As a compromise set up teleconferences with key customers who have scheduling conflicts provided you also have face-to-face interviews scheduled.

Getting to "The Right Customer"

While this sounds really simple, in practice it isn't. It all starts with who you want to interview and for what purpose. A compounding factor: Determining who to interview and the interview purpose likely changes over the course of a project. To discern who you want to interview first party research, reading and discussions with experts are needed. Ideally, the results of this process will allow you to create user profiles that guide you to your interviewees. Guide is important because the profiles represent your ideal target, and ultimately getting to "The Right Customer" requires compromise. 

Compromise can come in a variety of forms, and I believe it largely depends on the surrounding organization you're a part of. Let's say you're tied to an organization that specializes in Information Technology (IT). This inherently limits your ability to reach a target audience beyond IT, especially if you're leveraging your sales organization. Conversely, if you're in a non-IT business and trying to enter another industry or pull in Information Technologies to push the boundaries for innovation you're similarly limited. A compromise way to get closer to your ideal is to pull in outside experts, and leverage their networks. This is a compromise to your budget because you're relying on an outside party/consultant who must be paid for their time and to open up their network.

For my recent projects, focused on Financial Services and Oil and Gas, we were able to engage strong outside experts like Nirvana Farhadi, Doug Gibson, Matt Hall, Mark Wiseman and David Pinski. We engaged them in a variety of ways to ensure that my team was able to move beyond our Information Technology bias. These experts brought their own human networks to the table allowing us to get close to our ideal targets. Specifically, they enabled conversations with Oil and Gas Data Managers, Geophysicists, Financial Services Asset Managers, Compliance Officers, industry specific small businesses, and on, and on. Without them we'd have been stuck in our own bias and only able to reach Information Technology folks in our selected industries.

I've also participated in efforts to strengthen existing products. In this case our indigenous sales teams set up time with the people they regularly work with. This is how I was able to reach the right team at that electronics manufacturer referenced in the first paragraph of this article. Our expertise and organizational capabilities afforded us the chance to not only talk about the customer's current pain, but also look at problems that weren't yet solved. For this customer the handling of migrations at lease expiration was where we found inspiration. As a result of this and other interviews we built data safety features (shred/format after migration), an easier to use UX, and overall improvements to an aggregated system (storage management software and the core storage system) enabling lease expiration use cases, migrations between competitors' products and more. These sorts of exchanges with customers, over time, enabled Hitachi Data Systems to have a material revenue stream, in excess of $300M, in the Storage Management area.

Section Summary

1. To mitigate organizational bias bring in outside experts in your target industry. You will have to compromise on your budget to get to the best possible outcomes.
2. When organizational speciality and business imperative match, this is a case where sales teams can help find the right target audiences. Compromise in this case comes in later stages as we will see in a moment.

Just how many do we need again?

Ok we've talked about ways to mitigate organizational bias to get as close as possible to your ideal or that we can complement an organization's strength with minimal compromise. However, even with all of the tools in your toolbox (i.e. expertise from the outside, extending an existing company's core mission, etc.) the righteously dirty work of setting up a schedule, confirming appointments, and then actually getting to conduct the interview is also fraught with compromise. For most projects I've ran we usually set a goal of 30 face-to-face interviews. We did this such that we had a broad enough sample to base decisions on. (I say this because 30, statistically speaking, is about the number of samples you need for any sort of statistical analysis with some significance. I'd like to thank Dr. Tsao for this pearl of wisdom.) However, there were times when getting to 30 customers was impossible due to scheduling mismatches, cancellations, sales teams focused on closing deals, etc. Actually, while morbid I had a case where a sales team called me up with a short notice cancellation. My initial reaction was anger until I realized the very human and sad reason behind the cancellation: The principal we were secheduled to interview died while on holiday. Therefore, we compromised on our schedule, travel plans, etc. Overall, while sad and extreme this event was a great reminder that working with people is touching, glorious and messy. As a late colleague of mine used to say, "Come early, leave late and be flexible!" What great words to live by beyond the dynamic world of interviewing a target audience. So, while you set a goal of some number of interviews to conduct you may not hit that goal an will have to compromise.

Section Summary

1. When possible try to get up to 30 face-to-face interviews across multiple cities, countries, regions and geographies scheduled, but understand that you may have to compromise that number.
2. Be there early and be flexible as some scheduled and confirmed interviews may suddenly be cancelled. Here you're compromising your time with your family and significant others.
3. Similarly be prepared if your consultant or sales team finds a "blue bird" interview and go with the flow. This may be a vanilla sales call or a fortunate turn of events, but don't turn down this compromise to your schedule.

Do we really have to go there? Oh and haven't we been here before?

If you're focus is on a global industry that is a key point to take into account for setting up your interview schedule. Essentially, not all perspectives you need come from one geographic location. Let me illustrate through a story: During our Oil and Gas work we determined that automated distribution and collection of massive numbers of seismic sensors in the field was important, after working with a partner in North America. However we soon learned the realities of the Oil and Gas business in China were totally different. Essentially, field workers who could distribute and collect seismic sensors were low to no cost (i.e. prisoners). This basically blew up the idea of building an automation system. So, while the idea sounded great in our own internal monologue and it fit in the developed world, it did not work in developing nations. If the team had not built in their schedule time to go to other regions we may have spent a lot of time building automation systems that would not have been purchased. Therefore, adding multiple relevant geographic locations to your schedule is critical to drive out country specific bias.

One visit to the same customer is seldom enough, and there are two reasons for this. Firstly, even though you've carefully planned who you should visit, confirmed and conducted the interview this initial meeting may not be who you actually needed to meet with. Perhaps this first interaction produced a lead to another candidate interviewee and you'll need to chase that down. This means a second visit. Secondly, you must build into your schedule follow-up time to talk with customers after the initial interviews are done. Actually, if you do not do these follow-up sessions interviewees can wonder what happened to the project. Questions like, "Did my input make it into the product?" arise and that is never a good thing. In fact, one downside to no follow-up: Users and customers thinking their input doesn't matter, and believe your group/company doesn't listen. However, if you do follow-up you'll find that the partnership with your users/customers can deepen, plus you may be able to recruit co-creation, alpha or early beta customers.

Section Summary

1. If you're focused on a global problem you'll need to ensure that you've scheduled appropriate visits in as many relevant geographic locations as possible to avoid country, region, etc. bias.
2. You'll need to plan on repeat visits to customers potentially for the purposes of really getting to the right candidate.
3. Additionally, a repeat visit to customers to discuss findings of the study and potential next steps in development is needed to ensure a strong relationship with your target audience. This secondary visit doesn't have to be face-to-face and could be done via teleconference either in group or one-on-one settings.

Wait so we don't have a firm schedule and we're still traveling?

If we brew all of these learnings together and reflect a bit it should be obvious that getting a firm and reliable schedule is nontrivial. Arguably, this phase of such a project is more challenging than any other part. It is this way because I believe a good part of this phase isn't in the team's control. The various parties involved in setting up the schedule include sales people, outside experts/consultants, users and customers. Given all of this and based upon my experiences the phase of producing a solid schedule can take a quarter or more. Here's a story that illustrates the challenge of making a quality schedule.

In late December, 2015 we wanted to socialize our Oil and Gas efforts with relevant sales people who sold into the Oil and Gas industry. To ensure we targeted the best possible sales people I went to one of my go to contacts JD Edmonds. He delivered and connected me to the right people, and we set up an initial briefing including a request to get to our ideal users. During socialization we covered the background and communicated our thinking about data management. Essentially, we use early hypothetical concepts to help the sales people understand and be sufficiently armed for robust interviewee recruitment discussions. I then left the task with the sales team to progress and checked in again in January, 2016. The response from the sales organization was, "We've got this." I checked in again one month later, and as you'd expect the response was, "We've got this." Diligently, I checked in again in March, and once more, "We got this." Well as you may have guessed it they didn't have it, and my next discussions were with outside experts Doug Gibson and Matt Hall plus back to JD Edmonds to get things set up. After that it took about 45 days to confirm an appropriate set of meetings getting us on the road by the middle of June, 2016. So from early socialization to actual meeting time took 6 months. Some of this was expected because we were focused on getting beyond our Information Technology bias. Other reasons for the delays include sales people under heavy pressure to deliver results, and us not being known in the customer base for doing more than Information Technology in Oil and Gas. Finally, I also suspect that the sales people were maybe trying to do it all by themselves instead of asking for help. If they had asked for help early on from say Doug Gibson or Matt Hall, the total clock time to generate the schedule may have been lessened. During this process my internal team was hounding me for the dates, were visibly agitated and overall expressed words and actions of worry.

Section Summary

1. Expect the total clock time for setting up a firm interview schedule to take at least 3 months. In my case where a transition beyond a company bias was needed it took 6 months before we got to visit a customer.
2. For those new to the process setting up the schedule will be stressful and cause feelings of anxiety. As the primary you'll likely have similar feelings to folks working with you, and you'll need to acknowledge those feelings yet keep everyone moving.

Conclusion and associated end thoughts

As I write this article I remind myself of a quote from the movie Goodwill Hunting by Robin Williams character Sean, "Even if I did know, I wouldn't tell a piss ant like you." There's more to this quote beyond the "colorful metaphor" that immediately precedes it. Essentially, telling someone isn't sufficient for true understanding of a thing. While in this article I've tried to guide you, the reader, down a specific path experience is the best teacher! Therefore, I encourage you to give it a go and if you have experiences that complement or refute anything I've put forward in this series please feel free to comment. One thing that I am pretty sure of, if you're adding this as a new discipline to your organization you'll find out for yourself that scheduling is hard. So, hang in there and keep plugging away until you've got something firmed up.

What follows are some additional thoughts which didn't quite fit into flow of the article, but I thought were worth mentioning.

1. Building up User Profiles can start by looking at profiles on LinkedIn, postings on Job Boards (ex - Dice.com), and job listings on your interviewee targets' web sites. This isn't perfect, but it will help in figuring out how to build up a profile, what their motivations might be, compensation ranges, etc. Heck, depending on your tenacity and courage you may be able to use LinkedIn or similar to have some early exploratory interviews. We used social media to contact a self proclaimed Seismic Data Geek, Matt Hall, and his help was beyond valuable.
2. If you have a team who is dedicated to performing user interviews I've seen that over time they will have an easier go of scheduling. Essentially, in the organization they become known for their purpose and when they come calling people respond more quickly and favorably. The team that I ran who focused on this could usually get at least two projects done per year, and the results were a bank of interviews which could be leveraged in a variety of ways.
3. If you commit to well managing your interview results, how to do this will come in later articles, then you can create a data bank which can be mined for future projects. I've used key term analysis and other text mining techniques to find past interviews which helped in bolstering or refuting hypotheses for new projects. In essence the first place to go for interviewing users are past interviews. Besides if you repeat visit reviewing what was done in the past can be helpful so that you show up with all of the context in your head.
4. Smaller "advisory boards" can be very valuable especially if you can leverage things like Matt Hall's un-sessions to quickly gather industry challenges from a larger set of users. The black and white photo in this article is from such a session my team and I executed in 2017, which was very helpful for digging in more deeply and establishing additional industry connections.