Customer Discovery with AI Agents

I spend a lot of time helping startups as they find Product-Market Fit (PMF) and I've found customer interviews to be invaluable. For years I've been using Otter.ai to record these interviews and I use its AI chatbot for summaries and retrievals.

Following a conversation over dinner, I wanted to experiment with ChatGPT to explore how agent workflows—a current hot topic in AI—could unearth insights I might have missed.

Experiment Success

The experiment exceeded expectations:

• I quickly transitioned from a Miro value proposition canvas to a well-written value proposition statement.
• The agents synthesised disparate information from around ten transcripts into coherent benefit statements.
• The agents facilitated faster iterations towards a solid value proposition statement compared to a single context window.

Despite the rapid progress, there’s ample room to refine the prompts. Here's a detailed account of my approach.

Agent 1: Custom GPT - Interview Assistant

Export Transcripts: I exported a set of customer interview transcripts as text files from Otter.

Create Custom GPT: I created a custom GPT “Interview Assistant” by uploading the transcripts with the following prompt:

You are a product manager at a cybersecurity company following a lean startup methodology. You use Jobs to be Done. Your job is to identify jobs, pains, needs, gains, risks, and assumptions; create and validate value propositions; and define features, benefits, and values. Always use the uploaded files, which are transcripts from interviews.

Generate Value Propositions: I asked the assistant to generate value propositions, for example:

I am focusing on a value proposition for industrial OEMs like ABB. Reference your relevant materials to help shape these ideas into a value proposition...[LIST OF IDEAS]

Agent 2: ChatGPT Persona - VP Engineering

Identify Potential Buyer: Our customer discovery pointed to VP Engineering inside OEMs as a potential buyer.

Create Persona: Using a regular ChatGPT 4.0 window, I prompted:

Adopt the persona of a VP Engineering inside an industrial device OEM like Siemens, Philips, or ABB. You receive this description of a new security service you are considering purchasing. Critique it, commenting on how pressing the need is, alternative solutions, highlights from the value proposition, and areas for improvement. Be fair but critical.

Draft Review: I pasted my draft value proposition from Miro directly into the chat. ChatGPT’s flexibility with unstructured inputs is particularly helpful early in the process. Even Miro stickies can yield great results.

Glean Insights: This step revealed insights I had missed—novel phrasing and clustering of value-adds into a concise value proposition.

Agent 3: ChatGPT Persona - Master of Customer Discovery

Feedback Integration: I copied the output from Agent 2 into Agent 3, using this prompt:

Adopt the persona of a master of customer discovery like Steve Blank or Matt Lerner. You have completed your user interviews and received this feedback from a VP Engineering. Consider all the feedback, decide which you agree with and propose changes to improve the value proposition.

Iterative Agent Interaction

Agent Collaboration: I iterated back and forth between my three agents (the interview assistant, VP Engineering, and the master of customer discovery).

Rapid Gains: Most significant gains came quickly, suggesting limited benefits from constant iteration. Different agent viewpoints proved more valuable.

Human in the Loop

Refinement: While I cold-pasted some outputs from one agent to another, I maintained a human oversight loop most of the time.

Polishing Statements: I refined value proposition statements, removing stylistic blunders and false promises.

Efficiency: This approach saved several hours and potentially reduced iteration cycles with the engineering team.

Future Enhancements

Contextual Feeding: Providing agents with more context about the current product, including documentation or code, could help identify pain points and solutions I might miss.

Additional Personas: Adding personas like a CISO could help evaluate security holes in my solution.

Conclusion

AI agents are a fast, powerful way to get greater insights from customer discovery.

Thanks to Alexander Walsh, CFA and Adam Martin for pushing me in this direction.