AI-Powered Knowledge Base for Product Managers
Building an AI-powered Knowledge Base for Product Managers
An IBM study shows product managers are early adopters of generative AI. They rank in the top-ten professions that use AI. The report states that 21% of product managers use AI daily. Product Managers are leading the AI charge.
As AI's role in product management increases, product managers must learn how to leverage AI to stay competitive. Product managers using AI differ from their traditional counterparts by leveraging their technical expertise to harness AI's potential in enhancing product management processes
Let's cover some practical ways to use generative AI in your product management role.
How AI is Revolutionizing Product Management
As technology advances, industries find new ways to incorporate AI into their processes, including product management. Companies succeed or fail based on effective product management. Product management is essential to any successful SaaS company or software product company.
The challenge is consolidating real-time product information
Challenges in getting the correct contextual information to inform decision making
Product managers need to find relevant information to make key decisions. The problem is that the product data might be spread across multiple portals, documents, and systems. This results in lower productivity or poor choices because the required knowledge is not at your fingertips.
Consolidating real-time information across tools like Figma, ServiceNow, Slack, JIRA, GitHub (PRs, releases), CloudWatch (logs, metrics), DataDog, customer metrics, project plans, WIP tracking, business analyst spreadsheets, market data, product data, and customer feedback channels is a daunting task for any product manager. Challenges in finding information can significantly affect product quality and timelines. Then if you do collect all of the data, the next challenge is finding the right product data to make informed decisions.
But having a wealth of knowledge and information is only worth little if you can't find it when needed. It has to be accessible, discoverable, and up-to-date. Having an abundance of information is only valuable with recency. Old data can be as helpful as drinking spoiled milk. Having the data and being unable to find and use it is also useless. Getting data from multiple sources to make informed product decisions is often manual and time-consuming. Correlating data across disparate systems is also tricky.
AI Knowledge Base Solutions
One solution to these challenges is generative AI. Large language models (LLMs) can be fine-tuned for domain-specific text, allowing for deeper understanding. Semantic search with embeddings offers agility and scalability. Semantic search lets product managers easily access essential information with a simple search. Now it is easy to find that needle in the haystack answer to inform product decisions quickly.
Continuous knowledge management
Main AI Options
Generative AI tools like ChatGPT, Bard, Perplexity, Poe, and Anthropic, can be used for generation, classification, extraction, and summarization, all of which are essential components of product management. AI can streamline product development, enhance collaboration, and improve decision-making in SaaS and software product companies. All tech companies, really, and these days companies are not using technology for logistics, planning, operations, automation, data analysis, etc.
The main AI options for product management are fine-tuning LLMs, semantic search, AI-enhanced search, and smart agents:
With generative AI, the key to getting good answers is the right contextual knowledge. The correct contextual information means good quality data related to the task at hand--focused data. The main AI options for product management include fine-tuning LLMs for specialized understanding, but scalability concerns may arise, and the knowledge of how to do this remains very specialized. Semantic Search with Embeddings is flexible, but they rely on well-maintained knowledge bases. The quality and recency of the data are essential to get the best answers—garbage in equals garbage out. Maintaining a high-quality knowledge base requires rigorous data hygiene and frequent scraping, scrubbing, and ingestion.
Using AI-enhanced search (e.g., Embeddings, HyDE) allows you to comb through many data sources and answer direct questions spread across multiple documents in various systems. Think of it as English as the "new programming language" you ask, and it goes and finds it.
It can answer particular questions: “Who won the Super Bowl 1966? Who won the MVP? Give me the teams involved and which team won the coin toss?” As long as you provide the proper context to the LLMs, it can provide additional insights. Smart Agents with Vector lookup mix-ins offer the most relevant parts of workflow for context and get specially tasked via prompt engineering automation, chain of thought, tree of thought, synthetic prompts, and customizing AI for task/use cases for best results.
Smart agents powered by LLMs and Idea databases allow you to be the Tony Stark of product managers, but just J.A.R.V.I.S aspect, no hand-rocket propulsion.
Recommended by LinkedIn
Understanding AI Enhanced Search, Vector Databases, and Text Embeddings
AI-enhanced search is a form of semantic search that relies heavily on looking at a multidimensional vector called text embedding. Think of text embedding as an idea key. With this idea key, you can look at parts of a conversation or document, in the case of a chatbot, or some other workflow, in the case of an agent. Then the agent can see which parts of this workflow or conversation are the most relevant to the current step in the workflow and mix that content in with the instruction to the LLM. A vector database is an idea database that does not look up things by keys or queries but by taking the text embedding, which is an idea key, and looking up other ideas.
The first step in an AI-enhanced search would be to take the question and ask the generative AI (LLM) to create what an ideal answer would be like (HyDE), or perhaps several variations of an ideal answer. Then, It will take that same question and create many queries to downstream search engines or specialized databases. It can also use idea databases (vector databases). Or it can even take the English question and form a series of queries and searches. Then it takes these searches that it just generates and uses them to perform searches or queries in various downstream systems.
Then it compares the text embedding (idea key) you made for the ideal answer (or answers) and sorts the results from the searches based on how closely it matches the ideal answer (called HyDE). Then it gives you an answer by combining excerpts from several documents found in various systems.
It quickly automates the search generation, collection, sorting, and quickly sifting through vast amounts of data. What could take you hours or days to find through searching each system now takes mere minutes or seconds. This lets you use plain English and get complex answers from many downstream systems. This data can be up to the second if needed.
Idea databases (Vector databases) are staples of creating smart agents and any advanced prompt engineering. Remember, the key to using LLMs is context, having the correct contextual data for the task at hand.
An intelligent agent can act in specific roles. Perhaps the agent acts as a UX expert who knows all of your style guides or a sports fanatic who knows all the team stats for fantasy football. Imagine creating high-quality virtual textbooks on performing complex tasks with sample input and output and a wealth of role-based knowledge that you can seed that agent with—then seeding the agent with role-level instructions. When the agent hits a task, it can decide which parts of the conversation or workflow are the most important (using the same idea key and idea comparison above), then load the task-related contextual data from the virtual textbooks (or other sources using an AI-enhanced search) which have been stored in your idea database. It generates an idea key for the task at hand, pull the parts of the workflow that match, and load enriched specialized contextual information from the idea database. Thus for each step, it is always passing the right context to the LLM to perform the next task.
To make the agents even more brilliant, you can provide feedback loops to give them a score on their current task or even have them create multiple solutions, follow various paths, score all of the outcomes, and pick the highest score from the ones that did not fail. This is approaching Tree of Thought levels. With minimal feedback, proper prompt engineering, and mixing in the right ideas with a feedback loop, I have seen the accuracy of complex tasks move from about 70% to 95% plus. Using this technique, I have automated complex tasks that would take a person months (and more recently years) to do that can be done in mere hours. The accuracy can increase with more time and better training and feedback loops.
Conclusion
In conclusion, successful product management is the key to product development and continuous improvements and has tremendous ROI. The role of the product manager continues to become more complex. The abilities of the product manager are only enhanced through AI integration. It is already essential for real-time access to information. Careful consideration of approaches and rigorous best practices must be maintained to ensure that AI provides value and improves the overall quality of the product. For technology companies and companies that rely on technology, implementing AI for product management is a give. It is now a competitive advantage. Later product managers and companies that don’t adopt it will be left behind. Generative AI continues to shape the future of the industry. As a CIO, CTO, VP of Product Management, Director of Product Management, or Product Manager, staying ahead of the curve and incorporating AI into your product management process is essential.
Product managers are increasingly incorporating AI systems into their work. The new trend is that Product Managers using AI can also assist product teams in capitalizing on AI's commercial potential. Some question if all product managers will one day be AI product managers. AI Product managers differ from their traditional counterparts by leveraging their technical expertise to harness AI's potential in enhancing product management processes.
Follow-up and references
Product management teams often face challenges in consolidating real-time information across various tools such as Figma, ServiceNow, Slack, JIRA, GitHub, CloudWatch, DataDog, custom metrics, project plans, WIP tracking, and customer feedback channels. These challenges can affect product quality and timelines. AI solutions can help address these issues and streamline product development, enhance collaboration, and improve decision-making. Some AI solutions and insights include:
There are two main AI options for product management teams:
Product managers are increasingly incorporating AI systems into their work. The new trend is that Product Managers using AI can also assist product teams in capitalizing on AI's commercial potential. Some question if all product managers will one day be AI product managers.
Resources
Products and companies that can assist product managers in enhancing their decision-making, collaboration, and overall efficiency include:
Follow-up articles from the author
Marketing | Branding | Social Media | Partnerships
1yThanks for the shoutout to Collato 💜 We're preparing a new version that will be even more helpful for Product Managers!
Engineering Consultant focused on AI
1yKim Morello have you seen this one?