- A profit pool can be defined as the total profits earned in an industry at all points along the industry’s value chain. A profit-pool map answers the most basic questions about an industry: Where and how is money being made?
- Lets look at this simplfied Gen AI stack
- Vertical apps: Full stack apps with proprietary models (might be using 3rd party now but will move to in-house models eventually)
- Foundational Models: Commercial (not open source) models such as OpenAI, Anthropic, Cohere, etc.
- Cloud Services: Running the managed inference service (e.g MosaicML or AnyScale) or hosting vector database
- AI Dev platforms: They are platforms that helps enterprises build an end user app like chat or customer service. Includes front-end development, connecting data sources, fine tuning, prompt engg, RAG, etc.)
- Enterprise apps: End user apps (either employee facing or customer facing) such as chat bot, enterprise search, customer service, etc.
- Monitoring: This is full stack run time monitoring for performance, cost, hallucinations, etc.
- Developer tools: Tools such as Langchain with abstractions, sdks to build single or chained apps
- Security, Governance: This includes model security (training, data poisoning, drift, run time attacks), product security (LLM firewalls, etc) as well as end user security (DLP, governance, access control)
- So which products will make the most profits? I am taking a stab at predicting the profit distributions based on conversations with vendors, customers and the value each product brings to the end goal - either bringing new revenue or productivity/operational gains to the customers
- Vertical Apps: Generative AI (GenAI) is poised to deliver significant advantages in terms of productivity, knowledge, and content creation, which would otherwise be costly and resource-intensive. Certain applications have the potential to establish a data moat, leveraging network effects to gain pricing power and reduce customer acquisition costs. If these GenAI models are relatively compact or originate from open-source solutions, they can be more cost-effective to maintain and support over time. Vertical application vendors are likely to take an in-house approach to develop essential infrastructure tools such as monitoring, tuning, certain security features, and cloud deployment capabilities. This internal development can enhance profit margins by reducing reliance on external solutions and tailoring tools to their specific needs and business strategies.
- Commercial Foundational Models: For horizontal use cases like code, language, content, video, audio generation, quality will be key. Leading model providers will continuously train models and provide them as a service to developers or building end-user applications. Currently we have OpenAI, Google Bard/Vertex, Cohere, Anthropic, A121, Hugging Face, Mistral, StabilityAI, MidJourney, Poolside, Inflection, etc. This is becoming a competitive space and becoming fragmented. You really need economies of scale to amortize the model building costs over a huge number of users. I expect some consolidation, prices to drop and hence lower margins compared to vertical apps
- Cloud Services (Inference, Storage, etc.): Similar to OpenAI partnering with Microsoft Azure for a cloud service (for ChatGPT and the API service), other model providers will need a GPU cloud service for inference serving. Vendors like MosaicML can also serve open source models as a service as an alternative to the commercial foundation models mentioned above. Anthropic has partnered with Amazon. CoreWeave and Lambda Labs provide GPU hosting. MosaicML (Databricks), AnyScale Endpoints, Replicate are some non-cloud providers. Vector database are another example of cloud service providing storage for retrieval. Economically the large cloud providers or cloud service providers like Databricks, Snowflake, MongoDB are better suited. But the end to end fine tuning for costs or performance is an opportunity for pure plays to exit. Could be niche but highly profitable.
- AI Development platforms: As customers are scrambling to build Gen AI apps, vendors are building platforms to help them build Gen Ai based copilot/agent/chat products. They provide interface to data and access to models, some front-end UI, fine tuning, RAG, etc. There is no moat in these products and customers will eventually build bespoke applications instead of relying/paying for a 3rd party vendor
- Developer Tools: This particular category falls within the broader domain often referred to as "tools and shovels." Orchestration tools like Langchain, Fixie, Dust, as well as deployment and scalability tools like Weights and Biases, have gained popularity for their role in expediting the development of Proof of Concepts (PoCs). These tools offer valuable abstractions and orchestration capabilities, enabling rapid prototyping and experimentation. However, as businesses continue to explore AI applications, they tend to seek greater flexibility and control. This leads to a growing desire to open up interfaces and establish internal platforms. In the realm of AI, the real value often lies in the last-mile customization, tailoring AI solutions to specific use cases. Moreover, this customization requirement can vary significantly from one use case to another. Therefore, businesses are inclined to develop their internal platforms and interfaces to meet their unique AI needs and maximize the potential for customization and optimization.
- AI Monitoring, Observability: This category falls under the "tools and shovels" domain. Previous-generation vendors such as Arize and Fiddler have experienced a resurgence in relevance with the advent of Generative AI (GenAI). Many of these vendors are expected to expand their existing product offerings to cater to the GenAI market. Standalone GenAI-focused companies may find it necessary to consider consolidation as the industry matures. One intriguing aspect of GenAI is the constant necessity for "monitoring" to detect hallucinations, drift, and the need for fine-tuning models and their outputs as data and models evolve. Foundational model providers and companies developing specialized vertical applications may find themselves compelled to develop in-house tools for these purposes as they become integral to running their businesses.
- LLM Governance/Security: Many businesses are expected to integrate some form of Generative AI (GenAI) into their operations in the near future. While GenAI offers remarkable generative capabilities, it also comes with persistent challenges, such as unintended outputs (hallucinations). To address security and governance concerns, third-party tools will be essential. Furthermore, since GenAI will be used across multiple applications, it makes sense to have a unified tool for governing and securing these technologies. Many organizations may be willing to invest in such solutions, but they may opt to use open-source options for infrastructure components spanning from #4 to #6 in the context you mentioned.
CEO & Co-Founder @ Hydra Host | Forbes 30 under 30
1yBecause the biggest cost is infra, optimization in the core infra and hosting is going to be the biggest business of them all. We are years and years away from GPUs being similarly priced as CPUs. If you accept that open source is going to be the core market motion for AI adoption, infra optimization is going to be even more important.
Completely agree that Governance and security solutions should come from 3rd parties based on Cloud security experience.
is not opened for a career change! | DevOps & Site Reliability Engineer | STACKITEER | AWS Certified Solutions Architect & System/Database Operations Administrator
1ynice info 😁
Entrepreneur, CEO, Executive, Engineer, Mathematician
1yGreat analysis as always Pramod! I would think the share of Monitoring, Evaluation, RAG, tuning etc.. and LLM Security, Governance will likely be similar. Foundation models will actually get commoditized with the fast evolution of open source and exhaustion of newer organic datasets. That means it will lead to some price wars and will likely stack up like Cloud Infra.
Interesting view. What are example vendors you are classifying under "AI development platforms"?