5 Key Changes in Building and Buying Generative AI in 2024

In 2023, the explosion of consumer engagement with generative AI marked a tipping point with over a billion dollars spent. In 2024, the enterprise sector is rapidly catching up and revenue opportunities are expected to multiply.

Fortune 500 companies and leading enterprises are not only expanding their use cases for generative AI, but also significantly increasing their budgets, signaling a shift towards more strategic and production-oriented implementations.

This evolution from experimental to mainstream enterprise use is changing the way organizations approach AI in general, with a focus on scalable products rather than service-heavy models.

Industry leaders are now aligning their AI investments with regular software budgets and moving away from one-off innovation spend. This shift underscores the growing realization that generative AI has the potential to improve productivity, efficiency and customer service, resulting in significant cost savings and a high ROI.

In addition, the trend towards open-source models is gaining momentum, offering companies more control, customization options and cost efficiency.

Here are some interesting figures and charts to make the whole thing more tangible.

16 Changes to the Way Enterprises Are Building and Buying Generative AI

I came across the results of a16z (venture fund of two co-founders, Andreessen and Horowitz) survey of leaders of Fortune 500 companies + ~70 startups regarding AI plans. Below is a thesis squeeze of numbers from the market:

Budgets for generative AI are skyrocketing.

2024 budgets for AI are on average 2.5 times bigger than for 2023 (18 million vs. 7 million).

In 2023, the average spend across foundation model APIs, self-hosting, and fine-tuning models was $7M across the dozens of companies - nearly every single enterprise from the study saw promising early results of genAI experiments and planned to increase their spend anywhere from 2x to 5x in 2024 to support deploying more workloads to production.

In the same time, implementing and scaling generative AI requires the right technical talent, which currently isn’t in-house for many enterprises. This scarcity of specialized generative AI talent makes startups providing tools that simplify the integration and development of generative AI technologies internally more attractive, potentially accelerating their adoption in the enterprise sector.

Enterprises are trending toward a multi-model, open source world

OpenAI models are used by all businesses surveyed, with only Google honored by more than half of the respondents. Moreover, it's easy to notice that if we cut by stage ("in progress" VS "still testing"), the gap is incredible - 66% of already deployed in production solutions are based on OpenAI.

Despite the thesis above, all companies are testing multiple models and looking for alternatives. A third of respondents say they are trying models from three providers, and none say they are trying just one:

Measuring ROI is still an art and a science.

Calculating the return on investment (ROI) in generative AI is complex, not only due to the challenge of quantifying gains in productivity, efficiency, and customer satisfaction but also because of the unpredictable consumption of computational resources, such as tokens, which can vary widely depending on the use case. This variability can significantly impact costs, making it difficult to forecast expenses and accurately measure ROI. As enterprises continue to navigate the implementation of generative AI technologies, developing a reliable model for estimating these costs will be essential for assessing their true value and impact.

Open source is booming!

Many are looking at replacing API models with opsensor-based solutions. The main thing is the ability to control the behavior of the model (including control of the data fed to it as input), as well as customizing it for their needs.

46% of survey respondents mentioned that they prefer or strongly prefer open source models going into 2024

But there are still aspects, which are holding open source models from being adapted in the industry. Control (security of proprietary data and understanding why models produce certain outputs) and customization (ability to effectively fine-tune for a given use case) far outweighed cost as the primary reasons to adopt open source.

Leaders generally customize models through fine-tuning instead of building models from scratch.

72% of businesses responded, that they are fine-tuning models! My subjective percentage for fine-tuning would be lower. 22% just using RAG (Retrieval-Augmented Generation) and only 6% are creating custom models.

I wonder, if in this context "fine-tuning" was used wrong by some of the companies, rather as "adaption and integration" efforts.

Enterprises are excited about internal use cases but remain more cautious about external ones.

Interesting overview on popular tasks for LLMs and GenAI - The most common are text summarization, knowledge management within a corporation (read smart search), development assistance to engineers, and, surprisingly, already contract/document review, which many of us expected to be in production later this year:

In the last six months, the push within enterprises to embrace generative AI has intensified, significantly speeding up the adoption process. Deals that traditionally took a year to close are now being finalized within months and are notably larger - this rapid adoption spans the entire tech stack.

And we are only at the beginning of the journey: The global generative AI market was valued at USD 13.0 billion in 2023 and is expected to grow at a compound annual growth rate (CAGR) of 36.5% from 2024 to 2030 (link).

#GenAI #GenerativeAI #OpenSouce, with Jiri Kram , Christof Horn , Jonathan Tipper , Daniel Spiess , Thomas Reisenweber and many others!

We are at an inflection point for genAI in the enterprise and look forward to helping our customers and partners scale the solutions in this dynamic and growing market.

👉 How has your organization been navigating the integration of Generative AI? What challenges have you encountered, and how are you overcoming them? I'm eager to hear your opinion and challenges!