Major Changes in Large Language Models (LLMs) You Need to Know in 2024

The landscape of large language models (LLMs) is rapidly evolving, and it’s imperative for developers, startups, and businesses to keep up with these changes to stay competitive. Here, I’ll break down the four major changes that are transforming the way we interact with and build upon these models.

Models Are Getting Smarter

It’s no surprise that models are becoming more intelligent with each iteration. The announcement of Anthropic’s Sonnet 3.5 is a testament to this ongoing evolution. But what’s critical is how we adapt our strategies to this intelligence boost.

Two Strategies to Build on AI: Sam Altman, in a recent discussion, highlighted two primary strategies for startups working with AI:

1. Assuming Models Won’t Improve: This approach involves building robust, intricate systems on top of existing models without expecting significant advancements.

2. Betting on Continuous Improvement: This strategy assumes that models will continue to get better, and thus, designs products to leverage future improvements.

While the former might seem safer, the latter is where the real opportunity lies. The key takeaway is to design products that function well with current models but are also scalable with smarter, more advanced models. This means being prepared to remove redundant processes as models get better, rather than adding unnecessary complexities.

Synthetic Data: Another critical factor contributing to smarter models is the rise of synthetic data. Training models on synthetic data allows for higher quality and more precisely formatted data, particularly for instruction fine-tuning and alignment. This method unlocks more of the model’s potential, enhancing their performance and adaptability.

Multimodality: The ability of models to operate across various modes (text, image, audio) strengthens their overall performance. Multimodality enables models to ground knowledge more effectively and improves their understanding and responses.

Tokens Are Getting Faster

Speed is becoming a defining factor for modern LLMs. The emergence of models like Grok, which generate tokens significantly faster than previous models, is changing the game. This speed increase is largely due to advancements in how GPUs and TPUs are utilized, allowing for quicker model serving.

Implications for Product Development: With faster models, several possibilities open up:

· Multiple Calls vs. Single Calls: Faster models make it feasible to perform multiple calls for decisions, enhancing the reliability of outputs.

· Reflection and Reflexion: These techniques, where models reflect on their own or tool outputs, become more practical and can improve the quality of results.

· Prompt and Query Rewriting: The ability to rewrite prompts and queries on the fly without significant delays can dramatically enhance user experiences.

These advancements mean products can be developed with lower latency and higher quality, significantly improving user satisfaction and engagement.

Tokens Are Getting Cheaper

The cost of tokens is plummeting, making powerful AI more accessible. Conversations in tech hubs like the Bay Area indicate a dramatic reduction in token prices, potentially reaching one-seventh or one-eighth of their initial cost by the end of the year.

Economic Impact:

· Affordable Power: The capabilities we once only had from expensive models are now available in cheaper, faster models. This democratization of power enables more startups and businesses to leverage advanced AI without prohibitive costs.

· Increased Competition: Lower costs lower the barrier to entry, leading to more competition. While this can be challenging, it also pushes innovation and quality improvement across the board.

Context Windows Are Going Infinite

One of the most exciting developments is the expansion of context windows. Google’s keynote hinted at a future where context windows are virtually unlimited, a significant leap from the current limitations.

Redefining Context Use:

· In-Context Learning vs. Fine-Tuning: With expansive context windows, in-context learning can often replace fine-tuning. By using extensive examples within the context, models can adapt and respond more effectively without the need for traditional fine-tuning.

· Context Caching: This technique allows for faster and more efficient processing, enabling real-time adjustments to context based on user queries.

· Dynamic Example Selection: Selecting relevant in-context learning examples dynamically, based on the query, ensures more accurate and tailored responses.

Preparing for the Future

As these changes unfold, it’s crucial to design LLM applications with flexibility and adaptability in mind. Here are a few strategies to consider:

· Abstract Logic and Prompts: Ensure that the logic and prompts in your applications can be easily updated to leverage advancements in model capabilities.

· Embed and Chunk Data Effectively: Develop systems that can quickly adjust how data is embedded and chunked, allowing for rapid testing and iteration.

· Monitor Economic Impacts: Keep a close eye on how these advancements affect the cost and profitability of your applications and be prepared to adjust your business model accordingly.

In conclusion, the world of LLMs is moving fast. By staying informed and strategically planning for these advancements, you can harness the full potential of AI to build innovative, competitive, and successful applications.