Expectations from 2024

As we step into 2024, the atmosphere is charged with a sense of anticipation that sharply contrasts with 2023, primarily because of our expectations in the AI arena. We are poised for a plethora of breakthroughs. We anticipate advancements from both established players and emerging new entities in the field. While the specifics remain unpredictable, there is confidence in the efficacy of the models. The real question is, whose story is going to be leading the charge when we're looking back at all this a year from now?

We started last year with chatGPT and that was an iPhone moment but soon realized that it's going to be the race between multiple players with billions of dollars to fight for.

The things I expect in 2024:

Models:

I anticipate continued progress in AI models, with emerging developments already stirring interest. There's talk of a new LLaMa model on the horizon and potentially an updated version of GPT-4, though it's uncertain if it will be named GPT-5. We can also expect an upgraded version of Anthropic's Claude. Beyond these, I foresee the emergence of numerous smaller models, similar to Mistral's 7B-MoE, from various providers.

This year, I expect AI models to be benchmarked against GPT-4, much like last year's models were compared with GPT-3.5. Additionally, I anticipate a significant expansion in multi-modal capabilities from single providers, integrating vision, text, and audio. This convergence of different modalities in AI models will likely open new avenues for applications and innovations.

Context Length:

Currently, most AI models, have a context length ranging from 16,000 to 32,000 tokens, with some exceptional models reaching up to 128,000 (GPT-4 Turbo) or 200,000 (Claude 2.1) tokens. For 2024, I expect a trend towards models incorporating even larger context lengths beyond 32,000 tokens. Although extending context length entails handling more data, which in turn requires greater computational power and increased costs, advancements in hardware technology are improving price-to-performance ratios. This development might encourage model builders to explore models with larger context windows.

Such an expansion is particularly relevant in software engineering, where comprehending and documenting legacy code is crucial. For instance, in current models, a 32,000-token limit roughly translates to about 50 pages or 1,250 lines of code. This limitation is significant considering many software projects, especially those developed in object-oriented languages, span thousands of lines. The ability to process and understand larger codebases could greatly aid engineers, especially when dealing with legacy code created by personnel no longer with the company. Therefore, I am optimistic that 2024 will see breakthroughs in addressing this challenge, enabling AI models to handle more extensive and complex software projects

Architecture:

Today, the 'Attention is all you need' philosophy, emblematic of transformer-based architectures, has revolutionized the AI landscape in recent years. Even newer iterations, like those using a Mixture of Experts, have somewhat improved efficiency in training and inference by reducing computational demands.

However, as the need for explainability in AI increases, transformer architectures fall short in elucidating the rationale behind model decisions. Additionally, these architectures typically require substantial data for effective training, a resource not readily available to everyone. The industry is gradually shifting its focus towards smaller models (SLM) as opposed to large language models (LLM), but this approach still demands significant data input. This poses a challenge: what about scenarios where only a fraction of this data is available, particularly in private repositories, without resorting to public datasets?

I expect that in 2024, new architectures will emerge, designed to cater not just to models trained on large datasets but also to those with limited data availability. These innovations should aim to provide comparable intelligence and efficiency, regardless of the dataset size.

Hardware:

The year 2023 was dominated by GPUs, propelling Nvidia to become a trillion-dollar company. I anticipate that 2024 will continue this trend, perhaps with even greater demand for GPUs. This expectation stems from the fact that many companies, having spent the last year developing proof-of-concepts and low-risk applications, are now poised to transition these projects into production. The next 18 months should see a surge in new products entirely reliant on AI, thereby increasing the demand for GPUs. However, it's unlikely that Nvidia will remain the sole provider. The GPU landscape in 2024 is expected to diversify, with multiple providers like Nvidia, Intel, and AMD, and possibly others who have been working on accelerators or supercomputers in recent years. I am biased towards Amazon's Trainium and Inferentia. These offerings could enable customers to efficiently train and infer their models, marking a significant shift in the GPU market.

Humanity:

In our increasingly automated world, we are progressively entrusting our daily tasks to artificial intelligence. This reliance is becoming second nature, opening up unprecedented avenues for creativity. As we offload routine activities to these AI systems, we are empowered to accomplish more in ways previously unimaginable. However, it's crucial to avoid complacency. We must continue to engage our own thinking, applying personal reasoning and creativity. Drawing a parallel from Ironman, we should aim to be like Ironman with Jarvis as a sidekick, not the other way around. We need AI systems to assist us like Jarvis, not to replace our roles as the primary actors. By 2024, I anticipate that we will each have multiple 'Jarvis-like' AI sidekicks, aiding us in both our personal and professional lives.

What are your expectations from 2024?

Shameless plug:

Interested in Leading the AI Revolution in Your Organization?

Discover my "AI for Leaders" course – a program designed specifically for forward-thinking professionals who aspire to harness the power of AI in their organizations. This course isn't just about understanding AI; it's about becoming an AI leader.

Next batch starts on January 12th

Why Enroll in 'AI for Leaders'?

• Highly Rated: Join a cohort of successful alumni who are now driving AI initiatives at the forefront of their companies.
• Practical and Relevant: Learn through real-world case studies and hands-on projects that will equip you with the skills to implement AI strategies effectively.
• Expert Guidance: Benefit from my experience and insights as we explore the intersection of AI and leadership, preparing you to make informed decisions in this dynamic field.

Want a social proof? Here is what one of my student posted!

Whether you're an executive, a manager, or an aspiring leader, this course will empower you to be at the cutting edge of AI leadership.

Don't just watch the AI transformation unfold – be a part of it!