All the Non-NVIDIA GPUs, Please Stand Up

A surge of groundbreaking startups – like Tenstorrent, Cerebras Systems, D-Matrix, Groq, SiMa.ai, and Ola – is taking on NVIDIA’s dominance in AI hardware by developing highly efficient AI chips that claim superior performance, lower power consumption, and cost-effectiveness, competing directly with the GPU giant in data centres, edge computing, and AI inference tasks. Should NVIDIA be worried? 

CUDA or Nada 

All these startups have made substantial progress and some are preparing to launch their products in the near future. While having advanced hardware is crucial, the real challenge for these companies will be competing against the giant – CUDA.

These startups, which position themselves as software companies that build their own hardware, have come up with software to make their hardware compatible with customers’ applications. 

For example, Tenstorrent’s open-source software stack Metalium is similar to CUDA but less cumbersome and more user-friendly. On Metalium, users can write algorithms and programme models directly to the hardware, bypassing layers of abstraction.

Interestingly, they have another one called BUDA, which represents the envisioned future utopia, according Keith Witek, the chief operating officer at Tenstorrent. 

“Eventually, as compilers become more sophisticated and AI hardware stabilises, reaching a point where they can compile code with 90% efficiency, the need for hand-packing code in the AI domain diminishes,” he said.

Nonetheless, how these startups compete with CUDA remains to be seen. Intel and AMD have been trying for years, yet CUDA remains NVIDIA’s moat. 

“All the maths libraries… and everything is encrypted. In fact, NVIDIA is moving its platform more and more proprietary every quarter. It’s not letting AMD and Intel look at that platform and copy it,” said Witek. 

Today, NVIDIA’s CUDA has become the industry standard for parallel computing, with deep integration into AI frameworks like TensorFlow and PyTorch, while alternatives like AMD’s ROCm and Intel’s OneAPI struggle due to limited adoption, less mature software ecosystems, and weaker developer support, leaving competitors playing catch-up. 

NVIDIA vs the world 

However, one can not ignore the growing competition in the space. Tenstorrent, led by Jim Keller, is developing AI chips that claim to outperform NVIDIA GPUs in terms of performance, output per watt, and cost-efficiency by leveraging a higher compute density. 

Cerebras Systems, on the other hand, is focusing on AI chipsets designed specifically for large model training, with their WSE-3 chip being 8x faster than NVIDIA's DGX H100. 

D-Matrix, another contender, is targeting the inference market with its Corsair silicon, optimised for generative AI models up to 100 billion parameters, positioning itself as a more cost-effective alternative to NVIDIA’s GPUs. 

Groq, founded by Jonathan Ross, is also pushing the boundaries with products that are claimed to be ten times faster and cheaper while consuming significantly less power, specifically for AI inference tasks. 

Meanwhile, SiMa.ai is focusing on edge computing, developing chips for running generative AI models on devices like robots and drones, and outpacing NVIDIA on ML Perf benchmarks.

Last but not least, India’s Ola Krutrim, is also looking to launch the country’s first domestically designed AI chip by 2026, leveraging ARM architecture for competitive performance and efficiency. 

The initiative, led by CEO Bhavish Aggarwal, underscores India’s push towards self-reliance in AI technology, with plans for multiple AI chips, including the Bodhi-1 for large language models and Ojas for edge AI in electric vehicles. Manufacturing partnerships are being explored with top global foundries like TSMC or Samsung to bring this ambitious project to fruition.

These competitors are not just targeting niche segments but are directly challenging NVIDIA’s stronghold in various AI and computing markets, although they still face the significant challenge of overcoming NVIDIA’s software ecosystem, particularly CUDA, which remains deeply entrenched in the industry.

Enjoy the full story here. 

OpenAI ‘Orion’ Clash

Moving over NVIDIA, OpenAI also faces stiff competition as it develops its next-gen ‘Orion’ model, aiming to surpass current AI advancements in problem-solving. 

Interestingly, Google has countered with its new Gemini models, including the Gemini 1.5 Pro, showing strong gains in complex tasks like coding and maths, while Anthropic is expanding its Claude platform with the launch of Artifacts, offering customisable AI-driven applications to a broader user base.

Read the full story here. 

AI Bytes >> 

• California’s legislature passed the contentious AI Safety Bill SB 1047, mandating rigorous safety testing and compliance measures for advanced AI models. This has sparked concerns among tech companies about stifling innovation, while supporters argue it is essential for public safety. 

• Infosys has joined Stanford University’s Institute for Human-Centered Artificial Intelligence (Stanford HAI) as a new affiliate, collaborating on AI research focused on responsible and scalable enterprise AI solutions. This also enhances its own AI offerings through the integration of cutting-edge research into Infosys Topaz.
• Google has unveiled GameNGen, a groundbreaking neural model capable of real-time interactions in complex gaming environments like DOOM, marking a shift towards AI-driven game development where games are represented as neural models rather than code.