d-Matrix

🔴 Welcome to a new world! Today, d-Matrix introduces Corsair, the first-of-its-kind AI compute platform. 60,000 tokens/sec at 1 ms/token latency for Llama3 8B in a single server, 30,000 tokens/sec at 2 ms/token latency for Llama3 70B in a single rack.  Corsair shines with ultra-low latency batched throughput! Ideal for tomorrow’s use cases where models will “think” more. Supercharging reasoning, agents and video generation. Celebrated with a toast at SC'24. Onwards and upwards! #dmatrix #corsair

Recent breakthroughs in AI models have flipped scaling laws on their head as the world moves from training to inference. This shift puts technologies that were purposely built for ‘reasoning’ front and center. These new advancements in AI around inference are challenging the previously established cadence where simply increasing model size and training data would lead to proportional performance improvements. Enter d-Matrix. We are leading the charge around inference as the focus is now shifting towards making models more efficient and effective during actual usage – inference – instead of just training. Our new Corsair architecture prioritizes ultra-low latency batched #inference without sacrificing accuracy. Our innovations for inference-time compute significantly lower the computational cost and power of running #AI models at scale in production environments, making them commercially viable for enterprises and datacenters.  Here is more on how d-Matrix d-livers more 'thinking' for less. https://lnkd.in/gCQWCDCZ

Ready to join the Fastest GenAI #Inference Company! #hiring Chip Designers / Hardware System Engineers / Software Engineers - Compiler / Kernel / Infra / Product Applications / Head of Marketing Communications / many more https://lnkd.in/gZ6fzsbU

d-Matrix's new Corsair, the first-of-its-kind AI compute platform. 60,000 tokens/sec at 1 ms/token latency for Llama3 8B in a single server, 30,000 tokens/sec at 2 ms/token latency for Llama3 70B in a single rack. Corsair shines with ultra-low latency batched throughput! Ideal for models that “think” more. Supercharging reasoning, agents and video generation. Download the technical white paper file: https://lnkd.in/gQ7S9aMA

#Hiring 🚀 Be part of something disruptive, fun, challenging, and game-changing. Apply: https://lnkd.in/g3TzHCut

Exactly how do you architect for intelligence? Satyam Srivastava, Chief AI SW architect at d-Matrix walks through the innovations needed to be able to scale the paradigm shift in as #AI hits enterprises and #inference-time compute is the new math. He steps through how AI chip landscape is approaching the complex challenges and steps through his team's approach. https://lnkd.in/gukeaZf9

″[DeepSeek] has demonstrated that smaller open models can be trained to be as capable or more capable than larger proprietary models and this can be done at a fraction of the cost,” said Sid Sheth, CEO of AI chip start-up d-Matrix. “With the broad availability of small capable models, they have catalyzed the age of inference,” he told CNBC, adding that the company has recently seen a surge in interest from global customers looking to speed up their inference plans. AI Inference is on the agenda - thank you CNBC.

#Hiring at d-Matrix! 🚀 Join Us in Powering the Future of AI Inference Apply: https://lnkd.in/gvSyregS

We made a bet on inference early at d-Matrix and built our technology from the ground up for inference time compute. Here's the backstory at Cerebral Valley with our CEO/Cofounder Sid Sheth. "What we learned was that while training models was the focus at the time, inference would eventually dominate. Customers told us, “Training gets you the intelligent models, but scaling and deploying them in the real world will be the bigger challenge.” And logically, it made sense—models are trained a finite number of times, but they’re used for inference endlessly." Read it here: https://lnkd.in/ginaH-Yg