Velocity’s Post

Transforming ideas into impact! University of Waterloo’s Velocity GPU server is powering student innovation, enabling startups like Wave and AutomaxAI to tackle challenges in epilepsy diagnosis and property appraisal with cutting-edge AI solutions. By removing cost barriers, founders like Dhriti Gabani, Suhani Trivedi and Humza Ahmed are driving breakthroughs in data processing and computer vision. Learn more here: https://bit.ly/4eTp2ue University of Waterloo Faculty of Engineering #Velocity #UWaterloo #AI

🚀 Training large language models (LLMs) like Llama is a huge task that needs a lot of computing power. On my YouTube channel, Murat Karakaya Akademi, I often get questions like, "Can we train our own LLM model from scratch?" The truth is, training LLMs needs a massive amount of GPU resources. For example, Meta used 2048 NVIDIA A100 GPUs to train the first Llama model. They processed 380 tokens per second on each GPU, working with 1.4 trillion tokens. This took about 21 days and over a million GPU hours in total, over 5 months. For the more advanced Llama 3 models, Meta used over 16,000 H100 GPUs. They spent an incredible 39.3 million GPU hours to train on more than 15 trillion tokens. This shows just how many GPUs are needed to make top AI models. 💻 Getting this many GPUs isn't easy. High-end GPUs are more available than last year, but they are still hard for startups to get. To help with this, several groups have stepped in: 🔹 Andreessen Horowitz (A16Z) is getting over 20,000 GPUs to help startups they fund. 🔹 Ex-GitHub CEO Nat Friedman and Daniel Gross have created the Andromeda Cluster with over 4,000 GPUs, including 2,500+ H100s, and offer them at lower rates to their portfolio companies. 🔹 Index Ventures made a deal with Oracle to use their GPUs and give them to their portfolio companies for free. 🔹 Microsoft provides free GPU access through its Azure cloud service to startups funded by M12 and Y Combinator. These efforts show how important it is to share resources to advance AI. The number of GPUs needed for training LLMs is huge, and these initiatives help startups make progress in AI development. 🌟💻 If you want to learn more about training LLMs and other topics, check out my channel Murat Karakaya Akademi. #AI #LLM #GPU #Innovation #Startups #TechInvestment #MuratKarakayaAkademi

Here's a super interesting case on why the current valuation of GPU manufactures (e.g. Nvidia), LLM model providers (e.g. OpenAI) and startups in that spaces completely lost touch with reality. In short: - training frontier LLM models requires too much capital investments compared to EBIDTA - EBIDTA is hurt by high OPEX (newer GPUs are actually becoming less and less efficient in search for more raw power) - EBIDTA growth is limited by revenue growth and customers willingness to pay for using these LLM models (the cost of problem they are solving with this tech, lack of vendor lock-in and existence of open-source alternatives) I do believe that there is a way forward and LLMs are here to stay, but the current AI bubble will burst in a spectacular manner.

NVIDIA has finalized its acquisition of Israeli AI startup Run:ai, adding another powerful tool to its arsenal of AI computing solutions. Run:ai’s software, designed to optimise the performance of intelligence computing hardware, will now be open-sourced. #AI #AIstartup #Israel #Nvidia #Runai #STARTUP

Thrilled to finally reveal the launch of FlexAI, led by the incredible Brijesh Tripathi and Dali Kilani, which just raised a $30M seed round to make AI compute infrastructure more efficient and accessible to everyone. Our investment thesis: 1. Strong macro-tailwinds - Compute for AI is a mature market with sustainable demand growth from new models over the next years - But the compute demand is already exceeding data center capacities and supply of GPUs, creating a supply constraint for Nvidia’s top units - AI model development is attracting large amounts of funding, much of which will be dedicated to computation and costs/performance will matter more and more. 2. The need for a European AI infrastructure - Compute power is high on the geopolitical agenda. Who will control AI will control the world. - European AI model companies rely today on US-based hardware and US-based cloud providers to build "European" AI models - If Data sovereignty and security matter, then there is an urgent need for a European AI compute platform 3. An exceptionally skilled team to build this European AI compute platform - Brijesh has built customized systems and hardware for AI use cases at NVIDIA and Apple before, and designed + scaled compute systems for autonomous driving at Tesla and Zoox, as well as with the Intel super-compute platform - Dali has a demonstrated track record of executing and delivering both hardware and software projects, and leading engineering teams (this is the second time we back him, not a coincidence!) - both have known each other for a long time and worked together at NVIDIA. have built GPUs themselves and were there when Apple and Tesla decided their launch their own chips for the exact same reasons (cost and performance trade-off + reduce dependency on third party suppliers). - They have already demonstrated the strength of their network by signing key partnerships even before product launch and have assembled Tier 1 advisors around FlexAI. Thrilled to be partnering with Alpha Intelligence Capital, Heartcore Capital , Elaia , Frst and Motier Ventures to build the AI compute infrastructure the world needs, made in Paris (with love). https://lnkd.in/e3aXXZSB cc Partech

🌱💡 Is Monza, from Efficient Computer, the breakthrough moment for #SustainableAI? What's the market for deploying #AI algorithms on #edgecomputing versus #datacenters? 🔋 Chip efficiencies are crucial for energy and water-efficient Data Centers powering #AI models. China's projected data center water consumption surpasses South Korea by 2030. Last year, training GPT-3 alone consumed 185,000 gallons of water for Microsoft and 5.6 billion gallons for Google. 💡 Data Centers are energy guzzlers, making #SMR a necessity not a desire. Ask Meta or OpenAI about it. Can #AI #ML integration into edge computing solutions offload those from Data Centers? Case in point: Efficient Computer's Fabric architecture offers a #reconfigurable #dataflowprocessor for optimized code execution and reducing power consumption. #EfficientComputing #GreenTech #TechInnovation. 🚀 Exciting times ahead on how this impacts India's market, especially for software developers and in #Manufacturing and #Industry4.0 applications. These are deja-vu moments for #IndiaTech as it happened in case of #RPA and #IntelligentAutomation where it was both the market and the global innovation center. 🔑 For the milieu of players in similar orbit - which are still considering India, VIA Technologies, Inc. Imagination Technologies The key to win the market or development center is "to" and "through" the #Systemintegrators which also play big domestically.

The early-stage venture capitalist’s job is to harness extreme uncertainty, and to create a very large output from a small input. - Paraphrasing Keval Desai

Considering the current difficulty in buying / accessing reasonably priced AI focused GPUs this is particularly timely and welcome. Do you know anyone who could benefit from this programme? #AI #Startups #Innovation https://lnkd.in/ePJFBHDV

𝗘𝘃𝗼𝗹𝘂𝘁𝗶𝗼𝗻𝗮𝗿𝘆𝗦𝗰𝗮𝗹𝗲 𝗥𝗮𝗶𝘀𝗲𝘀 £𝟭𝟭𝟮.𝟭𝗠 𝘁𝗼 𝗥𝗲𝘃𝗼𝗹𝘂𝘁𝗶𝗼𝗻𝗶𝘀𝗲 𝗕𝗶𝗼𝗹𝗼𝗴𝘆 𝘄𝗶𝘁𝗵 𝗔𝗜 🧬 AI start-up, EvolutionaryScale, has announced a significant milestone with the successful raising of £112.1 million in seed #funding. This funding round, spearheaded by Nat Friedman, Daniel Gross, and Lux Capital, also saw backing from Amazon Web Services (AWS) and NVIDIA’s venture capital arm. 𝗕𝗿𝗲𝗮𝗸𝘁𝗵𝗿𝗼𝘂𝗴𝗵 𝗶𝗻 𝗣𝗿𝗼𝘁𝗲𝗶𝗻 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 𝘄𝗶𝘁𝗵 𝗔𝗜 💡 EvolutionaryScale unveiled the creation of the first large language model (LLM) specifically designed for generating novel proteins. “ESM3 takes a step toward a future of biology where AI is a tool to engineer from first principles, the way we engineer structures, machines, and #microchips, and write #computer programmes,” said Alex Rives, Chief Scientist at EvolutionaryScale. Read more at https://lnkd.in/equtrHuu #technews #airesearch #BiotechInnovation #ethicalai #artificialintelligence #protein #engineering #llm