Snowcell’s Post

🌟 Significantly boost the performance of your #AI workloads on GPUs by using llama.cpp on RTX AI PCs. ➡️ https://nvda.ws/406X6zp 🦙 With llama.cpp, you gain access to a C++ implementation designed for LLM inferencing, packaged in a lightweight installation. 🔎 Explore and begin utilizing llama.cpp through the RTX AI Toolkit. 🛠️

🌟WITH llama.cpp [通过 llama.cpp] Significantly boost the performance of your hashtag#AI workloads on GPUs by using llama.cpp on RTX AI PCs. 通过在 RTX AI PC 上使用 llama.cpp 显著提升 GPU 上标签#AI 工作负载的性能。 ➡️ https://nvda.ws/406X6zp 🦙 🌟With llama.cpp, you gain access to a C++ implementation designed for LLM inferencing, packaged in a lightweight installation.     通过 llama.cpp，您可以取用专为 LLM 推理设计的 C++ 执行，并包装在轻量级安装中。   🔎 Explore and begin utilizing llama.cpp through the RTX AI Toolkit. 通过 RTX AI 工具包探索并开始使用 llama.cpp。🛠️  

🚀 Quantized attention? SageAttention 2.0.0 beta is now available! This release brings enhanced speed and accuracy for inference, supporting INT8 and FP8 quantization, offering significant speedups without compromising performance across various models. The beta version introduces support for different sequence lengths and group-query attention, optimized for GPUs like RTX4090, A100, and more. Users can expect improved performance with head_dim values of 64, 96, and 128. Amazing work! 🔗 Repo: https://lnkd.in/eHgQ7b_2 ⤵ Helpful? Follow me and join ⚡️ AI Pulse (https://lnkd.in/eWudwDsd) for daily, curated, bite-sized updates on AI—focused on what truly matters to keep you ahead of the curve 🔥

Today we announce LTX Video (LTXV), our new open-source video generation model that achieves what many thought impossible: generating videos faster than real-time playback. The technical breakthrough? We've developed a video encoding model that compresses at a 1:192 ratio while significantly improving motion consistency. Some key technical achievements we’re particularly proud of: * 5-second video generation (121 frames @ 768x512) in just 2.5 seconds (20 diffusion steps on H100) * 2B parameter model - proving you don't need massive models for exceptional results * Optimized for both GPUs and TPUs (with PyTorch XLA) * Runs efficiently on prosumer GPUs like RTX 4090 - no need for specialized hardware When we founded Lightricks 12 years ago, our vision was to make creativity accessible to everyone. Today is no different. We built our model to run on accessible hardware because we believe the next breakthroughs in AI video technology should come from everywhere - not just from labs with unlimited computing resources. Can't wait to see what you guys are going to create with it!

🛠️ Exciting Update for NVIDIA Jetson Users! 🌟 We’re thrilled to announce significant updates to our Docker configurations, ensuring seamless support for both JetPack 4.x and 5.x in Ultralytics v8.2.31 now: https://lnkd.in/dXxtVHwW 📊 Key Changes: - Introduced separate Dockerfiles for JetPack 4.x and 5.x. - Renamed Dockerfile for JetPack 5.x to Dockerfile-jetson-jetpack5. - Added a new Dockerfile specifically for JetPack 4.x (Dockerfile-jetson-jetpack4). - Updated docs and examples for both JetPack 4.x and 5.x. 🎯 Purpose & Impact: - Ensures efficient operation on both newer and older NVIDIA Jetson devices by distinguishing between JetPack versions. - Clear setup instructions tailored to different Jetson hardware configurations make deploying Ultralytics easier. - Renaming and re-organizing Dockerfiles enhance maintainability and understanding of different supported setups. These updates bring enhanced compatibility, improved guidance, and better readability to our Docker configurations, making it easier to deploy Ultralytics on NVIDIA Jetson devices. 🚀 #TechUpdates #NVIDIA #Jetson #Docker #OpenSource

NVIDIA AI Yes, there is a new fast tool. Let's try a go/no-go test to see if it works without metadata. Is there any data solution, AI or NOT, that can answer the following questions of business intelligence? "How many entities, in the Ontario province of Canada, have new US patents granted on the nearest Tuesday (Eastern Time), when the USPTO releases the newly granted US patents on a weekly basis?" "How many entities, in the "江蘇" province of China, have new US patents granted on the nearest Tuesday (Eastern Time), when the USPTO releases the newly granted US patents on a weekly basis?" With our intellectual property (IP), a Chinese-English multilingual metadata, we can answer, simply by an ordinary laptop. Do you or any of your contacts need our expertise/IP to do the data analysis by a simple technology? Metadata is an enabler. It is like a treasure map for treasure hunting. Without metadata, like a treasure map, NO data can be found/retrieved, even by the most advanced technologies, like AI, high-end chips, quantum/super computers, etc. https://lnkd.in/g-aJFn

Good stuff NVIDIA - this is particularly interesting as KV cache offloading is a powerful technique that can significantly enhance the performance, efficiency, and scalability of #RAG models. By addressing the computational and memory challenges associated with large language models and extensive knowledge bases, KV cache offloading enables the development of more sophisticated and powerful RAG systems.

#Aedilic (YC W24) is building #GPUDeploy, a marketplace for renting low-cost on-demand GPUs for machine learning, like Airbnb for GPUs. Addressing the over $1B in yearly unrealized contract value from GPUs sitting around idly, these are the latest #Nvidia and #AMD GPUs in cutting-edge data centers, available for cheap because nobody books them. GPUDeploy makes it easy for research labs, AI companies, and anyone in need of GPUs to book them on-demand or reserve them long-term at wholesale prices that would normally have to be negotiated. Founded by Nicholas Waltz and Lukas Schneider, two engineers with experience in deep learning and robotics, the team understands how expensive and frustrating it can be to launch #GPU instances using existing solutions. GPUDeploy offers competitive pricing and ensures that the compute instances are reliable and meet benchmarks. You can simply connect to the machine to run your workloads. Congrats Nicholas and Lukas on the launch!

Boost your #LLM inference performance with NVIDIA GH200. Learn how this NVIDIA superchip minimizes trade-offs between user interactivity and system throughput, improving TTFT by up to 2x on the Llama 3 70B model. Read here ➡️ https://nvda.ws/4f6pG8I

𝐇𝐨𝐰 𝐦𝐚𝐧𝐲 𝐆𝐏𝐔𝐬 (𝐚𝐧𝐝 𝐰𝐡𝐚𝐭 𝐜𝐨𝐧𝐟𝐢𝐠𝐮𝐫𝐚𝐭𝐢𝐨𝐧𝐬) 𝐝𝐨 𝐲𝐨𝐮 𝐮𝐬𝐞 𝐭𝐨 𝐭𝐫𝐚𝐢𝐧 𝐡𝐢𝐠𝐡-𝐞𝐧𝐝 𝐦𝐨𝐝𝐞𝐥𝐬 𝐥𝐢𝐤𝐞 𝐋𝐋𝐚𝐌𝐀 𝟑.𝟏 𝟕𝟎𝐁? 🚀 Optimizing GPU Configurations for High-End Model Training🚀 When training massive models like LLaMA 3.1 70B, efficiency is key. To keep fine-tuning fast and effective, the NVIDIA GeForce RTX 4090 is a solid choice. For Enterprise problem solving this powerhouse delivers excellent performance at a cost of around ₹155,000 INR ($1,599–$1,799 USD). In past three years we have seen a staggering 10,000x increase in model sizes. Thankfully, PyTorch’s Fully Sharded Data Parallel (FSDP) has evolved from a prototype to a fully integrated solution with Hugging Face, making multi-GPU fine-tuning more accessible. Here’s how you can get started: 1. Configure your setup by defining the number of processes with `num_processes`. 2. Use the `Accelerator()` API for fine-tuning with FSDP and QLoRA. 3. Set up QLoRA by defining a `BitsAndBytesConfig`, and enable the `bnb_4bit_quant_storage` parameter for efficient weight storage. 4. Finally, launch your fine-tuning process using Accelerate. With FSDP and QLoRA, fine-tuning on multiple GPUs has never been faster or more efficient. #AI #MachineLearning #DeepLearning #GPUs #FSDP #QLoRA #ModelTraining #HuggingFace