Mantis - AI-native platform engineering’s Post

The most popular large language models (LLMs) today can reach tens to hundreds of billions of parameters in size and, depending on the use case, may require ingesting long inputs (or contexts), which can also add expense. NVIDIA blog #llm #education #generativeai #conversationalai #sciense #computing #nvidia #NeMo #transformers #tensorrt #telecomunications #deeplearning #machinelearning #consumerinternet

🚀 Just watched a fascinating video that perfectly illustrates why benchmarking is becoming a critical skill in our AI-driven engineering landscape. It's not just about writing code anymore - it's about validating our assumptions with hard data. 💡 🧪 Seeing an engineer methodically test function call capabilities across 30 function calls reminded me of a crucial truth: our intuition, no matter how experienced we are, needs to be validated through rigorous testing. The results? Let's just say they were quite different from my intuition! 😅 🤔 ⚡ I love how @IndiDevDan captured the essence of engineering mindset: "This is why we test. This is why we dig deeper - we are engineers. We don't follow the hype, we don't take news at face value, we don't take hype at face value, we don't care about others' opinions that isn't backed by stats, data, and raw information. If you want to make data-driven decisions and get the highest ROI, look at the tests, look at the evals, look at the data, and most importantly, run your own benchmarks." 🎯 From my experience leading tech teams, I've seen how great engineers naturally gravitate towards evidence-based decision making. It's not just about following best practices - it's about verifying that these practices actually deliver in your specific context. 🔍 See the full video here: https://lnkd.in/dmNecBqR #SoftwareEngineering #AI #Benchmarking #DataDrivenDecisions #Engineering

🌟 Exploring the World of Large Language Models (LLMs) 🌟 Getting started with LLMs was no easy feat! 🚀 Initially, the huge computational power needed—like GPUs and CPUs—makes it a difficult task for a beginner to run LLMs. But after some exploration, I discovered alternatives like Kaggle's 🌐 GPU computation and successfully imported and ran a vision LLM from Hugging Face to complete my task. 🖼️🤖 During my journey, I came across open-source AI tools like Ollama, which use quantization to shrink LLMs, making them suitable for local machines. 🖥️✨ Exploring Ollama introduced me to fascinating vision LLMs like LLaVA (13B parameters), Baklava, and LLaMA 3.2 (3B parameters). These models bring incredible potential for vision-based AI projects. 👁️📊 I also dived into libraries like LangChain 🧩 and Streamlit 💻, which are game-changers for building RAG (Retrieval-Augmented Generation) applications. These tools simplify complex workflows and make AI development approachable. 💡 One of my achievement was building an LLM-powered 🤖 wrapper that classifies images from webcams during online tests 🎥. It automatically detects potential cheating behavior—a step towards ensuring integrity in remote assessments ✅ The journey has been challenging yet rewarding, filled with learning, exploration, and creativity. ✨ If you're stepping into the world of LLMs, don't hesitate to explore the amazing resources out there—your next breakthrough might be just one model away! 💪 https://lnkd.in/dZg8qRiK #AI #LLM #MachineLearning #VisionModels #LangChain #Streamlit #Innovation #OpenSource

💥💥💥 No Free Lunch in LLM Watermarking: Trade-offs in Watermarking Design Choices Advances in generative models have made it possible for AI-generated text, code, and images to mirror human-generated content in many applications. Watermarking, a technique that embeds information in the output of a model to verify its source, aims to mitigate the misuse of such AI-generated content. Current state-of-the-art watermarking schemes embed watermarks by slightly perturbing probabilities of the LLM’s output tokens, which can be detected via statistical testing during verification. Unfortunately, this work shows that common design choices in LLM watermarking schemes make the resulting systems surprisingly susceptible to watermark removal or spoofing attacks—leading to fundamental trade-offs in robustness, utility, and usability. To navigate these trade-offs, we rigorously study a set of simple yet effective attacks on common watermarking systems and propose guidelines and defenses for LLM watermarking in practice. 👉 https://lnkd.in/dC9MmGpZ #machinelearning

Interesting LLM cost vs. performance showdown by Virat Singh. He recently explored how different LLMs perform on complex tasks like calculating net profit margin, debt-to-assets ratio, and free cash flow from financial statements. It was a practical test to see LLM's speed, accuracy, and cost. I like how he categorized the LLMs into three groups based on their performance: · 𝐓𝐡𝐫𝐨𝐮𝐠𝐡𝐩𝐮𝐭 𝐓𝐢𝐞𝐫: This tier includes models that respond quickly but might not always hit the mark with their answers. Groq was the sole model in this category. · 𝗪𝗼𝗿𝗸𝗵𝗼𝗿𝘀𝗲 𝗧𝗶𝗲𝗿: Models in this tier offer a balance of speed and reliability. Some, like Cohere's models and Haiku, really stood out for their performance. This tier also includes Mixtral 8x7B, Haiku, Command, Gemini, and versions of GPT up to 3.5. · 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 𝗧𝗶𝗲𝗿: These models tend to be slower and more expensive, but they're also more likely to give you the correct answer. It's where you find the more advanced models like Mistral, Command R+, Sonnet, Gemini1.5 Pro, GPT-4, and Opus. Once again, the importance of choosing the right model based on the task at hand, especially considering the trade-offs between speed, cost, and accuracy. #TechExploration #LLMPerformance #AIModels

LLM Evoulutinary tree represented with different Models for information, it is always important to know the model's internals that need to be used to address the use-case. Some choose the Models without any basic analysis if the model fits the use case. Note: The success of the AI-driven applications is based on the Models chosen.

Interesting article on how/if/why to shrink down (quantize) you model parameters when running a llm model locally... https://lnkd.in/dXDpQYVx

Sometimes the most exciting progress hides in plain sight. While GPT-4 and Copilot might seem the same, breakthroughs like 1-bit LLMs promise a future of faster, smarter, and more affordable AI tools. This work benefits us all! #ai #innovation #LLMs

Recently released O1 models are making waves with their impressive performance. The incorporation of chain of thought and STEM reasoning has the potential to lead LLM's to uncharted territory. Check out more details here: https://lnkd.in/gZEKckDQ

1.5 bit models; this paper (if confirmed) is a gamechanger for LLM inference. https://lnkd.in/gKnNMwv8 Big ups for the team putting this together, it seems so simple. Looking forward to reading the reviews and implementations.