Comet

🎙️ "99% of the time, when it comes to dev tools, open source wins." - Gideon Mendels Our CEO, Gideon Mendels, recently sat down with Eric Anderson on the Contributor podcast to discuss the journey of building Opik, our open-source framework for LLM evaluations, tracing, and dashboards. 👂 Tune in to learn more about the choice to make Opik open source and how Opik is addressing the challenge of getting GenAI apps into production. 🎧 Listen here: https://lnkd.in/eeK7_jqB And catch a sneak peek below 👇

🤩 Always a fan of tools that try to simplify LLM deployment as a whole (not just parts of it). And this open-source repo does just that ( 🌟 already at 4k stars!). If you’re working in the space, you know how tricky it can be to manage multiple tools and ensure they all work seamlessly together. Comet Opik streamlines the process with: ⛳ Input Handling: Test different prompts and models in the prompt playground. ⛳ Data/Model Layer: Store test cases and run experiments ⛳ Application-Level Monitoring and Annotations: Track LLM calls, traces, and feedback during development and production using code/UI ⛳ Evaluation: Detect hallucinations, use popular RAG metrics, and easily configure custom LLM judges. It’s a solid resource for anyone building and maintaining production-grade applications, definitely worth checking out! Link: https://lnkd.in/gAFmjkK3

🎉 What a way to kick off the new year! Opik has reached 4,000 stars on GitHub! 🌟 As we step into 2025, we’re excited to continue building the go-to open-source framework for LLM evaluations, tracing, and dashboards. Here’s to another year of community-driven success! 🍻 Follow along on GitHub 🔗 https://lnkd.in/dW4D6xMt

"Without integrating Comet into our ML development process, we would have faced significant productivity challenges due to the increased complexity in managing and tracking models, reporting, and planning." – Yoko Inaba, Head of Innovation Technology at NTT DATA. We're proud to be a trusted partner in NTT DATA's machine learning journey. 🔗 See how Comet is supporting their growth: https://lnkd.in/dAyqQuJq

 🚀 Haystack by deepset is a powerful open-source orchestration framework for building production-ready LLM apps. With the Opik + Haystack integration, all your Haystack defined chains & agents are seamlessly logged to Opik, giving you complete visibility into your LLMs performance from retrieval to generation. 🔎 Explore the integration here:  https://lnkd.in/dqyWebXC #RAG #GenerativeAI

💡 BERTScore was among the first widely adopted evaluation metrics to incorporate large language models. 📐 It operates by using a transformer-based model to generate contextual embeddings and then compares them a simple heuristic metric— cosine similarity ⚖️ Finally, it aggregates these scores for a sentence-level similarity score. 👉 Learn more about BERTScore, including how to code it from scratch in #Python in this new article from Comet's own Abby Morgan: https://lnkd.in/eM3XMY8i #OpenSource #AI #GenerativeAI #Opik

Thank you to our incredible community for helping Opik cross 3,000 stars on GitHub! 🙌🌟 Since launching in September, Opik has redefined how teams track, evaluate, and test their LLM apps – from RAG chatbots & code assistants to agentic systems. This milestone is just the beginning. 🚀 Join the journey: https://lnkd.in/dW4D6xMt #OpenSource #LLMEvals

🤩 Opik is trending #2 on GitHub! If you're working with LLMs and looking for a tool to simplify evaluation, testing, and monitoring, Opik is gaining traction for a reason. Join the thousands of developers already using this open-source framework to build better, more reliable LLM applications. ⭐ Check it out here and give it a star if you like what you see: https://lnkd.in/dW4D6xMt

Simplifying LLM evaluations just got easier with our partnership with IBM #watsonx. If you're using watsonx LLMs you can now easily run evaluations with Opik.

🥂 Let’s raise a glass! We got together with Intel Corporation, Hugging Face, Voxel51 — and nearly 1500 #NeurIPS attendees — last night to celebrate the latest in #AI and #ML innovation. Cheers to the people behind so many incredible #AI projects we heard about in Vancouver! #NeurIPS2024