Athina AI (YC W23)

We're hiring! We're building an IDE that lets engineering teams develop AI-powered products 10x faster. If you're a highly-driven, talented individual interesting in shaping the future of AI-powered software, then apply to work at Athina. Location: San Francisco

Hacker News has become an invaluable resource for developers exploring the latest in AI Development and Innovation 🧑💻🧠 This week, We’ve curated the top 5 most insightful posts on RAG (Retrieval-Augmented Generation)—highlighting key discussions and practical takeaways. 1️⃣ 𝗧𝗶𝘁𝗹𝗲: RAG Logger: An Open-Source Alternative to LangSmith 𝗨𝗽𝘃𝗼𝘁𝗲𝘀: 95 𝗟𝗶𝗻𝗸: https://lnkd.in/gcbcH98E 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗶𝘁 𝗮𝗯𝗼𝘂𝘁: RAG Logger is a simple, open-source RAG pipeline logging tool with suggested enhancements like visualization, OpenTelemetry support, and replay features. 2️⃣ 𝗧𝗶𝘁𝗹𝗲: Collab Notebook – RAG on Your Unstructured Data 𝗨𝗽𝘃𝗼𝘁𝗲𝘀: 14 𝗟𝗶𝗻𝗸: https://lnkd.in/ghK-EPQ8 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗶𝘁 𝗮𝗯𝗼𝘂𝘁: The post outlines using LangChain and Unstructured IO to address unstructured data challenges in RAG with FAISS, LLMs, and Athina AI evaluation. 3️⃣ 𝗧𝗶𝘁𝗹𝗲: Web RAG to generate perplexity like answers from your docs in browser 𝗨𝗽𝘃𝗼𝘁𝗲𝘀: 5 𝗟𝗶𝗻𝗸: https://lnkd.in/gH7_wj5X 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗶𝘁 𝗮𝗯𝗼𝘂𝘁: The system offers a private, browser-based solution for indexing, searching, and generating responses using GTE-small, SQLite, and WebLLM, with zero API costs 👩💻 4️⃣ 𝗧𝗶𝘁𝗹𝗲: LLM apps, AI Agents, and RAG tutorials with step-by-step instructions 𝗨𝗽𝘃𝗼𝘁𝗲𝘀: 3 𝗟𝗶𝗻𝗸: https://lnkd.in/gTt8qMy8 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗶𝘁 𝗮𝗯𝗼𝘂𝘁: A curated repository of RAG-powered LLM applications, showcasing models from OpenAI, Anthropic, Google, and open-source options like LLaMA. 5️⃣ 𝗧𝗶𝘁𝗹𝗲: GraphRAG SDK 0.4.0: Simplify RAG with Graph Databases 𝗨𝗽𝘃𝗼𝘁𝗲𝘀: 2 𝗟𝗶𝗻𝗸: https://lnkd.in/gDtM5CGA 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗶𝘁 𝗮𝗯𝗼𝘂𝘁: The module simplifies RAG application development with graph databases, multi-LLM support, smarter queries, LiteLLM integration, and cost-effective deployment 🚀

Unable to keep a track of latest LLM Research? 🧠 We made this comprehensive list of Top 10 LLM Papers of the week to help you keep with the advancements. Here’s a list of all the papers we covered: 1️⃣ Proposer-Agent-Evaluator(PAE): Autonomous Skill Discovery For Foundation Model Internet Agents 🧠✨ 2️⃣ MultiCodeBench: How Well Do LLMs Generate Code for Different Application Domains? Benchmark and Evaluation 3️⃣ Precise Length Control in Large Language Models 4️⃣ PROMO: Prompt Tuning for Item Cold-start Recommendation 🤖 5️⃣ Qwen 2.5 Technical Report 📖 6️⃣ AutoFeedback: Using Generative AI and Multi-Agents to Provide Automatic Feedback 🗃 7️⃣ Robustness-aware Automatic Prompt Optimization 8️⃣ DRUID: A Reality Check on Context Utilisation for Retrieval-Augmented Generation 9️⃣ Alignment Faking in Large Language Models 🛠 1️⃣0️⃣ TheAgentCompany: Benchmarking AI for Real-World Tasks 🚀 Curious to delve deeper into their details and understand their influence on our LLM pipelines? Read the full blog from the first comment 👇

🚀 DeepSeek just dropped DeepSeek V3, their latest open-source model, and it's turning heads! 🌟 This powerhouse model has been making waves by outperforming some of the best on standard benchmarks, earning a spot among the top 5 models alongside Qwen 2.5, Llama 3.1, Claude Sonnet, and GPT-4o. ✨ Key Highlights of DeepSeek V3: — 671B MoE parameters with 37B activated at any time 💡 — Input token cost: $0.27/M tokens — Output token cost: $1.1/M tokens — Speed: Processes 60 tokens/sec ⚡ — Training data: A whopping 14.8T tokens 🧠 — ~11x cheaper than OpenAI O1 mini! 💰 API access is live now 👉 https://lnkd.in/d69mycCh Which of these features excites you the most?

Evaluating your LLM's performance is crucial, but knowing when, why, and how to use the right metrics can make all the difference. 🎯 Our team published an in-depth article that covers all the aspects: https://lnkd.in/dQbjD4aU Here’s a quick breakdown: ⚙️ Broad evaluation categories to focus on: ⚡️Text similarity metrics: BLEU, ROUGE, METEOR, Levenshtein distance 📝 ⚡️Semantic similarity metrics: Cosine similarity, MoverScore calculation 🤖 ⚡️LLM as a Judge: Use eval libraries like RAGAS, Open AI evals, Guardrails AI, Protect AI, etc., that utilize LLMs as judges to assess quality 🧐 ⚡️Qualitative metrics: Don’t forget user feedback (score, thumbs up/down) or edit distance of responses (compression-based edit distance) 💬 🔍 When to use LLM evaluation metrics: 1️⃣ During development to identify model strengths and weaknesses 2️⃣ Before deployment to ensure reliability 3️⃣ For continuous improvement as part of your AI lifecycle Help meet ethical and fair AI standards Check out the full article by Haziqa Sajid here 👇

[Colab Notebook]: Improve your LLM output using RAG Fusion If you're building a domain-specific RAG (e.g. medical, financial, legal etc) and struggling to improve its performance, try RAG Fusion. 💡 When should you consider RAG Fusion? 1️⃣ Ambiguous or poorly formulated queries: Users often struggle to articulate their questions, especially when unfamiliar with domain-specific vocabulary. 2️⃣ Large-scale information retrieval: Perfect for applications handling massive datasets or diverse information sources. 3️⃣ Complex queries: When nuanced or intricate user queries demand a broader context to deliver accurate results. 🧠 What is RAG Fusion? RAG Fusion is an advanced technique that builds on the multi-query retriever method. Here's how it works: - Creates multiple variations of the user’s query. - Retrieves results for each query from your vector database. - Applies Reciprocal Rank Fusion to score and re-rank the retrieved documents. - Uses the re-ranked results to generate more accurate responses. Our team has simplified the implementation for you with a ready-to-run Colab notebook: https://lnkd.in/dSAwX8QV ⭐️ If you find this useful, please leave a star!

[Colab Notebook] Build a RAG on Your Unstructured Data 📄➡️💡 Building a RAG application is a powerful way to unlock insights from your data! But when you move to real-world data, things get tricky. 🤔 🔑 Key Challenges: 🏗️ Prototyping RAG with structured data is easy. But what about unstructured data? Pdfs, emails, images, tables, and Excel sheets? 🧩 It is often a pain to make unstructured data LLM-ready. If not handled correctly, you end up with broken tables, poor chunking, and low-quality outputs. 🛠️ To help solve this, our team created a Colab notebook that: Uses unstructured.io to parse and prepare unstructured data for LLMs Integrates LangChain to build the RAG on top of the open-source vector DB, FAISS 🔥 Ready to give it a try? Here's the link to the notebook: https://lnkd.in/dWxfnsQa ⭐️ If you find this useful, please leave a star!

🚀 Build AI Workflows in Minutes with Flows on Athina AI (YC W23) 🌟 We built a Flow that checks the sentiment of a keyword across multiple channels in just a few clicks. 🧩 Check it out here: https://lnkd.in/dzmGYHup Here’s how it works: 1️⃣ Input a keyword (e.g., “AI trends”) 2️⃣ Runs a neural search using Exa to fetch relevant results. 3️⃣ Focuses on results from two channels: News and Twitter. 4️⃣ Uses a custom code block to extract, process, and format data from both channels. 5️⃣ Calls GPT 4-O mini to calculate sentiment scores and renders them in a clean, structured format. 6️⃣ Finally, GPT 4-O mini summarizes the overall sentiment for easy interpretation. This is just the beginning—Flows lets you build and deploy multi-step AI workflows faster than ever. 🌍 What will you build today? 😉

Today's AI Spotlight: Athina AI (YC W23) - Enterprise AI Development This collaborative AI development platform is transforming how teams build, test, and monitor AI features in production environments. What makes Athina stand out: • Complete AI development lifecycle management • Real-time monitoring designed specifically for LLM traces • Seamless collaboration between technical and non-technical teams • Support for custom models and major AI providers • Enterprise-grade security with SOC-2 Type 2 compliance Fresh off raising $3M in new funding, Athina is accelerating AI deployment for enterprise teams with their end-to-end platform. Their solution enables teams to ship AI features up to 10x faster while maintaining robust monitoring and evaluation capabilities. The AI Spotlight is part of the Daily AI Brief newsletter: https://lnkd.in/ehXTGEsu #AIDevTools #MLOps #AIPlatform #EnterpriseAI #DevTools #AIMonitoring #MLEngineering #ArtificialIntelligence #AI

🚀 Improve quality and relevance of your LLM responses with RAG-Fusion (Colab Notebook Included)! 🤖 This technique uses multi-query retrieval and rank fusion methods to improve your RAG. We published an Open Source Colab notebook (using LangChain) about this technique on Github: https://lnkd.in/dSAwX8QV 🔍 Why RAG Fusion? ✅ Better Reranking: Assigns higher scores to the most relevant documents, ensuring quality results. ✅ Smarter Query Understanding: Multiple query variations help capture user intent more accurately. 🎯 How it works: 1️⃣ Generate multiple query variations. 2️⃣ Run vector searches for each variation. 3️⃣ Rank results using Reciprocal Rank Fusion (RRF). 4️⃣ Produce insightful, context-rich responses. Give us a star ⭐️ if you find the repository useful!