Quiq’s Post

Take #LLM powered #AIAgents from basic chatbots to truly useful and "get stuff done" bots, with a mastery of Function Calling. Prompt Writing, Prompt Chaining and Function Calling are just 3 absolute must have future skills you can learn with Quiq, in our amazing #AIStudio platform. Register for FREE access to check it out yourself. LINK in the first comment below

When people talk about the shortcomings of Retrieval Augmented Generation (RAG) systems and how to address them, the conversation often revolves around improving retrieval and document chunking. These are, without a doubt, critical areas to focus on. However, another important aspect, generation, can also be the source of unexpected (and sometimes hilarious!) inaccuracies. It's tempting to assume that with highly capable models like GPT-4o, once the correct information is retrieved and passed to the LLM, the model will handle the response accurately. However, this assumption doesn't always hold, especially when LLMs have to keep multiple things "in mind" before answering. A striking example of this is when I tested a RAG setup on financial documents. I asked about interest rates for an amount of Rs. 1 crore, where eligibility was strictly limited to amounts between Rs. 2 crore and Rs. 10 crore. Both GPT-4o and Gemini-1.5-Pro routinely considered the 1 crore amount to be between the 2 crore-10 crore range and hence gave incorrect responses (demonstrated in the attached video). Errors like these can undermine the utility and credibility of a RAG application. One way to mitigate such errors is through prompt engineering. Encouraging the model to think step by step or be mindful of specific constraints (like numeric ranges) can help. But this approach has its limits: 1. It doesn't guarantee correctness every time. 2. It's not possible to account for every type of potential mistake in advance. Even more challenging is the fact that LLM responses aren't deterministic, meaning these errors occur inconsistently, making them harder to detect and address. The other approach to reducing such errors in a RAG application is using better LLMs. This is where the new reasoning-optimized models, like the o1 series, are standing out. Because they initially spend some tokens internally thinking through the user question and information at hand, mistakes like the ones we see with GPT-4o are rarer. While testing the same RAG setup, I found that o1-mini consistently considered the eligibility criteria correctly and delivered the right answer. It did take a few extra seconds for reasoning, but that is an acceptable trade-off for many applications. That said, the o1 models aren't perfect either, and may not be able to handle more complex scenarios. Overall, this serves as an important reminder: even as LLMs become increasingly powerful, their correctness shouldn't be taken for granted. While we're making exciting progress, there's still a gap to bridge before these systems can be considered fully reliable. (PS - Google seems to be sitting on a powerful LLM right now, accessible via their AI Studio, called Gemini-exp-1121. It was consistently accurate in my testing and currently sits at the top of the LLM leaderboard on LMArena) #ai #llms #rag #gpt #gemini

# RAG is the concept, not a product I thank Sherman for showcasing the work of our ML team at Nyota AI: https://lnkd.in/epX965Ch His illustrative diagram captures well what RAG is all about. It is now up to me to build upon his introduction and delve a bit deeper under the hood. The "RAG" label hides a large variety of algorithmic approaches and technologies, primarily sharing in-context learning idea. The "A" in the RAG acronym stands for Augmentation -- the input of the Large Language Model (LLM) is augmented, in contrast with the "simple" LLM invocation where everything the model "knows" comes from pretraining. In other words, the input of LLM in the RAG case contains not only the user's question but also another useful information and we assume its work is easier. That's why RAG offers important promises: 1. Personalization/fine-tuning alternative: Unlike foundation models trained by providers on large volumes of publicly available and synthetic data (plus RLHF), RAG allows the model to directly use your local data, without the need for extensive data preparation and expensive and lengthy model fine-tuning. 2. Reduction of hallucinations - by finding facts and answers in your data, the model does not have to invent them uninformed. The above implies limitations of this approach: - Retriever must obtain relevant context and preferably not present irrelevant information (focus, noise reduction). - LLM must be able to process a larger amount of presented pieces of information using prompts and generate a quality response. Both represent significant challenges -- RAG is only as good as its critical components. It is not difficult to realize that the overall performance is largely influenced by the application's needs. The quantity, nature, and structure of stored documents, the way they are preprocessed, the models used, and prompt engineering -- all of this means that there is no one-size-fits-all solution. Instead, attention must be paid to details, as they will determine how well the system will work.

I wrote a short article on how you learn from doing, the benefits of working within constraints, and using different artificial intelligence APIs and wouldn't it be great if some orchestrator would just let me have it all.

It seems like nowadays, no product can afford to be without AI/LLM-based capabilities, whether those capabilities bring actual value to their users or are used purely from a marketing perspective. So, we're experiencing an LLM-boom. Everyone seems to be RAG-ing, fine-tuning, or even training models from scratch. This reality requires software architects and engineers, who may not have years of in-depth knowledge in the AI/ML field, to design, develop, and deliver production-ready LLM (or AI, if you will) capabilities. So, I thought I'd start a "for dummies" (from a dummy :-)) series of entries that cover just enough knowledge to deliver these capabilities in a timely manner, but not so in-depth as to include knowledge you will not necessarily use. First 2 kick-off entries: LLM Fine Tuning for Dummies: https://lnkd.in/dDmxnGBN Quantization and Post-Quantization Performance Assessment for Dummies: https://lnkd.in/djj_kHPU

#AI #ML #Tech LLM Agents Demystified: Hands-on implementation with LightRAG library Image source, credits o Growtika LightRAG library: https://lnkd.in/grJ67Ejp Colab notebook “An autonomous agent is a system situated within and a part of an environment that senses that environment and acts on it, over time, in pursuit of its own agenda and so as to effect what it senses in the future.” — Franklin and Graesser (1997) Alongside the well-known RAGs, agents [1] are another popular family of LLM applications. What makes agents stand out is their ability to reason, plan, and act via accessible tools. When it comes to implementation, LightRAG has simplified it down to a generator that can use tools, taking multiple steps (sequential or parallel) to complete a user query. What is ReAct Agent? We will first introduce ReAct [2], a general paradigm for building agents with a sequential of interleaving thought, action, and observation steps. * Thought: The reasoning behind taking an action. * Action: The action to take from a predefined set of actions. In particular, these are the tools/functional tools we have introduced in tools. * Observation: The simplest scenario is the execution result of the action in string format. To be more robust, this can be defined in any way that provides the right amount of execution information for the LLM to plan the next step. Prompt and Data Models DEFAULT_REACT_AGENT_SYSTEM_PROMPT is the default prompt for React agent’s LLM planner. We can categorize the prompt template into four parts: * Task description This part is the overall role setup and task description for the agent.task_desc = r"""You are a helpful assistant. Answer the user's query using the tools provided below with minimal steps and maximum accuracy. Each step you will read the previous Thought, Action, and Observation(execution result of the action) and then provide the next Thought and Action.""" 2. Tools, output format, and example This part of the template is exactly the same as how we were calling functions in the tools. The output_format_str is generated by FunctionExpression via JsonOutputParser. It includes the actual output format and examples of a list of FunctionExpression instances. We use thought and action fields of the FunctionExpression as the agent’s response.tools = r"""{% if tools %} {% for tool in tools %} {{ loop.index }}. {{tool}} ------------------------ {% endfor %} {% endif %} {{output_format_str}}""" 3. Task specification to teach the planner how to “think”. We provide more detailed instruction to ensure the agent will always end with ‘finish’ action to complete the task. Additionally, we teach it how to handle simple queries and complex queries. * For simple queries, we instruct the agent to finish with as few steps as possible. * For complex queries, we teach the agent a… #MachineLearning #ArtificialIntelligence #DataScience

What does Grok think of GPT 4o? The response to GPT-4o seems to vary among users, with some expressing positive impressions and others noting issues or limitations. Here's a summary of the feedback from the X posts: 1. @StefanJuang mentions GPT-4o being fast and free to use, but there's speculation that it might be a smaller model trained on synthetic data by GPT-4, which could affect its performance. 2. @krishnanrohit highlights GPT-4o's speed, noting it's less lazy but also prone to making things up. 3. @maceskridge acknowledges GPT-4o's improvements but points out that it still has issues with hallucination and sophistry, which can be problematic for deployment. 4. @realahart finds no significant change between GPT-4o and GPT-4, suggesting that the improvements may not be immediately apparent. 5. @Barnacules appreciates GPT-4o's lower API cost and faster response speed but notes that it still doesn't provide as good coding answers as expected. 6. @nielsgeo expresses disappointment, feeling that GPT-4o is less intelligent than GPT-4. 7. @mawntee clarifies that GPT-4o is a newer version of GPT-4-Turbo, designed for quicker and more natural responses, but standard issue GPT-4 might still be better for tasks requiring deeper comprehension. Overall, the community seems to have mixed feelings about GPT-4o, with some appreciating its speed and cost-effectiveness, while others are concerned about its accuracy and depth of understanding compared to its predecessor.

In the age of ever more capable and knowledgeable LLMs, how best to leverage proprietary data? RAG (retrieval-augmented generation) has become the go-to strategy and many AI products today use a RAG pipeline under the hood. There are many levers you can pull to augment a vanilla RAG pipeline - e.g. tweaking hyperparameters like size of chunks or chunk overlap, trying out different post-retrieval techniques like re-ranking or even agentic systems. But there is also fine-tuning. With OpenAI's recent fine-tuning capabilities it's easier than ever to try it out on your proprietary data and quickly see if it leads to an improvement. If it is, and you don't mind the extra effort/cost, you can always try out open-source LLMs afterwards. In this blog post we explore the following: can we improve the performance of our RAG pipeline by fine-tuning an LLM and leaving all other components untouched? Spoiler alert: we can. For more details read our latest blog post here: https://lnkd.in/eRedCcVX

🚀 Exciting News in AI! 🚀 Dive into the world of LongRAG, a cutting-edge project designed to revolutionize how we handle long documents with Retrieval-Augmented Generation (RAG). Developed by QingFei1, LongRAG is set to transform the landscape of document processing and information retrieval. 🔗 Check out the project here: https://lnkd.in/dDSBvYi5 Why LongRAG? 📄✨ - Efficiently processes long documents - Enhances information retrieval accuracy - Integrates seamlessly with existing workflows Stay ahead of the curve and explore how LongRAG can benefit your business or research. Let's embrace the future of AI together! 🌐💡 #AI #MachineLearning #Innovation #DocumentProcessing #InformationRetrieval #TechTrends

The data flow of LLM applications in one diagram: Prompts can be roughly divided into two parts: - System prompt, such as <task_desc>, <tools>, <examples>, <chat_history>, <context>, <agent_step_history>, <output_format>. - User part, such as <user_query>. LightRAG encapsulates the first part using <SYS></SYS>. The placement of these elements can be flexible. Leveraging the model’s internal knowledge: If you only ask a question, it is a simple QA using the model’s internal knowledge. To better distill the knowledge, you can add <task_desc>, known as zero-shot in-context learning (ICL), or add few-shot to many-shot <examples> (few-shot and many-shot ICL). Besides the internal knowledge, LLMs have four major ways to interact with the world: (1) taking external context such as <context> retrieved from a retriever (RAG) and <chat_history> to enable memory (MemGPT), (2) using predefined tools/function calls, (3) code generation with output from a code executor, and (4) working with agents to use tools/code generation either in series or in a DAG with parallel processing. The diagram we provide allows you to visualize all these parts clearly. 👉 Links in comments! #lightrag #llms #ai #ml ________________________ LightRAG : The Lightning Library for LLM Applications. It is light and modular, with a 100% readable codebase. Follow + hit 🔔 to stay updated.