Top RAG Papers of the Week (November Week 1, 2024)

[1] RAGViz Tool

This paper presents RAGViz, a RAG diagnosis tool that visualizes the attentiveness of the generated tokens in retrieved documents. RAGViz tool includes a built-in user interface, retrieval index, and Large Language Model (LLM) backbone. RAGViz provides two main functionalities: (1) token and document-level attention visualization, and  (2) generation comparison upon context document addition and removal. RAGViz operates efficiently with a median query time of about 5 seconds on a moderate GPU node. [Tweet] and [Paper]

[2] RAGulator (Irrelevant LLM output Detectors in RAG)

Detecting out-of-context LLM outputs is crucial for enterprises looking to safely adopt RAG applications. This paper introduces RAGulator, a lightweight out-of-context detectors for RAG systems. RAGulator involves training lightweight models to identify LLM-generated text that is semantically out-of-context from retrieved text documents. Results show that DeBERTa is the best-performing model which is also fast and does not require additional text preprocessing or feature engineering. [Tweet] and [Paper]

[3] Long Context RAG

This paper presents a comprehensive study of the impact of increased context length on RAG performance across 20 popular open source and commercial LLMs. Results reveal that while retrieving more documents can improve performance, only a handful of the most recent state of the art LLMs can maintain consistent accuracy at long context above 64k tokens. [Tweet] and [Paper]

[4] Rationale-Guided RAG

This paper rationale-guided RAG  a new framework for enhancing the reliability of RAG in biomedical contexts. RAG2 incorporates three key innovations: a small model trained on perplexity-based labels of rationales to filter out distractors; LLM-generated rationales as queries to improve the utility of retrieved snippets and  a structure designed to retrieve snippets. The proposed RAG approach RAG2 improves the state-of-the-art LLMs of varying sizes, with improvements of up to 6.1%. [Tweet] and [Paper]

"Top LLM Papers of the Week" newsletter is read by over 20k+ AI Researchers, Engineers and Developers. If you would like to promote with us, contact Kalyan KS

[5] Adaptive Filtering for RAG

This paper presents E2E-AFG, an end-to-end model with adaptive filtering for RAG systems. Adaptive filtering enables the model to focus more effectively on relevant content while reducing the influence of irrelevant information and generating accurate answers. The proposed E2E-AFG consistently outperforms baseline models across all tasks. [Tweet] and [Paper]

[6] Data Extraction Attacks in RAG

This paper investigates data extraction attacks targeting the knowledge databases of RAG systems. Specifically, the authors  propose a method to backdoor RAG systems by injecting a small amount of poisoned data into the LLM’s fine-tuning dataset. Results show that with only 3% poisoned data, the proposed method achieves an average success rate of 79.7% which underscores the privacy risks associated with the supply chain when deploying RAG systems. [Tweet] and [Paper]

[7]  CORAG (Retrieval Optimization System for RAG)

Existing RAG approaches select each chunk independently, overlooking potential correlations among them. To address this, the authors introduce CORAG, a cost constrained retrieval optimization system for RAG. CORAG employs a Monte Carlo Tree Search (MCTS) based policy framework to find optimal chunk combinations sequentially, allowing for the consideration of correlations among chunks. [Tweet] and [Paper]

[8] M3DocRAG (Multimodal RAG)

This paper presents M3DocRAG, a novel multi-modal RAG framework that flexibly accommodates various document contexts (closed-domain and open-domain), question hops (single-hop and multi-hop), and evidence modalities (text, chart, figure, etc.). M3DocRAG finds relevant documents and answers questions using a multi-modal retriever and an MLM. Results show that M3DocRAG with ColPali and Qwen2-VL 7B achieves superior performance than many strong baselines. [Tweet] and [Paper]

Do subscribe to the newsletter so that you won't miss interesting updates related to Generative AI, LLMs and RAG.

Kalyan KS, Research Scientist(NLP) at Akmmus AI Labs