Softlandia Ltd.’s Post

The current state of the RAG ecosystem and my bold predictions for 2025. 1. Vanilla RAG aka naive RAG The "original" RAG architecture gained popularity in 2022. It indexes static chunks of text data into a vector database, and the top n results from a text similarity search are injected into the final prompt. 2. Advanced RAG Better indexing and search strategies emerged in 2023 and the Advanced RAG was born. Techniques like semantic chunking and dynamic chunking replaced static chunking and search accuracy was further improved via reranking. 3. Advanced Agentic RAG In 2024, agents are everywhere and you can't build a RAG without them. Agentic patterns enabled RAGs to access services, databases, and APIs. Agents select the retrieval sources and combine the results into the final answer. 4. Advanced Agentic Vision RAG In the second half of 2024, vision-based retrieval with specialized models such as ColPali emerged. Multimodal vision RAGs understand text, images, tables, graphs, and other visual elements and could replace chunking completely in most RAG implementations. 2025 will be the year of multimodality and vision models.

Excellent summary of where RAG development is going now and where its heading. Text chunking was becoming unnecessarily complex anyways, so here applies the metaphor "a picture tells more than a thousand word" perfectly. Vision RAGs are a great upgrade. Good stuff and thnx for this great summary, Olli-Pekka Heinisuo !

This is one of the best papers on retrieval augmented generation (RAG) I've seen recently. It is a thorough survey of RAG techniques, frameworks, and related research, broken out into several dimensions that would be useful if you are thinking about your own architecture, as there are many ways to incorporate RAG. I especially also like how they look at major techniques not just by the foundation (shown below), but also by enhancement, such as: - Input: Queries and data augmentation - Retriever: Fine-tune retrievers (RAG + Finetuning is a great combination), metadata filtering (e.g. Pinecone), etc. - Generator: Prompt engineering - Result: Output rewrites - RAG pipeline: Adaptive retrieval and iterative retrieval If you want a comprehensive understanding of all the developments in retrieval augmented generation to date, this is a great paper to review. 🔗 Credit and paper: https://lnkd.in/deW3d_vy #retrievalaugmentedgeneration #artificialintelligence #machinelearning #genai

These are the four distinct approaches to #RAG systems. Naive RAG represents the most basic implementation, following a straightforward pipeline of document collection, basic chunking, dense embedding, and vector search. Advanced RAG builds upon this foundation with more sophisticated features, incorporating smart chunking, hybrid embedding, re-ranking, and query expansion to improve retrieval accuracy. Multi-modal RAG extends the capability to handle diverse media types through mixed media collection, multi-modal processing, cross-modal embedding, multi-modal fusion, and modal-specific ranking, enabling comprehensive understanding across different content formats. Graph RAG takes a relationship-centric approach, utilizing entity extraction, graph construction, graph embedding, and graph traversal to leverage interconnected knowledge structures. Each architecture is distinguished by its key features: Naive RAG offers simplicity and basic vector similarity search; Advanced RAG provides semantic chunking and enhanced relevance scoring; Multi-modal RAG excels in handling multiple data types and cross-modal understanding; and Graph RAG specializes in relationship modeling and complex query reasoning through knowledge graph integration. The below image demonstrates the progressive evolution from basic to more sophisticated RAG implementations, each serving different use cases and complexity requirements.

I've been working with #erkg a long time... before it had a name (Entity-Resolved Knowledge Graphs). Fell in love with the visual and analytical power of graph-based analytics and #sna in graduate school (the first time). Built several #graphs as part of my dissertation in graduate school (the second time). Then built several graphs for my professional community, all the while wondering why #knowledgegraphs weren't more of a thing. Well, the tech is finally catching up, for the purposes of scale, for the purposes of implementation. Software companies are providing very necessary pieces of graph-based data architectures. And LLMs just made the world of search and the world of graph a whole lot spicier. #graphRAG + #LLM = superior search, retrieval, and interactive analysis. "a graph provides a structured representation of data and enables reasoning on the whole dataset." "Building a comprehensive and accurate knowledge graph requires deep domain understanding and expertise in graph modeling, which is complex and resource-intensive." "We think that GraphRAG is here to stay, and has emerged as a natural extension of RAG." https://lnkd.in/eFYNAkv4

Nearly 70% of vector databases uses HNSW (Hierarchical Navigable Small Worlds) or its variant as an algorithm to organize the data in the vector index. HNSW is a highly efficient and scalable algorithm for approximate nearest-neighbor search (ANN). #ANN algorithms trade a small loss in accuracy for a significant improvement in query performance and ideal for large-scale vector search scenarios where brute-force methods (KNN) are computationally expensive. Based on the "small world" phenomenon where most nodes are connected by a few intermediate connections. HNSW structures data into layers of graphs, where nodes in higher layers represent broader connections, and lower layers provide finer-grained details. During graph construction process, nodes are added incrementally, connecting to their nearest neighbors. Higher layers contain fewer nodes with broader connections; lower layers contain all nodes with detailed connections. This hierarchical approach, coupled with the small-world concept, optimizes the path to high-precision results, essential for applications demanding quick and accurate retrievals. - Faster query times and scalability for millions of vectors, significantly outperforming brute-force approaches (KNN). - May return an approximate match instead of the exact nearest neighbor but optimizes query performance. - Query time increases logarithmically instead of linear growth with the number of data points. #ArtificialIntelligence #VectorDB #Scalability #Innovation More details on this graph based algorithm (HNSW) - https://lnkd.in/gqF5j5Sd

Interesting article about Graph-RAG by Yitae Jeong - 🧠 GraphRAG enhances RAG by integrating knowledge graph techniques for better information retrieval and reasoning, aiming to address RAG's limitations. - 🔄 The RAG process includes Pre-Retrieval, Chunking, Retrieval, and Post-Retrieval stages to ensure detailed and relevant information delivery. - ⚖️ GraphRAG potentially offers a solution to some of RAG’s drawbacks through semantic indexing and enhanced context management. - 🧐 Deciding on data granularity (the level of detail in data used for retrieval) is crucial for the effectiveness of RAG, balancing detailed response capability against information overload. - 🌐 GraphRAG uses a combination of semantic search and similarity search to refine the accuracy and relevance of information retrieved, incorporating advanced graph-based techniques. - 🔄 GraphRAG's architecture includes Query Rewriting and both Pre-Retrieval and Post-Retrieval processes to optimize the information retrieved and its integration into user responses. - 🤖 Despite its potential, GraphRAG, like RAG, faces limitations in query generation, reasoning boundaries, and information extraction, which must be managed to avoid overwhelming the system. https://lnkd.in/ewWYYeWb

Meta has just published a paper introducing the Byte Latent Transformer (BLT), a new type of large language model (LLM) architecture. What makes BLT different from traditional LLMs is that it reads text in "patches" of bytes instead of "tokens". This allows it to understand text more efficiently and accurately. The authors show that BLT achieves the same performance as traditional LLMs at scale, but with significant improvements in: * Inference efficiency: BLT can process text faster, leading to quicker response times. * Robustness: BLT is better at handling noisy data and has a deeper understanding of the text. * Scalability: BLT can handle larger amounts of data, making it more versatile. BLT also offers potential cost benefits due to its efficient use of compute resources. This could translate to lower energy consumption and reduced hardware requirements. BLT's superior performance with long input sequences and its ability to handle noisy data could be particularly beneficial for processing extensive medical records or analyzing unstructured clinical notes. This could lead to more efficient and accurate diagnostic tools or predictive models for patient outcomes. #ByteLevelLLM #DynamicPatching #NLProc #MachineLearning #AIResearch #TechAdvancementa https://lnkd.in/g93ezc9N

Stijn Verdenius, Andrea Zerio, Roy L.M. Wang introduce LatentTimePFN (LaT-PFN), a foundational Time Series model with a strong embedding space that enables zero-shot forecasting. To achieve this, they perform in-context learning in latent space utilizing a novel integration of the Prior-data Fitted Networks (PFN) and Joint Embedding Predictive Architecture (JEPA) frameworks. They leverage the JEPA framework to create a prediction-optimized latent representation of the underlying stochastic process that generates time series and combines it with contextual learning, using a PFN. Furthermore, they improve on preceding works by utilizing related time series as a context and introducing an abstract time axis. This drastically reduces training time and increases the versatility of the model by allowing any time granularity and forecast horizon. They show that this results in superior zero-shot predictions compared to established baselines. We also demonstrate our latent space produces informative embeddings of both individual time steps and fixed-length summaries of entire series. Finally, they observe the emergence of multi-step patch embeddings without explicit training, suggesting the model actively learns discrete tokens that encode local structures in the data, analogous to vision transformers. Article links: https://lnkd.in/dn84Z6JH

If you are in search, you would have to hide under a rock, and still, you would hear someone say, "Why not use ColBERT?" Simple answer? Token-Level Embeddings! ColBERT, or Contextualized Late Interaction BERT, is a game-changing approach to information retrieval! Here's why: 1. Token-Level Precision: Unlike single-vector models, ColBERT generates multiple embeddings for each token in queries and documents. This granular approach captures nuanced meanings and contextual relationships with unprecedented accuracy. 2. Late Interaction: ColBERT's secret sauce lies in its innovative late interaction mechanism. By delaying the matching process between query and document embeddings, it preserves fine-grained semantic information that would otherwise be lost in early aggregation . 3. BERT-Powered Understanding: Built on the foundation of BERT, ColBERT leverages deep bidirectional context to generate rich, contextualized embeddings for every token . How of Why? - Architecture: ColBERT employs separate query and document encoders, both utilizing BERT's power. A CNN layer and normalization step follow, producing sets of embedding vectors (E_q for queries, E_d for documents). - Query Augmentation: Queries are padded with [MASK] tokens to a fixed length, ensuring uniform processing. Special [Q] and [D] tokens explicitly mark query and document starts. - Efficient Indexing: ColBERT uses vector databases like FAISS for lightning-fast retrieval. Offline indexing and flexible embedding storage (32-bit or 16-bit) optimize for both speed and space. - ColBERTv2 Improvements: The latest iteration introduces residual compression, reducing storage requirements by 6-10x without sacrificing quality. In my testing, it is a bit intensive on system design. Apart from expensive training and inference, you now have to store embeddings for individual tokens as well.