Yu Cao’s Post

In Retrieval-Augmented Generation (RAG) and Knowledge Base Gap Analysis & Discovery, article chunk vector embedding plays a vital role in LLMs. Implementing dynamic chunking and reranking models enhances RAG performance. Learn more about generating effective vector embeddings for real-world Knowledge Base articles from Raghav Garg's insightful blog post: https://lnkd.in/dyGQgi25

GraphReader, a robust graph-based agent system to tackle the challenges of long-context processing in LLMs. This innovative approach segments lengthy texts into discrete chunks, extracting and compressing essential information into key elements and atomic facts. These components are then used to construct a graph structure that effectively captures long-range dependencies and multi-hop relationships within the text.

Is Cosine Similarity Really About Similarity? 🤔 Cosine similarity is often used to measure semantic similarity in learned embeddings, but this new paper by Netflix researchers raises some important concerns. Here are the key takeaways: 1. Cosine Similarity can give arbitrary or meaningless results in some cases, especially with learned embeddings. 2. Scaling Freedom: The flexibility in how embeddings are scaled during training can lead to non-unique, misleading cosine similarities. 3. Regularization Effects: Different regularization techniques can significantly affect the learned embeddings and distort cosine similarity. 4. Dot Product vs. Cosine: The unnormalized dot product often yields more stable and reliable results compared to cosine similarity. 5. Matrix Factorization Models: In linear models, cosine similarity can be particularly problematic due to scaling issues. 6. Deep Learning Impact: These issues may be even worse in deep models where multiple layers and different regularizations are applied. 7. Proposed Remedies: Train models specifically for cosine similarity or avoid using embedding spaces altogether to improve reliability. 8. Caution: The authors recommend carefully considering how embeddings are learned and normalized before applying cosine similarity. Full paper here: arXiv:2403.05440v1 #seo #Cosinesimilarity #netflix

Many prompt techniques have proven that they can improve performance on many datasets. However, the reasoning step still remains complex. Inspired by humans' ability to generalise and create abstractions, the "Let’s take a step back" prompt asks the model to provide higher-level concepts and abstractions related to the question. So how effective is step-back prompting? Does it actually solve the reasoning step? For more details, you can check the paper summary I wrote about the paper. Article: https://lnkd.in/dxdgY-9i Paper: https://lnkd.in/ddDVxReH

For those companies rushing to use embeddings to take advantage of transformer technologies and neural nets please take your time. The arbitrary logic applied to custom embeddings makes no sense to me. It loses the power of the innate relationship between the data. It's one of the fundamental limitations of the approach that's taken with LLMs in my opinion. https://lnkd.in/gnyZWHrU

Research info 🧐 Proposed Chain-of-Table method enhances the reasoning capability of LLMs by leveraging the tabular structure to express intermediate steps for table-based reasoning. It instructs LLMs to dynamically plan an operation chain according to the input table and its associated question. This evolving table design sheds new light on the understanding of prompting LLMs for table understanding. #google #research #llm

Often times I thought to myself, why bother learning outdated ML/DL techniques that are rarely used in the modern day? This paper reminded me the importance of knowing and understanding the history of a certain field, as the cumulated knowledge can give clues to new approaches to novel technology & techniques. The moment I saw this paper, the first thing that popped up in my head was GANs (Generative Adversarial Networks). Although being quite different, the notion of using an SLM as a discriminator to evaluate the results of another SLM (which acts as a "generator") has to have stemmed from the generator-discriminator idea of a GAN, in one way or another! https://lnkd.in/guDnNDvZ

Interpretable Features in LLMs

When selecting the answer for an multi choice question, LLMs prefer the choices in a given position, or with a specific choice symbol. This is known as selection bias. It hugely reduce the capability of using LLM to do labeling. Luckily, selection bias has been tackled by our intern Froilan (Hyeong Kyu) during his summer internship! Check this amazing work lead by him! Here are some key findings: 1. Selection bias is more extreme when the model’s response is incorrect. 2. Most of selection bias observed to be in the final output layer of the decode. Thus, pruning those nodes reduce selection bias. 3. Adding an "I don't know" option help LLM reduce selection bias even for black box LLM! Amazon Science

Just like us, LLMs are more likely to straight line (always picking A, for example) when they don’t know the right answer. Providing an “I don’t know” response improves model performance in multiple choice questions. New paper with Amazon colleagues and intern discusses new measure for straightlining (which LLM lit calls selection bias😆) and 2 approaches to mitigate