Watershed Bio’s Post

In a standard scRNA-seq analysis pipeline, you select the top ~2000 variable genes for downstream analysis (eg. clustering). However, my recent experiment suggests that you should not do this for foundation models. Here is what I did... The Universal Cell Embeddings (UCE) foundation model, part of a bigger "virtual cell" initiative, takes a raw cells x counts matrix as input and outputs a 1280 dimensional vector that contains biological meaning as output. This is then used for downstream analysis. The power here is that you get the same vectors every time. There is no fine-tuning of the model. So you can make comparisons with any datasets that have never been run through the model, and therefore do things like annotate, given metadata cells from other datasets. As I said in a previous post, this can take a long time if you're running it locally. One hypothesis, inspired by one of the comments, was that I could put in an abbreviated dataset of only variable genes, and get a faster result without sacrificing accuracy - a good thing when computational resources are limited. Experimental design: I ran the following 3 datasets through UCE. 1. The full dataset (positive control). 2. The dataset containing the most variable genes (experimental). 3. The dataset containing a random selection of genes (negative control). My results: I found that the dataset containing the most variable genes did not have the same level of cell type separation compared to the full dataset, with the negative control performing worse than both of them. This can be seen by assessing PCA space of the concatenated data (image below). Further quantification via Shannon entropy (to measure diversity) confirms this (see my jupyter notebook in the comments). What this means for you: This suggests that for UCE, and perhaps for other foundation models (geneformer, scGPT), you should run the full dataset through it to get the best results, and the typical practice of only selecting variable genes may not apply to the use of foundation models. Zooming out: There has been an uptick in people asking me questions around AI as it relates to single-cell in the past few weeks (perhaps because I'm posting about it). Even if you're a natural skeptic (like me), you should at least be familiar with them, because like the black boxes before it (eg. t-SNE/UMAP), these tools don't appear to be going anywhere. And they do indeed have potential to accelerate our workflows. If you are doing work in this space, or interested in doing work in this space, please let me know. A jupyter notebook showing my work is linked in the comments. I hope you all have a great day.

🌟 𝐌𝐈𝐓 𝐫𝐞𝐬𝐞𝐚𝐫𝐜𝐡𝐞𝐫𝐬 have introduced 𝐆𝐞𝐧𝐒𝐐𝐋, an innovative generative AI system for databases, simplifying complex statistical analyses on tabular data for users without requiring in-depth technical knowledge. 𝘾𝙖𝙥𝙖𝙗𝙞𝙡𝙞𝙩𝙞𝙚𝙨 𝙤𝙛 𝐆𝐞𝐧𝐒𝐐𝐋: Enables users to make predictions, detect anomalies, impute missing values, correct errors, or generate synthetic data effortlessly. Identifies anomalies in data, such as an unusual low blood pressure reading for a patient typically with high blood pressure. Seamlessly integrates tabular datasets with generative probabilistic AI models for enhanced accuracy. Creates synthetic data that replicates real data, ideal for situations involving sensitive information. 𝘼𝙙𝙫𝙖𝙣𝙩𝙖𝙜𝙚𝙨 𝙤𝙫𝙚𝙧 𝙚𝙭𝙞𝙨𝙩𝙞𝙣𝙜 𝙢𝙚𝙩𝙝𝙤𝙙𝙨: GenSQL is 1.7 to 6.8 times faster than popular AI-based data analysis techniques, delivering more precise results. The probabilistic models in GenSQL are explainable, allowing users to read and edit them easily. 𝘾𝙤𝙢𝙗𝙞𝙣𝙞𝙣𝙜 𝙢𝙤𝙙𝙚𝙡𝙨 𝙖𝙣𝙙 𝙙𝙖𝙩𝙖𝙗𝙖𝙨𝙚𝙨: GenSQL bridges the gap between SQL's data querying capabilities and probabilistic models' deep insights on individual-level implications. Users can query both datasets and probabilistic models using a single programming language with GenSQL. For more details, check out the link: https://lnkd.in/gK7WKMkg For the research paper check out the link: https://lnkd.in/eCBCaMjr #genai #datascience #llm #machinelearning #deeplearning #database #gensql #sql

📍 𝗪𝗲𝗲𝗸 𝟯: 𝗗𝗮𝘁𝗮 𝗣𝗿𝗲𝗽𝗮𝗿𝗮𝘁𝗶𝗼𝗻 𝗮𝗻𝗱 𝗣𝗿𝗲𝗽𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴 𝗳𝗼𝗿 𝗗𝗲𝗲𝗽 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 ☀ 𝗗𝗮𝘆 𝟮: 𝗛𝗮𝗻𝗱𝗹𝗶𝗻𝗴 𝗠𝗶𝘀𝘀𝗶𝗻𝗴 𝗗𝗮𝘁𝗮 Continuing with Week 3, today we delve into handling missing data—a common issue in real-world datasets. Effective strategies to manage missing data are crucial for ensuring robust deep-learning models. ❓ 𝗪𝗵𝘆 𝗠𝗶𝘀𝘀𝗶𝗻𝗴 𝗗𝗮𝘁𝗮 𝗠𝗮𝘁𝘁𝗲𝗿𝘀 Missing data can lead to biased estimates, reduced statistical power, and can compromise the validity of conclusions drawn from the dataset. ▶ 𝗧𝗲𝗰𝗵𝗻𝗶𝗾𝘂𝗲𝘀 𝗳𝗼𝗿 𝗛𝗮𝗻𝗱𝗹𝗶𝗻𝗴 𝗠𝗶𝘀𝘀𝗶𝗻𝗴 𝗗𝗮𝘁𝗮 ◾ 𝗥𝗲𝗺𝗼𝘃𝗮𝗹 𝗼𝗳 𝗠𝗶𝘀𝘀𝗶𝗻𝗴 𝗗𝗮𝘁𝗮𝗖𝗼𝗺𝗽𝗹𝗲𝘁𝗲 𝗖𝗮𝘀𝗲 𝗔𝗻𝗮𝗹𝘆𝘀𝗶𝘀:  Discard rows or columns with missing values. This is simple but can result in significant data loss if many entries are missing. ◾ 𝗜𝗺𝗽𝘂𝘁𝗮𝘁𝗶𝗼𝗻 𝗠𝗲𝘁𝗵𝗼𝗱𝘀 𝙈𝙚𝙖𝙣/𝙈𝙚𝙙𝙞𝙖𝙣/𝙈𝙤𝙙𝙚 𝙄𝙢𝙥𝙪𝙩𝙖𝙩𝙞𝙤𝙣: Replace missing values with the mean (for numerical data), median, or mode (for categorical data). This method preserves data size but can introduce bias and reduce variability. ◾ 𝗔𝗱𝘃𝗮𝗻𝗰𝗲𝗱 𝗜𝗺𝗽𝘂𝘁𝗮𝘁𝗶𝗼𝗻 𝗧𝗲𝗰𝗵𝗻𝗶𝗾𝘂𝗲𝘀: 𝙠-𝙉𝙚𝙖𝙧𝙚𝙨𝙩 𝙉𝙚𝙞𝙜𝙝𝙗𝙤𝙧𝙨 (𝙠-𝙉𝙉) 𝙄𝙢𝙥𝙪𝙩𝙖𝙩𝙞𝙤𝙣: Use the average value of the k-nearest neighbours to impute missing values. This technique takes the data structure into account but is computationally expensive for large datasets. 𝙈𝙪𝙡𝙩𝙞𝙥𝙡𝙚 𝙄𝙢𝙥𝙪𝙩𝙖𝙩𝙞𝙤𝙣: Create multiple imputations for the missing values and average the results. This method accounts for uncertainty in the missing data but is complex and computationally intensive. ◾ 𝗣𝗿𝗲𝗱𝗶𝗰𝘁𝗶𝗼𝗻 𝗠𝗼𝗱𝗲𝗹𝘀 𝗳𝗼𝗿 𝗜𝗺𝗽𝘂𝘁𝗮𝘁𝗶𝗼𝗻 𝙍𝙚𝙜𝙧𝙚𝙨𝙨𝙞𝙤𝙣 𝙄𝙢𝙥𝙪𝙩𝙖𝙩𝙞𝙤𝙣: Use regression models to predict and replace missing values. This method can handle relationships between variables but requires model fitting and can be computationally expensive. 𝘿𝙚𝙚𝙥 𝙇𝙚𝙖𝙧𝙣𝙞𝙣𝙜 𝙈𝙤𝙙𝙚𝙡𝙨: Train neural networks specifically to predict missing values based on the available data. These models can capture complex relationships and interactions but require significant computational resources and a well-prepared dataset. ◾ 𝗜𝗻𝘁𝗲𝗿𝗽𝗼𝗹𝗮𝘁𝗶𝗼𝗻 For time-series data, use interpolation methods (e.g., linear, spline) to estimate missing values based on existing data points. This method is suitable for sequential data but assumes a specific trend, which might not always be accurate. Each of the above-mentioned techniques has its tradeoffs and it’s essential to tune this to the requirement of the model and the problem at hand ❗ 𝗡𝗼𝘁𝗲𝘄𝗼𝗿𝘁𝗵𝘆 𝗿𝗲𝘀𝗼𝘂𝗿𝗰𝗲𝘀: https://lnkd.in/gnirfva6 https://lnkd.in/gQtJ6U4C #DeepLearning #AI #MachineLearning #NeuralNetworks #TechLearning #LearnWithMe #12WeeksOfDeepLearning

#Technology #DataAnalytics #DataDriven A Simple Regularization for Your GANs: How to capture data distributions effectively with GANs In 2018, I had the privilege of orally presenting my paper at the AAAI conference. A common feedback was that the insights were clearer in the presentation than in the paper. Although some time has passed since then, I believe there’s still value in sharing the core insights and intuitions. The paper addressed a significant problem of reliably capturing modes in a dataset with Generative Adversarial Networks (GANs). This article is formulated around my intuitions of GANs and derives the proposed approach from those intuitions. Finally, I present a copy-paste solution for those who want to try it out. If you are familiar with GANs, feel free to skip to the next section.Paper: [Sharma, S. and Namboodiri, V., 2018, April. No modes left behind: Capturing the data distribution effectively using gans. In Proceedings of the AAAI Conference on Artificial Intelligence] (paper, github) A quick intro to Generative Adversarial Networks GANs are used to learn Generators for a given distribution. This means that if we are given a dataset of images, say of birds, we have to learn a function that generates images that look like birds. The Generator function is usually deterministic, so it relies on a random number as input for stochasticity to produce a variety of images. Thus, the function takes a n-dimensional number as input and outputs an image. The input number z is typically, low-dimensional and randomly sampled from a uniform or a normal distribution. This distribution is called the latent distribution Pz. We refer to the space of “all possible” images as the data space X, the set of bird images as real R, and their distribution as Pr. The Generator at optimality, maps each value of z to some image that has a high likelihood of belonging to R. GANs solve this problem using two learned functions: a Generator (G) and a Discriminator (D). G takes the number z as input to produce a sample from data space, x = G(z). At any point, we call the set of all images generated by G as fake F, and their distribution Pg. The Discriminator takes a sample x from the data space and outputs a scalar D(x), predicting its probability of belonging to the real or fake distribution. Initially, neither G nor D is well-trained. We sample some random numbers at each training step and pass them through G to get some fake samples. Similarly, we take an equal number of random samples from the real subset. D is trained to output 0 for fake, and 1 for real samples via cross-entropic loss. G is trained to fool D such that the output of D(G(z)) becomes 1. In other words, increase the probability of generating samples that score high (produce more), and decrease it for those that score low. The gradients flow from the loss function through D and… #MachineLearning #ArtificialIntelligence #DataScience

🚀 GenSQL a #genai system for databases 🚀 MIT researchers have introduced a groundbreaking advancement in database technology with Generative AI Databases, dubbed GenSQL. This innovative approach leverages generative AI to transform how databases are designed, queried, and optimized. 🔍 Key Highlights: • Enhanced Query Generation: GenSQL uses AI to automatically generate complex queries, significantly reducing manual effort and errors. • Optimized Performance: AI-driven optimization techniques ensure faster and more efficient database performance. • Adaptive Learning: The system continuously learns and adapts to new data, improving over time without human intervention. • User-Friendly: Designed to simplify database interactions, making it accessible even for non-experts. 🌟 Why It Matters: GenSQL is set to revolutionize database management, offering unprecedented efficiency and accuracy. It’s a major leap forward for industries relying on data-driven decision-making, from finance to healthcare. Read more about this cutting-edge development MIT below #ArtificialIntelligence #GenSQL #DatabaseInnovation #MITResearch #TechNews https://lnkd.in/g4SrgkSe

A Simple Regularization for Your GANs: How to capture data distributions effectively with GANs In 2018, I had the privilege of orally presenting my paper at the AAAI conference. A common feedback was that the insights were clearer in the presentation than in the paper. Although some time has passed since then, I believe there’s still value in sharing the core insights and intuitions. The paper addressed a significant problem of reliably capturing modes in a dataset with Generative Adversarial Networks (GANs). This article is formulated around my intuitions of GANs and derives the proposed approach from those intuitions. Finally, I present a copy-paste solution for those who want to try it out. If you are familiar with GANs, feel free to skip to the next section.Paper: [Sharma, S. and Namboodiri, V., 2018, April. No modes left behind: Capturing the data distribution effectively using gans. In Proceedings of the AAAI Conference on Artificial Intelligence] (paper, github) A quick intro to Generative Adversarial Networks GANs are used to learn Generators for a given distribution. This means that if we are given a dataset of images, say of birds, we have to learn a function that generates images that look like birds. The Generator function is usually deterministic, so it relies on a random number as input for stochasticity to produce a variety of images. Thus, the function takes a n-dimensional number as input and outputs an image. The input number z is typically, low-dimensional and randomly sampled from a uniform or a normal distribution. This distribution is called the latent distribution Pz. We refer to the space of “all possible” images as the data space X, the set of bird images as real R, and their distribution as Pr. The Generator at optimality, maps each value of z to some image that has a high likelihood of belonging to R. GANs solve this problem using two learned functions: a Generator (G) and a Discriminator (D). G takes the number z as input to produce a sample from data space, x = G(z). At any point, we call the set of all images generated by G as fake F, and their distribution Pg. The Discriminator takes a sample x from the data space and outputs a scalar D(x), predicting its probability of belonging to the real or fake distribution. Initially, neither G nor D is well-trained. We sample some random numbers at each training step and pass them through G to get some fake samples. Similarly, we take an equal number of random samples from the real subset. D is trained to output 0 for fake, and 1 for real samples via cross-entropic loss. G is trained to fool D such that the output of D(G(z)) becomes 1. In other words, increase the probability of generating samples that score high (produce more), and decrease it for those that score low. The gradients flow from the loss function through D and then through G. Please refer to the… #MachineLearning #ArtificialIntelligence #DataScience

What does generative AI in databases look like? The short answer: imagine searching for anomalies within one individual rather than comparing to broader population trends. This article mentions that GenSQL, a generative AI model for SQL and allows users to modulate and extract data from databases without knowing the SQL query syntax, can be used to analyze an individual’s health data using a particular health metric and create a trend for the specific individual. For example, in analyzing BMI, instead of comparing that individual’s BMI to the greater population at large, it can look at the individual’s BMI and determine anomalies across the various measurement points and create a more custom statistical analysis for the individual. Furthermore, GenSQL is a probabilistic model that incorporates uncertainty in its decision-making to answer questions about likelihood or underrepresented data in the database. GenSQL can be useful in creating synthetic data for clinical trials when real data cannot be used. In conclusion, GenSQL, with its use of probability modeling, makes SQL more accessible for the user interested in providing experimental conditions to test different hypotheses. See the article from MIT News here: https://lnkd.in/gMVAcdMg

A great insight on the foundation of ML Algorithm worth reading, thanks to Abhishek For putting across the key algorithms together

Insightful ML Algorithms

Exploring the Foundations of Machine Learning: Key Algorithms for Data-Driven Decision Making As we navigate the complex landscape of data science and artificial intelligence, it's crucial to understand the core algorithms that power modern machine learning applications. Let's examine ten fundamental techniques that form the backbone of data-driven insights: ◈ Random Forest: An ensemble method that leverages the wisdom of multiple decision trees, offering robust performance in both classification and regression tasks. Its strength lies in mitigating overfitting through collective decision-making. ◈ Naive Bayes: Rooted in probabilistic theory, this algorithm excels in text classification and spam filtering. Its efficiency stems from the assumption of feature independence, allowing for rapid training and deployment. ◈ Decision Trees: These intuitive models provide transparent decision-making processes, making them invaluable for both predictive modeling and explanatory analysis in business contexts. ◈ AdaBoost (Adaptive Boosting): A pioneering boosting algorithm that iteratively improves model performance by focusing on misclassified instances, demonstrating the power of ensemble learning in handling complex datasets. ◈ Gradient Boosting Machines (GBM): An advanced ensemble technique that sequentially builds models to correct errors, offering state-of-the-art performance in various domains, from finance to healthcare. ◈ Logistic Regression: Despite its simplicity, this algorithm remains a cornerstone of binary classification, providing interpretable results and probabilistic outputs crucial for risk assessment and decision boundary analysis. ◈ K-Means Clustering: An unsupervised learning approach essential for market segmentation, anomaly detection, and pattern discovery in high-dimensional data spaces. ◈ Support Vector Machine (SVM): Renowned for its effectiveness in high-dimensional spaces, SVM's ability to define optimal hyperplanes makes it indispensable in image classification and bioinformatics. ◈ K-Nearest Neighbors (KNN): A versatile, non-parametric method that shines in recommendation systems and pattern recognition tasks, leveraging the principle that similar data points cluster together. ◈ Regression Techniques: From linear to polynomial models, regression analysis forms the foundation of predictive modeling, offering insights into variable relationships and forecasting trends. The mastery of these algorithms empowers data scientists to extract meaningful insights, drive innovation, and solve complex business challenges. As we continue to push the boundaries of AI, a deep understanding of these foundational techniques remains paramount. What are your experiences with implementing these algorithms in real-world scenarios? How have they transformed your approach to data-driven decision-making? #MachineLearning #DataScience #ArtificialIntelligence #AdvancedAnalytics