Jan Beger’s Post

Important insights from Jan Beger, as always, regarding the potentials and pitfalls of multimodal AI in medical diagnostics and the importance of ensuring compatibility and proper surveillance of these models before implementing to comply with i.e the EU AI act, as they currently raise both ethical and reliability concerns

This paper reviews the progress and challenges in using multimodal AI in medicine, with a focus on applications across medical specialties and the technical issues involved in implementing these systems. 1️⃣ Multimodal AI in healthcare combines diverse data sources (e.g., imaging, clinical data, genomics) to improve clinical decision-making by enhancing data interpretation, with models reviewed showing a 6.2% improvement in AUC over unimodal AI approaches. 2️⃣ The review analyzed 432 studies from 2018 to 2024, finding multimodal AI research prevalent in radiology and text data combinations, especially for nervous and respiratory systems, while less focus was observed in musculoskeletal and urinary systems. 3️⃣ Data fusion is a critical component, with intermediate fusion (merging encoded features before final model layers) being the most common method (79%), though earlier fusion techniques show promise for maximizing cross-modal data learning. 4️⃣ Challenges for multimodal AI include inconsistent data availability across patients, cross-departmental data silos, and model architecture complexity, often needing different encoding strategies for varied data types. 5️⃣ Public datasets are pivotal for model training, though most studies rely on internal validation, and external validation remains limited, hindering generalizability and regulatory progress. 6️⃣ Current multimodal AI systems lack regulatory clearance, underscoring barriers to clinical integration like interoperability, privacy concerns, and the need for explainable AI, which are necessary to bridge research and practical application gaps.

Evaluating multimodal AI in medical diagnostics - npj Digital Medicine Title: Evaluating Multimodal AI in Medical Diagnostics #Introduction Multimodal AI systems combine different types of data to enhance medical diagnostics, improving accuracy and efficiency. #Challenges in Medical Imaging The integration of various imaging modalities poses challenges in data interpretation and analysis for healthcare professionals. #Benefits of Multimodal AI Multimodal AI offers improved diagnostic accuracy, reduced interpretation time, and enhanced patient outcomes in medical imaging. #Case Studies Several case studies demonstrate the effectiveness of multimodal AI in diagnosing complex medical conditions with high accuracy. #Future Implications The future of medical diagnostics will be shaped by the continued development and implementation of multimodal AI systems. #Conclusion Multimodal AI has the potential ai.mediformatica.com #medical #digital #gpt4 #this #medicine #clinical #diagnostics #intelligence #study #vision #digitalmedicine #preview #digitalhealth #healthit #healthtech #healthcaretechnology @MediFormatica (https://buff.ly/3LTpi06)

👉🏼 Generating synthetic clinical text with local large language models to identify misdiagnosed limb fractures in radiology reports 🤓 Jinghui Liu 👇🏻 https://lnkd.in/entpHxYh 🔍 Focus on data insights: - 🌐 Local LLMs can generate synthetic clinical reports, providing a cost-effective alternative to commercial models. - 📈 Performance of synthetic reports is comparable to using real-world annotated data for training AI systems. - 🧠 Incorporating synthetic data significantly enhances model training, achieving over 90% effectiveness relative to real-world data. 💡 Main outcomes and implications: - 🔑 Open-source LLMs facilitate the generation of essential training data while addressing privacy concerns associated with sensitive clinical information. - ⚙️ The study supports the notion that synthetic data can be reliable and beneficial in healthcare applications, particularly in diagnostic processes. - 🏥 The findings could lead to greater adoption of synthetic data generation techniques in medical AI, improving efficiency and outcomes in diagnostics. 📚 Field significance: - 🔬 Highlights the transformative potential of artificial intelligence in healthcare through effective data utilization. - 💻 Encourages further exploration of open-source tools in medical applications, promoting transparency and accessibility. - 📊 Provides a framework for integrating synthetic data generation in clinical settings, enhancing research and development in medical imaging and diagnostics. 🗄️: [#largeLanguageModels #syntheticData #healthcareAI #radiology #dataPrivacy #machineLearning #diagnostics #clinicalReports]

After the OMICS era it seems the AI era! Despite the many benefits of AI, like improving Precision Medicine and remote patient monitoring, its full use in healthcare faces key challenges. These include poor data quality, lack of large labeled datasets, data privacy and safety issues, biases in AI, legal and ethical issues, and cost and implementation hurdles. Specially look at the section 😀 AI: myth versus reality 😀 A wonderful paper for the people who are thinking of ML to use in their research in precision medicine. https://lnkd.in/dd6gAdka

Fascinating and timely article highlighting the importance of including the #PatientPerspective when applying #AI to #MedicalResearch and delivery. "While AI holds transformative possibilities for medicine, it is important that we are clear-eyed about both the risks and the opportunities... The many other factors that influence medical care, including the systematic disregard for patients’ knowledge of their own experiences, are often ignored in these discussions... ...some of the most promising areas of research are participatory approaches to machine learning and patient-led medical research." https://lnkd.in/d9mkxg3n

The controversy is served: how will we identify those physicians who could benefit from AI in their performance & differentiate them from those who will get "hurt" by it? Really interesting data on this article about the potential use of medical IA tools on image interpretation.

Having fun with a full day hackathon to lay the foundation for a clinical QA-4-AI layer - together with developers from UMCU, Sectra Benelux and Sectra Sweden. The QA-4-AI layer will further augment our Sectra AI Amplifier infra that forms the basis for radiological AI orchestration in our hospital. Continuous, prospective AI Quality Assurance is crucial because despite the massive potential of AI in radiology, currently efficiency and cost gains are not being achieved because AI is applied in an incomplete way: successfully implemented from an IT perspective; but due to a lack of trust and unestablished local safety margins, it is operating with minimal impact on the medical process. What's holding us back is a lack of quantitative knowledge about the local perfomance of neural networks: how it works in *your* hospital on *your* patients, in a multitude of different clinical settings. [Briefly, local perfomance is a function of the quality of the neural network itself (initial training), and external factors that manifest in AI generalization gaps, model drift, and long-tail failure. Examples of relevant external factors are local population, equipment, scan protocols, and guidelines, but also impactful factors like human-AI(-output) interaction.] Edwin Bennink, Christian Mol, Nick Besselink, Martin Lindvall, Erik Gabrielsson, Emilia Ståhlbom, Sectra, Universitair Medisch Centrum Utrecht

If this is a field we really want to venture into, I think the future is one where doctors will need some sort of baseline software engineering training or else, as the article has explained to us, we won't really understand the relationship between AI and medicine! To be able to truly understand how to utilize the power of computer code in medicine it makes sense that the person attempting to do that should understand both coding and medicine well! As a doctor who's also trained in coding languages that can be used in AI building programs, having the knowledge of when human intellect will trump computer intellect and vice versa is priceless! It becomes a team effort between the human brain and the machine brain, where strengths and weaknesses from each "entity" are evaluated and taken into account on a case by case basis. This is why I've taken a pause on my overall career of getting into medical school and am currently getting trained to become a software engineer! I truly think the future of personalized medicine and medicine in general is in this overlap!

At the AWS Summit in Washington, D.C., Dr. Christine Tsien Silvers from Harvard Medical School highlighted the transformative potential of generative AI in healthcare. She discussed how a large language model can assist clinicians by accurately interpreting arterial blood gas (ABG) test results. These tests, crucial in emergency situations, measure oxygen and carbon dioxide levels in the blood, aiding in the diagnosis of conditions like sepsis and respiratory failure. Through the innovative use of generative AI, Harvard Medical School has enhanced the speed and accuracy of ABG result interpretations, improving from less than 50% to 98% accuracy with a two-step process involving prompt engineering and mathematical calculations. This advancement not only increases clinical efficiency but also reduces errors, ultimately contributing to safer patient care. Dr. Silvers emphasized that while challenges remain, the integration of generative AI in healthcare is a promising step towards mitigating clinician burnout and enhancing overall patient outcomes. https://lnkd.in/gb-3Zm4D https://lnkd.in/drFpsjMU healthtechmagazine.net