🚨 Our paper was accepted for @COLM_conf!
As we face a mental health crisis and lack of access to professional care, many turn to AI as a solution. But how does ethical automated care look like and are models safe enough for patients?
Paper: https://lnkd.in/g2nk86pq
Generally, AI-powered digital mental health tools could be a game-changer, potentially reaching patients stuck on waitlists or without care. The idea? Task-autonomous agents could do individual tasks and chatbots could offer real-time, personalized support and advice.
But hold up! 🛑 As these AI models enter mental healthcare, we need to ask: Are they ready for this high-stakes field where mistakes can have serious consequences? How do we ensure ethical implementation?
Our study tackles these questions head-on, proposing a framework that:
1️⃣ Outlines levels of AI autonomy
2️⃣ Sets ethical requirements
3️⃣ Defines beneficial default behaviors for AI in mental health support
We put 14 state-of-the-art language models to the test, including 10 off-the-shelf and 4 fine-tuned models. With mental health clinicians, we designed 16 mental health-related questions covering user conditions like psychosis, mania, depression, and suicidal thoughts. 📊
The results? 😬 Not great. All tested language models fell short of human professional standards. They struggled with nuances and context, often giving overly cautious or sycophantic responses. Even worse, most models could potentially cause harm or worsen existing symptoms.
Fine-tuning for mental-health-related tasks is also not a magic fix, as safe patient interaction requires awareness of mental health care and inherent safety.
We explored solutions to boost model safety through system prompt engineering and model-generated self-critiques. Adjusting the system prompt yields spare results for all tested models, although the fine-tuned models seem to respond more effectively to the system prompt changes.
Alternatively, we probe how good some models are in recognizing mental health emergencies or unsafe chatbot messages from the previous test (a requirement for self-critiques à la Constitutional AI). The selected models do not perform reliably well.
Conclusion: AI has potential in mental health care, but we're not there yet. Developers must prioritize user safety and align with ethical guidelines to prevent harm and truly help those in need. While AI tools could help address the mental health crisis, safety must come first. 🎯
Great interdisciplinary research collaboration with Declan Grabb, MD, Nina Vasan, MD, MBA and the Stanford Center for AI Safety, Stanford University School of Medicine, Stanford Center for International Security and Cooperation (CISAC), and Freeman Spogli Institute for International Studies that came out initially in April this year: https://lnkd.in/guRPnxaM
#MentalHealth #AIEthics #AISafety
