Learning Systems and Robotics Lab

We are very excited for #ICRA2024! We have six publications and several workshop contributions at this year's conference! Dates, times, and rooms of each talk are given in the link below, along with links to their PDFs, videos, and code. Link: https://lnkd.in/gRcjUhZy Angela Schoellig, Munich Institute of Robotics and Machine Intelligence (MIRMI) at the Technical University of Munich, Vector Institute, University of Toronto Robotics Institute

Our #CoRL2024 Workshop on Robot Manipulation in a World of Abundant Data is starting in an hour! Join us in person in Room Jupiter or online at https://lnkd.in/d-MnVXa7. You can post questions to the speakers on Slido: https://lnkd.in/dSZkVEqm Workshop website: https://lnkd.in/dJGk-Ad9

We are just four days away from our #CoRL2024 Workshop on Robot Manipulation in a World of Abundant Data! Check out our fantastic lineup of speakers and join us on Saturday, November 9 @ 8:45 CET: https://lnkd.in/dJGk-Ad9 🏠 Location: Room Jupiter, Science Congress Center Munich 🌐 Livestream: https://lnkd.in/d-MnVXa7 📑 Contributed Papers: https://lnkd.in/dt9mTutg ✍ Ask Your Questions to the Speakers: https://lnkd.in/dSZkVEqm 🗓 Workshop Program: 08:45 – 09:00: Opening Remarks and Theme A Introduction 09:00 – 09:30: Invited Talk: Sergey Levine 09:30 – 10:00: Invited Talk: Jens Lundell 10:00 – 10:30: Invited Talk: Ankur Handa 10:30 – 11:00: Coffee Break and Morning Poster Session 11:00 – 11:30: Invited Talk: Carmelo (Carlo) Sferrazza 11:30 – 12:00: Theme A Panel Discussion 13:45 – 13:55: Theme B Introduction 13:55 – 14:30: Spotlight Talks 14:30 – 15:00: Invited Talk: Ted Xiao 15:00 – 15:30: Invited Talk: Christian Gehring 15:30 – 16:00: Coffee Break and Afternoon Poster Session 16:00 – 16:30: Invited Talk: Mohsen Kaboli 16:30 – 17:00: Invited Talk: Katerina Fragkiadaki 17:00 – 17:30: Invited Talk: shuran song 17:30 – 18:00: Theme B Panel Discussion 🔖 Organizers: Angela Schoellig, Animesh Garg, Karime Pereida, Oier Mees, Ralf Römer, Martin Schuck, SiQi Zhou

How can robots act with common-sense safety in mind? Our new framework combines 3D mapping, LLM reasoning, and safety filters to avoid semantically unsafe actions like carrying a cup of coffee over a laptop. Arxiv: https://lnkd.in/gZFbqPQz Full video: https://lnkd.in/gQW4vTU8 Website: https://lnkd.in/g3wRsGaq How does it work? Our semantic safety filter uses a perception module to create a 3D semantic map. LLMs generate constraints on object positions, behaviour, and poses, defining safe actions. Every command is certified before execution, ensuring safe robot motion. For each scene, collision constraints are generated from object point clouds. In addition, semantic constraints are based on point clouds, semantic labels, and LLM-derived safety conditions: spatial, behavioural, and pose-based. Using our framework, a robot transports a dry sponge at normal speed, but when holding a cup of water, the robot now also considers semantic constraints, for example, not moving above the laptop and avoiding rotating the cup to avoid spillage. Researchers: Lukas Brunke, Yanni Zhang, Ralf Römer, Jack Naimer, Nikola Staykov, SiQi Zhou, Angela Schoellig #Robotics #AI #RobotSafety #LLM #AIResearch Technical University of Munich, Munich Institute of Robotics and Machine Intelligence (MIRMI) at the Technical University of Munich, University of Toronto Robotics Institute, Vector Institute

How can a robot navigate a crowd while jointly predicting human motion & planning robot motion, all while satisfying explicit safety constraints? Our latest paper accepted to IEEE Transactions on Robotics (T-RO) tackles this challenge using a Bilevel MPC formulation. Preprint, videos and code: https://lnkd.in/gJJUFWKm Angela Schoellig Munich Institute of Robotics and Machine Intelligence (MIRMI) at the Technical University of Munich Vector Institute University of Toronto Robotics Institute

 🚀Federico Pizarro Bejarano and Lukas Brunke's latest paper is out! They tackle safety in reinforcement learning by integrating safety filters during training, not just evaluation. This approach boosts performance and sample efficiency, verified on a Crazyflie 2.0 drone. Check it out! 📰 arXiv: https://lnkd.in/gSm2wfit 📽️ video: https://lnkd.in/gAbPgAge #ReinforcementLearning #AI #SafetyInAI #drones #Control #robotics Angela Schoellig, Vector Institute, Munich Institute of Robotics and Machine Intelligence (MIRMI) at the Technical University of Munich, University of Toronto Robotics Institute

Big thanks to everyone who visited us at the TUM Open House! It was amazing to meet the next generation of roboticists, with Hello Robot passing out high fives to excited kids! Angela Schoellig, Munich Institute of Robotics and Machine Intelligence (MIRMI) at the Technical University of Munich, Robotics Institute Germany, Hightech Agenda Bavaria.

Interested in doing a PhD with us? We are offering exciting PhD positions in #robotics and #machinelearning through multiple PhD programs: - Munich Center for Machine Learning (MCML): https://lnkd.in/dufBrBvm (apply between Oct. 15 and Nov. 14) - relAI Konrad Zuse School of Excellence: https://lnkd.in/dzrXHrZE (applications open Dec. 2024 / Jan. 2025) - Three positions directly at our lab: https://lnkd.in/dUhBYg3g (apply until Nov. 15) Angela Schoellig Munich Institute of Robotics and Machine Intelligence (MIRMI) at the Technical University of Munich Robotics Institute Germany University of Toronto Robotics Institute Technical University of Munich Hightech Agenda Bavaria

Join us Oct. 3 at the TUM Open House in Garching! From 10:00 to 17:00, visit our booth (#3) to see our Lego-building robot and Hello Robot teleoperation demo! Don’t miss Prof. Angela Schoellig's talk from 13:15 to 13:45! German: https://lnkd.in/gBvVXenK English: https://lnkd.in/gpt9kfTy Munich Institute of Robotics and Machine Intelligence (MIRMI) at the Technical University of Munich, Robotics Institute Germany, Hightech Agenda Bavaria, Technical University of Munich

Are you a TUM student with a passion for machine learning and robotics? Don’t miss out on our upcoming courses this semester, where you will dive deep into machine learning, computer vision, control systems, and their applications in robotics. Check out the full course offerings here: https://lnkd.in/dJ-5Q4MF #Robotics #MachineLearning #AI #Control #ComputerVision