OpenCV reposted this
AI Engineer | R&D Specialist | AI, Computer Vision, Industrial, Biomedical Engineering
🔹Using DepthPro and YOLO for Depth Measurement and 3D Reconstruction🔹 Distance + Depth + Point Cloud + 3D Bounding Box I’ve been working on integrating the DepthPro model with YOLO segmentation to create a system for metric distance measurement, depth estimation, and 3D point cloud reconstruction with object segmentation. The goal is to develop an efficient and accurate setup for capturing depth and spatial relationships in a single process. DepthPro is a powerful tool for monocular depth estimation, producing high-resolution depth maps that allow us to measure distances without needing detailed camera data. Combined with YOLO for object detection, this approach enables the generation of 3D point clouds with bounding boxes around objects — bringing us closer to creating monocular systems for use in autonomous vehicles. Big thanks to Nicolai Nielsen and TAI DO for their inspiring ideas on reconstruction and distance estimation that helped shape this approach. Looking forward to seeing where this technology can take us in the field of 3D vision! #ComputerVision #3DReconstruction #DepthEstimation #AI #MachineLearning
What algorithm are you using for the 3D reconstruction?
Is it one or multiple models fusion?
If we are using depth camera here, where is the reconstruction happening. Depth camera already gives a point cloud which is clustered to form objects of interest. Just wanted to be clear about this wonderful work. As for as I know reconstruction done based on monocular vision, not from depth vision
What hardware you are using for real-time application
Very cool. From your experience, is DepthPro works in certain domains / environments better than other?
repository or paper research? looks amazing!
Did you use the DepthPro model as is or did you retrain it ?
A lot of research is spent on MDE analysis and improvements, but I believe people forget the fact that even with massive training and processing, in the end it is still only a model. A model that is trained to work in 90% of all circumstances, but that will not work in unpredictable environments and/or alternating ambient light conditions whilst coping with some of the weird things autonomous vehicles will experience during driving. We also forget the calculations needed to derive depth data are expensive and that the data is not even real-time. Do we really want cars to make decisions that affect people's lives & safety when alternative, better, technologies exist? I hope not ! Let's conclude this should be good enough for home robotics, but it should not be acceptable for what is considered a safe car.
Hi, This looks really cool. what is the latency like for Depth pro having combined it with yolo?
AM-R&D-ARAS at TVS Motors | IITP-Mechatronics, Robotics, Deep Learning, Computer Vision.
2wHow are you able to calculate depth using monocular data only, I guess you are only getting approximate idea of the depth