Chao Xie’s Post

Pairing Gaussian Splatting with our 3D Full-Body Scanner NEO unlocks a new level of 3D reconstruction. See the result for yourself. 3D professionals will know the difficulties of reconstructing hair, transparent glasses and mesh using traditional scanning and modelling techniques. Gaussian Splatting makes easy work of this, even capturing the reflection of light on the glasses as the model turns. Book a scan service session with us to explore the mind-blowing results of Gaussian Splatting for yourself. #3dscanning #3dmodeling #innovation #gaussiansplatting

Check out how #GaussianSplatting looks with the data from our #FullBodyScanner NEO. Looking forward to the next steps! 🚀

The challenge with existing mixed-reality devices is they only track hands but ignore objects. HOLD, built with PyTorch Lightning, introduces a new self-supervised method for 3D hand-object reconstruction. https://lnkd.in/gREPCDYA

Leveraging Vision Transformers to Elevate Accuracy, Depth, and Efficiency in 3D Reconstruction Techniques

Introducing LaRa: The Future of 360° Radiance Field Reconstruction! 🌟 LaRa, a groundbreaking feed-forward model, significantly advances 3D reconstruction by unifying local and global reasoning in transformer layers. This innovative approach leverages Gaussian Volumes, an image encoder, and Group Attention Layers to achieve high-quality and rapid 360° radiance field reconstructions. Unlike traditional methods constrained by small baselines, LaRa excels in handling large baselines, offering high fidelity and robustness even in zero-shot and out-of-domain testing scenarios. LaRa addresses the core challenge of reconstructing objects' shape and appearance from multi-view images, a critical task in computer vision and graphics. By overcoming limitations associated with dense image captures and small camera baselines, LaRa introduces a novel volume transformer. This method progressively and implicitly performs feature matching, efficiently aggregating local and global features to produce photorealistic renderings from just four input images. LaRa demonstrates exceptional performance, achieving high-resolution 360° novel view synthesis with minimal training resources—only four GPUs over two days. The model's efficiency is enhanced by a coarse-to-fine decoding process and efficient rasterization, enabling high-quality mesh reconstruction with off-the-shelf algorithms. With its robust zero-shot generalization and ability to handle diverse viewpoints, LaRa sets a new standard in 3D reconstruction, pushing the boundaries of what's possible in visual effects, e-commerce, virtual and augmented reality, and robotics. "LaRa: Efficient Large-Baseline Radiance Fields" Anpei Chen, Haofei Xu, Stefano Esposito, Siyu Tang, Andreas Geiger Github Repo: https://lnkd.in/gJ78NjqB Arxiv Preprint: https://lnkd.in/g5nrJsuK #Transformers #FewShot #3DReconstruction #ComputerVision #AiResearch

Reconstructing detailed 3D objects from single-view images remains a challenging task due to the limited information available (scale, texture). How about combining 2D diffusion models and Gaussian Splatting? Here is "FDGaussian: Fast Gaussian Splatting from Single Image via Geometric-aware Diffusion Model" and here what makes it special: 🛬 Orthogonal plane decomposition mechanism to extract 3D geometric features from the 2D input i.e consistent multi-view images 👀 Incorporating epipolar attention to fuse images from different viewpoints. This improves the current pipeline of 3DGS reconstruction. ✨ High consistency across different views and reconstructs high-quality 3D objects, both qualitatively and quantitatively All links in the comments #gaussiansplatting #imgto3D #machinelearning #diffusionmodels #3Dgeneration #sfm #multiviewgeometry #futurism #img23d

My pleasure to share another achievement in 3D Geometric Modeling by the grace of Allah Almighty:

Precise 3D monocular reconstruction without any priors is hard and ambiguous problem. Many priors however can be computed from a 1st person perspective and interactions like hands. How far can we go? I stumbled upon the great paper: "HOLD: Category-agnostic 3D Reconstruction of Interacting Hands and Objects from Video" from the teams at ETH Zürich and Max Planck Institute. Most existing methods for hand-object reconstruction from RGB either assume pre-scanned object templates or heavily rely on limited 3D hand-object data, restricting their ability to scale and generalize to more unconstrained interaction settings HOLD is the first category-agnostic that can reconstruct detailed 3D geometries of novel objects and hands in interaction from in-the-lab static 3rd-person videos and challenging in-the-wild 1st-person videos. They develop a compositional articulated implicit model that can reconstruct disentangled 3D hand and object from 2D images. They also further incorporate hand-object constraints to improve hand-object poses and consequently the reconstruction quality. The method does not rely on 3D hand-object annotations while outperforming fully-supervised baselines in both in-the-lab and challenging in-the-wild settings. Checkout the links in the comments! #machinelearning #deeplearning #computervision #futurism #densereconstruction #objecttracking #objectrecognition #objectreconstruction

A new issue (Octrober 2024) of Graphical Models journal is out! Check new papers about image editing, 3D reconstruction, image restoration, NURBS, and image vectorization. https://lnkd.in/gMx3RzVy

I always thought camera pose estimation is necessary for 3D reconstruction, until Zequn and Stephen proved me wrong! Introducing PreF3R, purely feed-forward 3D Gaussian Splatting without any intermediate pose estimation and COLMAP initialization. Video in, 3D Gaussians and novel view rendering out. Key technique: spatial memory network from Spann3R and Gaussian head supervised by pointmap loss plus photometric loss. See more at: https://lnkd.in/eGHaks2x