Animationsinstitut’s Post

Transcript

In the MAX-R project, Filmakademie Baden-W��rttemberg's Animationsinstitut is dedicated to developing a rich set of open-source XR tools. These tools are built on the TRACER FOUNDATION framework, where TRACER stands for: Toolset for Realtime Animation, Collaboration and Extended Reality. TRACER is a software-agnostic communication framework, providing an infrastructure and API designed for integrating and connecting open-source and proprietary tools into an XR media production pipeline. It provides functionality to connect to data sources from any tool and to distribute and modify them via an open network communication protocol or directly by an abstract, parameter-based user interface. It includes its own scene exchange and storage protocol, but can easily be expanded to include any additional existing scene data format. Thereby, it connects and integrates with any game engine and digital content creation tool such as Unreal, Blender or Maya, through its open API and protocols. Since the TRACER FOUNDATION was developed in C#, it is well-suited for targeting any desktop, VR or XR platform. In MAX-R, we are creating and advancing three tools using the TRACER framework: AnimHost, DataHub and VPET, the Virtual Production Editing Tools, which my colleague Jonas Trottnow, will give you an introduction to, now. VPET or Virtual Production Editing Tools, allow real-time collaborative scene editing with an intuitive interface. VPET offers a holistic approach to light, asset and animation editing for virtual production and XR applications. The initial version of BPD was developed in previous research projects. In MAX-R, VPET's functionality was extended to meet the requirements of virtual studio productions from our partner Brainstorm and their Infinity-Set platform. Additionally, character animation control and customisable user interface elements are currently in development. VPET is an integral part of DIGITAL LOCATIONS demonstrator, which allows access to potential digital film locations either through a browser interface, realised through the web technology wgpuStudio developed by our partner UPF or through VPET making collaborative interaction with the data possible in Augmented Reality. The data between all clients can be synchronised through DataHub, another part of the TRACER framework, that my colleague, Francesco Andreussi, will now introduce. DataHub serves as a communication infrastructure, establishing interoperability between open-source and proprietary tools in XR production pipelines. It can store incoming data and act as a scene server, allowing for the saving and loading of complete scenes as well as their changes over time. DataHub has also been used in another project to synchronise clients in a multiplayer VR experience, which proves its versatility and swift performance. Now, Simon Haag will get into the details of our AnimHost application. AnimHost, part of the TRACER ecosystem, connects animation generators to digital content creation applications, on-set tools like VPET or game engines in general. These animation generators could include deep neural networks trained on motion capture data, video-based, low-cost motion capturing and many more. AnimHost is functionally independent of the animation receiving application and provides an intuitive node-based user interface and an API to support new generators, focusing on real-time interaction. We are currently developing a pipeline to explore and evaluate the effectiveness of AnimHost in the animated short production SURVIVOR. This work explores enhanced XR production processes for animated films utilising machine learning with a "fair use of data" and an "artist in the loop" approach, aiming to empower and support artists by simplifying their workflows. Let's now hear from Simon Spielmamn for a summary and outlook of our work. Thank you Simon, Volker, Jonas and Francesco for this overview of the current TRACER FOUNDATION framework and its associated tools: VPET, DataHub and AnimHost. To sum it up, all TARCER tools represent significant advancements in XR production, paving the way for new possibilities in animated filmmaking by emphasising collaboration and creativity. Note that TRACER FOUNDATION, VPET, DataHub and AnimHost are open-source and can be obtained from our GitHub repository. We also released two additional datasets: An example motion capture dataset we used to train our machine learning model in AnimHost and one of the digital location datasets. More datasets will be released by the end of the project. We hope you enjoyed this brief overview of the current MAX-R developments here at Filmakademie Baden-W��rttemberg. Stay up to date with the latest news, by visiting our website and following our social media channels.