AI at Meta

🎥 Today we’re excited to premiere Meta Movie Gen: the most advanced media foundation models to-date. Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. We’re excited for the potential of this line of research to usher in entirely new possibilities for casual creators and creative professionals alike. More details and examples of what Movie Gen can do ➡️ https://go.fb.me/00mlgt Movie Gen Research Paper ➡️ https://go.fb.me/zfa8wf 🛠️ Movie Gen models and capabilities • Movie Gen Video: A 30B parameter transformer model that can generate high-quality and high-definition images and videos from a single text prompt. • Movie Gen Audio: A 13B parameter transformer model can take a video input along with optional text prompts for controllability to generate high-fidelity audio synced to the video. It can generate ambient sound, instrumental background music and foley sound — delivering state-of-the-art results in audio quality, video-to-audio alignment and text-to-audio alignment. • Precise video editing: Using a generated or existing video and accompanying text instructions as an input it can perform localized edits such as adding, removing or replacing elements — or global changes like background or style changes. • Personalized videos: Using an image of a person and a text prompt, the model can generate a video with state-of-the-art results on character preservation and natural movement in video. We’re continuing to work closely with creative professionals from across the field to integrate their feedback as we work towards a potential release. We look forward to sharing more on this work and the creative possibilities it will enable in the future.

Meta FAIR researchers, in collaboration with BCBL - Basque Center on Cognition, Brain and Language are sharing two breakthroughs that show how AI can help advance our understanding of human intelligence. 1. Research that successfully decodes the production of sentences from non-invasive brain recordings ➡️ https://go.fb.me/w81a3s 2. A study toward understanding the neural mechanisms that coordinate language production in the human brain ➡️ https://go.fb.me/a9drx7 More details on all of this work ➡️ https://go.fb.me/4drvq1 We're excited to share these breakthroughs with the neuroscience community and hope that it will help to inspire and unlock new research in this important space.

Excited to share updates about a new collaboration that showcases the potential of open source AI models to unlock new creative possibilities! This partnership with artist Ruben Fro, Fisheye Immersive, and the Bibliothèque Nationale de France, using our SegmentAnything2 model, has resulted in a breathtaking work called "Deep Diving". We've also updated SAM to make it more easily accessible to artists and developers everywhere. SAM 2.1 is now available on Amazon SageMaker JumpStart, making it easier to deploy and integrate into your applications and workflows. Check out our update here: https://lnkd.in/g83tJYqz 

Starting today, SAM 2.1 is available in #AmazonSageMaker JumpStart, making it easier than ever to deploy it across new projects!

New open source release from Meta FAIR: Audiobox Aesthetics was trained on 562 hours of audio aesthetic data annotated by professional raters across four dimensions to create a model that enables the automatic evaluation of aesthetics for speech, music and sound. Model weights & code ➡️ https://go.fb.me/mh8tm3 Research paper ➡️ https://go.fb.me/irwljd More details ➡️ https://go.fb.me/4gyb2f Audiobox Aesthetics has already been used to enhance our work on Meta Movie Gen and we’re excited about how this research will be able to support the development of more sophisticated generative audio models.

Building on Habitat work, we’re releasing Meta PARTNR: a research framework supporting seamless human-robot collaboration that includes a large-scale benchmark, dataset and large planning model that we hope will enable the community to effectively train social robots. PARTNR repo ➡️ https://go.fb.me/m1uo04 Access to the dataset ➡️ https://go.fb.me/y0dj3l There is an immense amount of potential for innovation and development in the field of human-robot collaboration — and we’re excited to share this work on PARTNR to jump start additional research in this exciting field.

Today we’re sharing a collection of models, datasets and some exciting milestones in the impact of open source — all laddering up to our ongoing work to achieve Advanced Machine Intelligence. How we’re advancing machine intelligence ➡️ https://go.fb.me/dvp6q3 Advancing understanding of human communication with AI ➡️ https://go.fb.me/a4qoi5 Today’s updates from Meta FAIR include: 1. Meta PARTNR, a framework for human-robot collaboration that builds on our existing work in this space with a new dataset and a large planning model enabling robots to accomplish complex tasks alongside humans. 2. Meta Audiobox Aesthetics, a model that enables the automatic evaluation of audio aesthetics, providing a comprehensive assessment of audio quality across speech, music and sound. 3. Open Source Machine Translation Benchmark, a carefully crafted collection from linguistic experts with an aim to collectively build an unprecedented multilingual machine translation benchmark for the community. 4. Two new breakthrough studies using AI to further our understanding of language in the brain. In addition to these releases, we’re also sharing updates on how our research is being put into action in the world — including a new Language Technology Partner Program and how Seamless Communication research is being used in WhatsApp.

Using DINOv2, BrightHeart developed tools to help clinicians identify or rule out signs of congenital heart defects in children faster and more accurately. Founded by two pediatric cardiologists, they've made it their mission to improve fetal heart screening. How BrightHeart uses Meta’s DINOv2 to transform fetal heart screenings ➡️ https://go.fb.me/l9owcf

Leaving Davos today after a whirlwind week of panels, fireside chats, interviews, debates, and Orion demos. I've been talking about the importance of Open Source AI, the way Meta's open research and open source technology is enabling the AI ecosystem worldwide, and the idea that AI is going to go through another revolution over the next half-decade: systems that understand the physical world, have persistent memory, can reason, and can plan hierarchically.

Today, we’re publishing the first stable release of Llama Stack. With this release, Llama Stack now includes streamlined upgrades with backwards compatibility for future API versions and automated verification for supported providers.