BetterFutureLabs’ Post

Transcript

Hey everyone ready to dive into something pretty wild? I'm ready. Let's do it. We're talking Medas movie Jan today and. It's not just about making film little videos. This is the next level stuff exactly. We're talking AI that can create videos and audio that looks and sounds very good. Yeah, look shockingly real. And it's not just one thing. It's like this whole suite of AI models all working together, right from generating the video to the sound effects, music, editing, the whole 9 yards, the whole thing. OK, so we've got to start with movie Gen. video. That's the kind of the core AI here. It takes your words. And spits out moving pictures. And the secret sauce. The secret sauce is this thing called a temporal autoencoder. TA E for short. Yeah, that's a mouthful, but it's actually a pretty cool concept. Imagine like building a house, right? OK, I'm with you. It's a huge, complicated task, but you break it down into smaller steps. Laying the foundation right, framing the walls, exactly, putting on the roof. That's what the TA does. It breaks down this complex video creation process into these smaller, more manageable chunks. So instead of this jumbled. Yes, yeah, you get a final product that's actually, you know, polished, professional, watchable. Yeah, but it's not just about making it look pretty, right? No, not just about aesthetics. This design is what lets movie Gen. handle all these different video length, sizes, resolutions, you name it, which is something that a lot of those earlier AI video models, yeah, really struggled with. Ohh, totally. Those were limited. But we're talking about movie Gen. creating videos up to 16 seconds long and in crystal clear 1080P. That's huge like. Actual movie quality stuff. Yeah. It's incredible how far it's come. OK, but how does it get so good? Does it just watch like, a million movies and then figure it out? You say that like it's a bad thing. It's kind of like that. I mean, it learns from a crazy amount of data. I'm picturing like a robot with its eyes glued to a TV screen. Pretty much. Billions of images, millions of videos, all sorted by length and size. It's amazing. OK, that's a lot of data. And here's the kicker it learns from. Both still images. And videos. So it's like it learns the basics from pictures. Yeah. And then it figures out how to make those pictures move by watching videos. That's a great way to put it. Yeah, that's pretty clever. It's a really smart approach. First just images, then it starts learning from both at the same time. And as it gets better, the resolution ramps up too. So it's like this gradual process of. Yeah, OK, so it's learning the ropes, becoming a. I don't know, A decent editor, maybe. A promising film student, sure. But how does it go from film student to like? Spielberg, what's the next step? OK, so this is where it gets really cool. It goes through this intense training. Called supervised fine tuning. Supervised fine tuning. What did that? So instead of just any old footage, yeah, it learns from this curated selection of really high quality videos. So we're talking like the best of the best. Exactly. Award-winning stuff, beautifully shot. OK, so it's learning from the masters. Exactly. And that's how it develops its style, that knack for creating those stunning natural looking visuals and movements. So it's not just about quantity, it's about quality. Absolutely. OK, so we've got AI making these amazing. Videos from scratch while. But what about editing? Can it like make a montage? Editing is tricky, right? It's not just cutting and pasting clips together. There's an art to it. Exactly. It's about timing, pacing, knowing how to tell a story visually. And that's where movie Gen. edit comes in. Yeah. This one tackles the AI video editing problem, which I imagine is no easy task. It's really hard. Like, how do you even teach an AI to edit Well, yeah, because there's not like a rule book for that and there aren't these. Massive data sets of like perfectly labeled video edits to train an AI on So how do they get around that? So they've come up with this really cool three stage approach OK, hit me stage one start small single frame editing. So instead of tackling a whole movie, it's like learning the individual brush stroke right before you try to paint the Mona Lisa exactly OK makes sense. And then it gets really cool because it learns to handle edits across multiple frames OK so that's where you start getting into the. The transitions and stuff exactly, making it look smooth and seamless. That's the hard part. And they do this using 2 main techniques. I'm listening. Tell me more. First one, animated frame editing. Animated frame editing. Think of it like a flip book. OK, yeah, you're flipping through the pages and the image smoothly changes. That's what Movie Gen. Edit is learning to do. It's trained on pairs of original and edited frames, so it learns how to create that seamless transition between them. So it's like mastering the art of the flip book, but on steroids. Exactly. It's incredible. That's awesome. And then there's this other technique. It's called generative instruction, guided video segmentation. Generative What? Now that's a mouthful. It's a bit of a tongue twister, but. Imagine you're watching a video of, say, a bunch of kids playing, and you want to just edit the kid in the red shirt. OK. This technique is all about teaching the AI to do that so it can isolate, exactly isolate and edit specific parts of the video without messing with the rest of it. Precisely. OK, So it's like having this super precise editing tool. That's a great way to put it. That's pretty amazing. And then in the final stage of this whole process, movie Gen. edit becomes its own critic, judges its own work in a way, Yeah. It generates its own edited videos and then it filters out any that look off, any that have weird jumps or glitches. It's like, Nope, that it it's no good. Try again. Exactly. And by learning from its mistakes, it gets better and better at creating these seamless, natural looking edits. So it's like having an editor who's constantly learning and improving exactly, always striving for perfection. OK, so we've got the AI creating these awesome visuals, making this really complex edits, but can it get? Any more personal can like put ME in the movie get this. With movie Gen. personalized, you're not just the director, you're the star. Wait, seriously? It can take your image and make you the main character. No way. Yeah, it's crazy. So, like, it can capture how I look, Andy, how I move. You got it. It learns to match a reference image of your face to videos of you, even if those videos are from different places. So if I get a picture of me from like 10 years ago, it could still use that. It's wild, right? It analyzes your movements, your expressions and learns to generate. The ones that are consistent with you. So it's not just like, yeah, pasting my face onto someone else's body. Nope. It's actually learning how you move. That's incredible. But how does it even learn that? It's not like there's this whole Internet database of this is how host speakers name dances. You'd be surprised, but you're right. It's a fascinating process. It focuses on videos with one clear subject for starters, so it can really study how that person moves. Then it learns to match a reference image of a face to videos of that. Person even if they're from totally different clips OK so it's not just recognizing a face it's understanding yeah how that face moves within the context of like a whole body you got it and here's the key it trains on both paired data paired data that's where the image and video come from the same clip but also cross paired data where the image and video are from different clip so it's getting a really well-rounded education on how this person moves in all sorts of situations exactly that's how it. Learns to generate those really natural movements. So no more like creepy pasted on smiles. No more creepy smiles. It's getting really good at this stuff. That's amazing. So we've got AI generating these incredible visuals, making these really seamless edits and now it can even personalize videos with our own images and even our own movements. It's like having a digital twin. Yeah, like a digital twin who can star in our home movies. The possibilities are pretty mind blowing. It's insane. But we're missing 1 crucial. Piece of this whole puzzle. Ohh yeah, what's that? You know what's funny? We've been so caught up in the visuals, we haven't even talked about the sound. It's true. You kind of forget about it with all this AI video stuff, right? It's like you could have the most amazing video in the world. But if the sound is terrible, it's going to be unwatchable. Exactly. It's like watching a movie with the volume off. So how does Movie Jen handle that? So that's where Movie Gene audio comes in. Movie Gen. audio. All right, lay it on me. This is the AI that's gonna blow your mind sound wise. OK, so we're not talking about those cheesy robot sound effects from the 80s. No, no, we are way past that. OK, Movie Gen. audio is creating some seriously impressive soundscape of. So what can it do? Like, give me the rundown. Think everything from those. Subtle ambient noises like what kind of ambient noises? Wind in the trees, birds chirping. You know those little details right. The stuff that makes it feel real exactly to full blown musical scores. No way it can compose music too. Ohh yeah. And the quality is incredible we're talking 40 kilohertz high fidelity audio. Wow that's a professional grade stuff yeah OK so it can make all these sounds but. How does it know what sounds to create? Does it just like throw things at the wall and see what sticks? I mean, there's probably some of that going on behind the scenes, but it's actually really smart about it. It understands the difference between diegetic and non diegetic sound. Diegetic, OK, now you're just using big words. Diegetic sound is basically sound that makes sense within the world of the video. So if there's a dog barking in the video exactly, or a car horn honking, OK, I get it. What's non diegetic sound? That's the stuff that added for effect, like the soundtrack. Ohh, right. Like the scary music when something bad is about to happen. Exactly. Or the triumphant music when the hero wins. So it's like having a composer and a sound effects team built right into the AI. That's a great way to put it. Yeah. It analyzes the video and says, OK, this scene needs some spooky music or this scene needs some birds chirping. That's wild. But how does it know? Like, how does it learn that? So it's trained on this massive data set of audio. I'm talking speech, music, sound effects, all kinds of stuff. And it's learned to analyze the video content and understand the context. So if it sees a car chase, it knows to add the sound of screeching tires. You got it. It's really quite amazing. OK, so we've got AI that can generate crazy realistic videos, edit them, personalize them with their own faces and movements, Andy create amazing soundtracks all from scratch. What's the catch? Is it actually any good? That's the question, right? And honestly, it's. Really good, like how good are we talking? Movie Genvideo consistently outperforms other commercial AI video generator. Ohh wow. So it's like it's beating the competition. That's impressive. What about movie Gen. edit? Same story. People actually prefer its edits over other top of the line models. So it's not just like a cool tech demo. No, this is the real deal. It actually works. OK, last one, movie Gen. audio. How does that stack up? So not only does it sound amazing to the human ear. It also scores really high on those technical audio quality tests. So it's not just us the pros agree to. Exactly. That's incredible. I mean, we just spent, what, an hour talking about this AI movie? Jen, It's a lot to take in. It really is. And I gotta say, I'm blown away. This technology, it has the potential to completely change the way we make movies, home videos. It's a game changer for sure. It really is. Yeah. But it also raises a lot of questions, right? Like, how do we distinguish between reality and AI creations? What does it mean? For the future, the storytelling. These are good questions. These are things we're going to have to figure out as we go. It'll be interesting to see how it all unfolds. It will, but one thing's for sure, the future of filmmaking is here. And it is powered by AI, that's for sure. Well, that's our deep dive on movie Gen. for today. We hope you enjoyed it and until next time, keep exploring and keep asking those big questions. See you next time.