Video Production in the AI Era

This week’s newsletter was written by AppAgent Motion Designer, Polina Anisimova.

As a motion designer, I've been exploring the potential of using AI tools in my video production workflow to improve efficiency, boost creativity, and ensure consistent quality.

But it's crucial to distinguish advanced tools that streamline the process from those that may complicate it further. 

In this article, we will delve into two AI tools: Runway Gen-2 and LeiaPix, and evaluate their capabilities, advantages, and limitations for enhancing video production.

Runway Gen-2

Runway Gen-2 is an innovative AI tool that has caught the attention of the industry due to its potential to reshape the creative landscape. Let's examine whether this tool truly lives up to the hype and how it can empower idea-makers and designers with enhanced creative expression.

By uploading a video and selecting a style reference, we can generate personalized and stylized video results. ⚠️ But it’s important to note that credits are deducted with each creation.

Runway Gen-2 also offers another significant advancement: the synthesis of new videos through two different approaches. Firstly, it can apply the composition and style of an image or text prompt to an existing video, seamlessly transforming it (video-to-video). 

Alternatively, it can generate an entire video based solely on a text prompt (text-to-video). Although the length of the generated video is currently limited to 4 seconds, the possibilities are still impressive.

Let's take a look at how the project appears in action. We have the option to simply type a prompt or even upload a reference frame along with it. 

Below is an example of what I achieved using only the text prompt: "a boy eating ramen fast in a cyberpunk city bar, front view face, cinematic, neon lights."  Some glitches in the visuals were noticeable, such as wrong anatomy and unnatural movements, but I’m still impressed with the quick results.

Pros of Runway Gen-2:

• Usability: The latest update offers a user-friendly interface for a seamless experience.
• Speed: Runway Gen-2 delivers impressive results quickly.
• Adjustability: By experimenting with the settings, we can achieve impressive outcomes.

Cons of Runway Gen-2:

• Time Limitation: The maximum length restriction for generated videos is 4 seconds.
• Watermark: The presence of a watermark can only be eliminated by upgrading to the standard or pro plan.
• Visible Noise: The output may exhibit visible noise, affecting the overall quality of the results.
• Anatomical Issues: AI still struggles with generating the correct anatomy, especially fingers and eyes.
• Unpredictable Movements: We can preview just the style of our video, but we can’t preview camera movement or character movements.

A notable improvement with the latest update is the introduction of a preview option. Before applying changes and spending credits, we can now preview four different style options based on their text prompt. The preview process takes approximately 20-30 seconds, while the final video may take several minutes to generate.

LeiaPix

This tool focuses on enhancing still images by creating depth map animations. It enables motion designers to add dynamic elements to packshots, create subtle camera movements with tracked objects, and swiftly generate animations based on input images.

LeiaPix allows us to adjust both the animation and depth map, providing precise control over the movement according to specific needs.

The tool also offers different formats for downloading the animations.

Pros of LeiaPix:

• Time-Saving: LeiaPix swiftly creates animations within seconds, saving considerable time and effort.
• Depth Map Control: The tool generates a depth map that can be used in software like After Effects, offering precise control over the movement.

LeiaPix can be valuable in various practical use cases, such as creating 3D images for marketing campaigns, enhancing product images for e-commerce platforms, and generating visual content for social media. 

While the tool has some limitations on movement, its flexibility in controlling animation dynamics makes it a good example of AI used to enhance the video production process.

Have a look at more use cases and examples HERE.

Today's Insights 🤖

During a recent call with UBS analysts, Emad Mostaque, CEO of Stability AI, expressed his views on AI investments:

“I think this will be the biggest bubble of all time. I call it the ‘dot AI’ bubble, and it hasn’t even started yet.

I think this is real. I think that there aren’t many investable opportunities here, and you’ll see people moving from the best chip manufacturers to companies that are using this to impact their bottom line and their top line appropriately. And you will see the market punishing those that don’t use this” 

Mind-Blowing News 🤯

• Hell Energy is now officially the “world’s first energy drink powered by AI.”
• The use of AI, and the lack of regulations around it, are playing a huge role in the WAG and SAG-AFTRA strikes.
• Are AI matchmakers going to be the future of dating?

Let’s see what next week will bring, stay tuned (and subscribe)!