What I learned from making my first 100% GenAI-made music video

It’s not as easy as you think.

If you were thinking that making a GenAI music video would be like waving a magic wand, I was right there with you. Spoiler: it’s not.

Granted this was my first time ever using these tools, and as such I had a lot of figuring out to do–the whole process took me about a week, working after hours and spending an entire weekend on this—probably around 20-30 hours in total. (Or maybe I’m just slow as a snAIL?)

But I learned a ton. And honestly, it was a lot of fun exercising my creativity, even though I was infuriated half the time.

Here’s what surprised me the most and what I took away from the experience:

1. AI Isn’t Going to Take Away Jobs

Aside from having to iterate over and over again to get the output I wanted, I quickly realized how much I lacked in the skills and experience that a professional producer, musician, or cinematographer would bring to the table.

"AI is an enablement tool, not a replacement tool."

For instance, I didn’t even know how to properly storyboard to make my narrative coherent. I had no clue whether my lighting or shots were cinematic, or if my framing was any good at all. I just ended up leaving it up to the AI gods.

When it came to the music, I had even less control. I didn’t know how to refine ChatGPT’s lyrics to make them cooler or more profound. Heck, I couldn’t even come up with the right prompts to create the melodies I vaguely played back in my head. Even if I had known, I wouldn’t have had the skills to bring it all together the way. That takes real knowledge and mastery.

It dawned on me that while GenAI does make it easier for people without experience to create (which is great, and I’ll touch on this next), professionals are still the ones who can truly harness its power. AI is an enablement tool, not a replacement tool.

2. AI Will Explode Creation

Never in a million years would I have imagined that I could produce and create a music video (including the song!) 100% by myself—well, with GenAI tools, of course.

"GenAI has the potential to help people share their “inside” creativity with the world."

I’ve had countless ideas over the years for stories, narratives, videos, games, books… you name it. But creating any one of them would have required so much time, resources, and mastery of a specific domain. GenAI tools change all of that.

Like me, I believe millions—if not billions—of people have creativity and stories stored up inside their noggin’s, just waiting to be shared. GenAI empowers these stories to come into being and be shared.

Just like social media enabled people to share their “outside” lives, GenAI has the potential to help people share their “inside” creativity with the world.

3. Prompting is a Skill in Itself

The biggest skill I picked up during this whole process was prompting (with patience coming in as a close second).

There’s definitely an art to the madness, and reading how-to guides and watching YouTube videos really helped me refine how to generate what I had envisioned. There are little keywords and “tricks” you can use to adjust how the model interprets your input. Sometimes, I think lawyers would truly excel at GenAI creations because they’re so careful with their words 😂

Patience, though, is much needed. The models rarely get it right on the first try. Or the second. Or the third. Often times even the fourth or fifth. Newer models may improve on this, but I doubt by much. The issue is that language is inherently ambiguous, and I’ve come to realize that most of the time, it’s actually my own fault if the output doesn’t match my vision. Prompting is definitely a skill that needs to be honed—and at a tool-by-tool level at that.

All in all, it’s been a blast, and I’m excited for my next video—any ideas?

Here is the video if you would like to see it!

Here are the tools I used.

Lyrics: ChatGPT

Music: SunoAI

Image: MidJourney

Video: RunwayML

Editing: Capcut (not AI)

Blood sweat and tears: Arkar