Big news! Adobe released it’s beta Generative Video model to the public!!
Anyone can sign up and use the great and terrible power of AI video generation.
Read on to learn about its exciting new features, its costs, how to use it, and much more!
What are the AI Video New Features?
Adobe announced a lot of new Firefly Features today, but the video-specific ones are:
There were other cool non-video features announced today such as translate audio and scene to image, but I’ll do separate articles for those. Stay tuned!
Midjourney & Firefly Tested Keywords
Monthly Inspo Right in Your Inbox
Sign-up and get style-specific keywords directly to your inbox every month.
How Much Does it Cost?
Adobe now uses a credit system for all its AI products. Usage may vary, but Adobe estimates generating a video—with text or images—appears to cost 100 credits. Translating a video costs roughly 330 credits per minute of translation.
If you have a Creative Cloud plan, you technically get a monthly allotment of hours but these credits do not apply to the new generative video features. You can test prompting for a video or two, but after that you will be hit with a paywall.
Their starter plan costs $12.99 CAD/month and gets you 2,000 credits. Adobe estimates you’ll be able to generate 20 five second videos or up translate up to 6 minutes video with that plan.
The Pro plan costs $38.99 CAD/month and gets you 7,000 credits. They estimate this will get you 70 five second videos or up translate up to 23 minutes video.
Prices will likely vary depending on your area.
How to Use Text-to-Video?
While there are a handful of AI video features within Adobe apps such as Extend Video in Premier pro, as of right now you can only fully generate new videos on the Firefly web app.
Once you land on the Firefly homepage, click on the Text-to-Video card.
This will take you to the video generator UI.
Click in the text bar and enter your prompt.
Refine your prompt with the toolbar on the left.
Then press generate.
How to Use Firefly Video Generator’s Toolbar?
Similar to Firefly’s Image generator, the video generator has a tool bar full of helpful settings.
Unlike the Image Generator toolbar, Firefly’s Video Generator Toolbar doesn’t focus on style, rather tries to give you video specific options such as:
Camera Motion
Camera Angle
Camera Size
You can still use the image generator styles as prompt inspiration.
Camera motion helpfully showcases the motion when you hover over it.
The video toolbar has areas for FPS and different video models that aren’t changeable right now. However, the fact that they are included hints that you will be able to adjust these settings in the future.
Lastly, they give you limited video aspect ratio options in the tool bar. You can essentially selected between the two most common video dimensions 16:9 and 9:16. Sounds like Adobe is looking to extend the options here soon.
What’s the Length of Videos Can You Generate?
As of the beta launch, you can generate a 5-second clip. This will likely change in the future as the model moves out of its beta.
However, if you need a longer clip, you can quickly hack the system by taking a still of your last frame, uploading it to Firefly’s Image-to-Video, and then generating the rest of your video.
How Long Does it Take Firefly to Generate a Video?
Adobe boasted a 90 second generating time, but warns that it might take a little longer these first few weeks of launch as everyone is excited to get using it.
You can track your video’s generation progress by looking at the loading icon on the video’s thumbnail.
Prompt Tips
I’ve only had a couple hours to mess around with this new product, so I am by no means a Firefly Video prompting expert. However, it appears most common Firefly prompting tips and tricks hold true:
- Shorter prompts ensure the AI doesn’t get confused
- No proprietary content (no artist names, no book franchises, no character names, ect.)
- If you have a specific vision, give a specific description. Anything not described in the prompt will be imagined by the AI.
The biggest difference I found between prompting in Firefly’s image model compared to its video model is the need to descrie the video’s motion. How is your subject moving? Does the camera zoom? Do you want the camera to move with your subject or is it static? Any special angles? It’s helpful to keep these questions in mind as your prompting the video model.
How Good is Adobe Firefly’s Text-to-Video Model?
It’s pretty good compared to other video models!! But keep in mind this field is relatively new, and AI videos in general have their flaws.
Take a look at this cat video. That is quite a realistic-looking cat, and at most points, the cat’s movement looks pretty convincing! However, it’s certainly not perfect. Its paw totally misses the ball, a weird extra leg goes in and out of existence throughout the video, and its back leg randomly becomes a soccer ball at one point.
However, as we saw with Generative Image AI’s, this technology will move insanely quickly. Just take a look at Midjourney! Its first model was only released 3 years ago, and look how much it’s improved! There’s been 8 models since then, and they are on the verge of releasing a new one!
v 1
v 2
v 3
v 4
v 5
v 5.1
v 5.2
v 6
v 6.1
Firefly Text-to-Video Use Cases
Firefly’s AI videos aren’t perfect yet, but they still have some awesome use cases you can start using right away!
B Roll
The most obvious use case for AI generators is b-roll! Instead of paying $20+ for a measly handful of 5 second clips that just anyone can have, you can pay $13 and generate b-roll perfectly styled to you!
B-roll doesn’t need to be the focus of the video, and is usually only shown for a couple seconds, so its perfectly suited for Firefly generated videos.
Animated Elements
Something Adobe’s heavily featured in is ads for Firefly Video model is that you can easily create branded animated elements you can layer on top of your other videos.
Quick proof of concept
Who cares about perfection when you’re just trying to create a proof of concept? Instead of picking an idea with just some storyboards, you can use AI to quickly generate your idea and bring it to life. Heck, with Firefly’s Image-to-Text, you can even upload your storyboard to the generator and create a full video version.
Art that Embraces the Imperfection
Art isn’t about perfection, and the quirks of AI-generated video can actually be leveraged to make something beautiful. This wasn’t generated in Firefly, but I couldn’t pass up this opportunity to show off the amazing work of James Gerde, an AI animator who leans into the imperfection of AI video. He creates dancing origami or organic figures that change and morph to the character’s movements. It creates beautiful videos that are hard to rip your eyes away from!
Inspiration
The classic use case for AI, you knew it had to be on this list somewhere! 😛
It’s a clique, I know, but it’s true! AI’s great, being able to iterate through ideas quickly and help generate inspiration for your next project. It’s like working with your own personal idea buddy!
Can Firefly Generate People in its Videos?
Yes, but to varying degrees of well.
It does a great job of this close up of a man smiling.
However, this long shot of a woman spinning in a lavender flower tunnel has an AI face that will haunt my nightmares! (shivers)
Or this big crowd of people walking accross the street mysteriously don’t have faces…
Similar to Midjourney’s first couple of versions, if you focus the AI on a close-up of one person, it can do a pretty decent job, but as soon as you zoom out or add multiple people, it struggles.
Can Firefly Text-to-Video Model Generate Text?
Yes, and relatively well too! I usually find it helpful to wrap the word you’d like depicted in quotes “” to make it ultra clear to the AI which word you’d like in your video.
Describing how you’d like the word to animate, on the other hand, was oddly difficult. See my example below where I attempt to make a word out leaves that “wiggle” as if in the wind and well… Firefly took that to mean that the whole canvas should wiggle, lol.
How to Use Image-to-Video?
It’s quite simple! Click on the image-to-video card on the Firefly homepage.
Then you can add an image as the first frame clicking on the upload fields right next to the prompting area.
Then add your prompt and press generate!
How to Delete An Uploaded Image?
Just hover your mouse over the image until you see a little “X” and then click it.
Can You Upload a Start and End Frame?
Yes! Since you can only generate up to 5 seconds at a time, you can actually use both a start and end frame as “keyframes” to animate a longer video using images.
It’s revolutionary!
How can you Make longer Videos?
You can use an AI Image Generator that has consistent character-generating abilities, such as Midjourney’s –cref, to generate your original character and then multiple frames of your character to use as references.
Here’s an example: I’ve started out with my original ninja character and then used Midjourney’s –cref to generate that same character in a punching stance (aka where I’d like my character to be after 5 seconds of video).
I then upload those two images as my starting and ending frames.
Then I write my prompt as usual, and then press generate.
Remember, 5 seconds is a lot longer than you think it is. Make sure your start and end frames have enough space between them to fill that time.
So far I have my ninja standing, then get into a punching stance. Next I’ll want him to kick! Using Midjourney’ –cref again, I’ll generate my next end frame of my ninja kicking.
Once I have the image, I’ll use my previous final end frame as my starting frame and upload the new kicking photo as my end frame. Then I’ll adjust my prompt to better describe the motion of what I’d like to happen between these two frames and then press generate.
Now I have two videos that can be merged together to create a longer video.
If you keep going using that process, you can generate as long of a video as you’d like!
Pro Hint: Be sure to double check that Firefly actually uses your end frame before generating your next clip. For example, check out my last end frame versus what Firefly ended my video with:
Firefly Image-to-Video Use Cases
Here as some specific usecases for Firefly’s Image-to-Video.
Animate your Logo
This is one of the use cases you can get using right away!
Looking to brainstorm logo animation ideas? Or even generate clips to use in your logo animation? Just add your logo as the starting frame, describe your vision, then sit back and watch the magic happen!
Animate Traditionally Un-Animatable Images
Traditionally, raster photos have been incredibly hard to animate. The emergence of image-to-video AIs has open up those possibilities! Like everything else about this video model, its certainly not perfect, but the fact that it’s as good as it is is mind blowing and gets the mind racing with future possibilities. Check out how I can animate this stock photo by uploading it as a starting frame.
Starting Frame
Extend a Video
Full discloser for this use case: if you have Creative Cloud, you’re probably going to be better to use Adobe’s Generative Extend function in Premier Pro than to extend a video in Firefly.
However, those that don’t have CC, this is a great way to extend you videos that are just not quite long enough. Simply grab the last frame of your video, upload it to Firefly’s Image-to-Video, enter your prompt, and watch your video magically get 5 seconds longer! 🤯
Can You Upload a Start and End Frame?
Yes! Since you can only generate up to 5 seconds at a time, you can actually use both a start and end frame as “keyframes” to animate a longer video using images.
It’s revolutionary!
How to Use Translate Video?
From the Firefly homepage, click on the “Translate video” card.
This will take you to a file upload area. Simply click and drag your video into the upload area.
Original: English
FF Translation: German
IMPORTANT
Keep in mind, this service costs you 5 credits/second, per language. So be cautious, this could add up quickly!
You also need to upload a video with a minimum of 5 seconds for the AI to do its magic, so one translation costs a minimum of 25 credits.
Can you Use these Features in Creative Cloud Apps?
No directly yet. While Premiere Pro has the cool new Generative Extend feature, generating video outright from a prompt appears to only be within the Firefly website right now.
Is Firefly Video Tools Commercially Safe?
Yes! All of Adobe’s AI products are commercially safe to use.
They do not train their models on any proprietary images or videos, so it’s actually the safest AI tool out there right now.
What FPS Can You Generate Video in?
You can only generate in 24 FPS as of right now. However, this is just the beta version, so the selection is likely to expand as the model gets more features.
What Dimensions Can You Generate the Video in?
720p with 16:9 AR, but you can only generate in 16:9 and 9:16 aspect ratios.
What File Formats Can You Export in?
As of right now, just MP4.
Does Firefly have a Character Limit in its Text-to-Video Prompts?
Yes, 175 words. This should be plenty enough to describe the video you need. Plus, I find the shorter the prompt the better, otherwise the AI can get confused.
