Introducing Veo 3.1 and new creative capabilities in the Gemini API

Google is releasing Veo 3.1 and Veo 3.1 Fast, an updated video generation model, in paid preview via the Gemini API. This version offers richer native audio, greater narrative control, and enhanced image-to-video capabilities. New features include guiding generation with reference images, extending existing Veo videos, and generating transitions between frames. Companies like Promise Studios, Latitude, and Whering are already using Veo 3.1 for various applications.

Alisa Fortin, Luis Cobo, Guillaume Vernade
3 min readbeginner
--
View Original

Overview

The article introduces Veo 3.1 and Veo 3.1 Fast, which enhance video generation capabilities through the Gemini API. Key improvements include richer audio, greater narrative control, and enhanced image-to-video capabilities, allowing developers to create more engaging content.

What You'll Learn

1

How to guide video generation using reference images

2

How to extend existing videos using Scene extension

3

How to create smooth transitions between images with First and last frame

Key Questions Answered

What new capabilities does Veo 3.1 offer for video generation?
Veo 3.1 introduces capabilities such as using reference images to guide video generation, extending existing videos, and creating transitions between images. These features enhance narrative control and maintain character consistency across scenes, making content creation more engaging.
How can developers use the Gemini API to generate videos?
Developers can use the Gemini API to generate videos by specifying prompts and reference images in their code. The API allows for advanced configurations, enabling richer audio and visual outputs, which enhances the overall quality of generated content.
What is the Scene extension feature in Veo 3.1?
The Scene extension feature allows users to create longer videos by generating new clips that connect to the final second of a previous video. This maintains visual continuity and is ideal for extending shots with background audio.
How does the First and last frame feature work in Veo 3.1?
The First and last frame feature enables users to create transitions between two different images by providing a starting and ending image. This feature generates smooth scenes accompanied by audio, enhancing the storytelling experience.

Technologies & Tools

API
Gemini API
Used for generating videos with enhanced capabilities in Veo 3.1.
AI/ML Model
Veo 3.1
The improved model for video generation that offers richer audio and visual outputs.

Key Actionable Insights

1
Leverage the new reference image capabilities to maintain character consistency across multiple scenes.
This is particularly useful in projects where visual continuity is crucial, such as in animation or storytelling, ensuring that characters remain recognizable throughout the video.
2
Utilize the Scene extension feature to create longer narratives without losing coherence.
By connecting new clips to existing footage, you can enhance storytelling and keep viewers engaged, which is essential for content creators looking to produce more in-depth videos.
3
Experiment with the First and last frame feature to enhance transitions in your videos.
This can significantly improve the flow of your content, making it more professional and polished, which is vital in competitive media environments.

Common Pitfalls

1
Failing to provide adequate reference images can lead to inconsistent character representation in generated videos.
It's crucial to select clear and relevant images to guide the generation process effectively. Without proper references, the output may not align with the intended style or narrative.

Related Concepts

Video Generation Techniques
AI In Media Production
Generative Storytelling