Utiliser RAG pour la génération vidéo avec Amazon Bedrock et Amazon Nova Reel

Generating high-quality custom videos remains a significant challenge, because video generation models are limited to their pre-trained knowledge. This limitation affects industries such as advertising, media production, education, and gaming, where customization and control of video generation is essential. To address this, we developed a Video Retrieval Augmented Generation (VRAG) multimodal pipeline that transforms structured text into bespoke videos using a library of images as reference. Using Amazon Bedrock , Amazon Nova Reel , the Amazon OpenSearch Service vector engine , and Amazon Simple Storage Service (Amazon S3), the solution seamlessly integrates image retrieval, prompt-based video generation, and batch processing into a single automated workflow. Users provide an object of interest, and the solution retrieves the most relevant image from an indexed dataset. They then define an action prompt (for example, “Camera rotates clockwise”), which is combined with the retrieved image to generate the video. Structured prompts from text files allow multiple videos to be generated in one execution, creating a scalable, reusable foundation for AI-assisted media generation. In this post, we explore our approach to video generation through VRAG, transforming natural language text prompts and images into grounded, high-quality videos. Through this fully automated solution, you can generate realistic, AI-powered video sequences from structured text and image inputs, streamlining the video creation process. Solution overview Our solution is designed to take a structured text prompt, retrieve the most relevant image, and use Amazon Nova Reel for video generation. This solution integrates multiple components into a seamless workflow: Image retrieval and processing – Users provide an object of interest (for example, “blue sky”) and the solution queries the OpenSearch vector engine to retrieve the most relevant image from an indexed dataset, which contains pre-indexed images and descriptions. The most relevant image is retrieved from an S3 bucket. Prompt-based video generation – Users define an action prompt (for example, “Camera pans down”), which is combined with the retrieved image to generate a video using Amazon Nova Reel. Batch processing for multiple prompts – The solution reads a list of text templates from prompts.txt , which contain placeholders to enable batch processing of multiple video generation requests with structured variations: – Dynamically replaced with the queried object. – Dynamically replaced with the camera movement or scene action. Monitoring and storage – The video generation is asynchronous, so the solution monitors the job status. When it’s complete, the video is stored in an S3 bucket and automatically downloaded for preview. The generated videos are displayed in the notebook, with the corresponding prompt shown as a caption. The following diagram illustrates the solution architecture. The following diagram illustrates the end-to-end workflow using a Jupyter notebook. This solution can serve the following use cases: Educational videos – Automatically creating instructional videos by pulling relevant images from a subject matter knowledge base Marketing videos – Creating targeted video ads by pulling images that align with specific demographics or product features Personalized content – Tailoring video content to individual users by retrieving images based on their specific interests In the following sections, we break down each component, how it works, and how you can customize it for your own AI-driven video workflows. Example input In this section, we demonstrate the video generation capabilities of Amazon Nova Reel through two distinct input methods: text-only and text and image inputs. These examples illustrate how video generation can be further customized by incorporating input images, in this scenario for advertising. For our example, a travel agency wants to create an advertisement featuring a beautiful beach scene from a specific location and panning to a kayak to entice potential vacation bookings. We compare the results of using a text-only input approach vs. VRAG with a static image to achieve this goal. Text-only input For the text-only example, we use the input “Very slow pan down from blue sky to a colorful kayak floating on turquoise water.” We get the following result. Text and image input Using the same text prompt, the travel agency can now use a specific shot they took at their location. For this example, we use the following image. Travel agency can now add content into their existing shot using VRAG. They use the same prompt: “Very slow pan down from blue sky to a colorful kayak floating on turquoise water.” This generates the following video. Prerequisites Before you deploy this solution, make sure the following prerequisites are in place: Access to a valid AWS account Familiarity with Amazon SageMaker notebook instances Deploy