Automating On-Brand Marketing Image Generation with Amazon Nova

Overview

The presentation discusses a pipeline for automating the generation of on-brand marketing images using Amazon's Generative AI (GENAI) capabilities, specifically the Amazon Nova model. The key aspects covered include:

1. Challenges with current image generation approaches for marketing
2. A multi-stage pipeline to address these challenges
3. Detailed breakdown of each pipeline component
4. Performance evaluation and comparison to out-of-the-box image generation
5. Opportunities for further improvements and adaptations

Challenges with Current Image Generation Approaches

• Manual process for creating marketing images, requiring significant time and effort for fine-tuning
• Lack of brand safety and consistency with current image generation models
• Difficulty in controlling specific product details, textures, and visual elements
• Non-deterministic nature of image generation leading to inconsistent outputs

Automated Pipeline for On-Brand Marketing Image Generation

1. Prompt Decomposition and Asset Retrieval

• Extracting key entities (product, background, theme) from the user's input prompt using a lightweight language model
• Leveraging vector search to retrieve relevant assets (products, backgrounds) from the company's media library

2. Optimized Prompt Generation

• Automating the creation of prompts that leverage best practices for image generation models
• Ensuring captions follow a specific structure and length to improve output quality

3. Image Composition

• Determining the optimal relative size, position, and rotation of the product within the scene using a grid-based approach
• Utilizing Amazon Nova's "out-painting" capabilities to generate the final image composition

4. Intelligent Filtering

• Employing Amazon Nova's multimodal capabilities to evaluate generated images against brand guidelines and quality criteria
• Applying a flexible, weighted approach to filtering out low-quality or non-compliant images

5. Final Ranking

• Leveraging pre-trained aesthetic scoring models (e.g., Aesthetic Model, Image Reward, HPSV2) to rank the filtered images
• Selecting the top-ranked images for final review by human experts

Performance Evaluation and Comparison

• Quantitative metrics: MSSIM and Dinov2 to measure object similarity between original and generated images
• Qualitative evaluation: 70% user preference for the pipeline-generated images compared to out-of-the-box model outputs

Opportunities for Improvement and Adaptation

• Incorporating feedback loops to iteratively refine the pipeline based on human expert feedback
• Leveraging fine-tuning to adapt the pipeline to specific brand guidelines and requirements
• Staying up-to-date with the latest advancements in generative AI models and techniques

Key Takeaways

• Automating the creation of on-brand marketing images can significantly improve efficiency and consistency
• A multi-stage pipeline approach, leveraging various GENAI capabilities, can address the challenges of current image generation methods
• Continuous adaptation and integration of the latest GENAI advancements are crucial for maintaining a competitive edge
• The presented pipeline serves as a starting point for companies to build and customize their own automated image generation workflows