TalksAWS re:Invent 2025 - Simplifying orchestrations with JSONata and AWS Step Functions (API323)
AWS re:Invent 2025 - Simplifying orchestrations with JSONata and AWS Step Functions (API323)
Simplifying Orchestrations with JSONata and AWS Step Functions
Introduction to AWS Step Functions
AWS Step Functions is a low-code visual workflow service that provides orchestration for AWS services, third-party APIs, and custom application APIs.
It supports common workflow patterns like parallel processing, iterative workflows, human approvals, and long-running workflows.
Step Functions simplifies automation and orchestration use cases such as ETL, security/IT automation, microservices orchestration, and agent-based workflows.
Limitations of Traditional Step Functions Approach
Proliferation of Lambda functions to handle data transformation and integration
Limited data transformation capabilities within Step Functions
Maintenance challenges with multiple Lambda functions
Increased latency and cost from hopping between Step Functions and Lambda
Introducing JSONata
JSONata is a lightweight query and transformation language for JSON data.
It provides powerful capabilities for navigating JSON objects and arrays, flattening data, and performing numeric/string operations.
JSONata has native integration with AWS Step Functions, allowing complex transformations to be handled directly within the workflow.
Simplified Step Functions Approach using JSONata
Removing Lambda Functions: Use direct HTTPS API calls and JSONata to transform payloads, eliminating the need for separate Lambda functions.
Optimized Service Integrations: Leverage JSONata to perform complex data aggregation and analysis within Step Functions, rather than using Lambda.
Direct SDK Calls: Make direct SDK calls to services like DynamoDB, using JSONata to format the payload correctly, without requiring additional Lambda functions.
Benefits of the Simplified Approach
Reduced maintenance overhead: No need to manage and maintain separate Lambda functions.
Lower latency and cost: Eliminating the need to hop between Step Functions and Lambda.
Improved workflow readability and maintainability: Complex transformations handled directly within the Step Functions workflow.
Agent-based Workflows with Step Functions and Generative AI
Demonstrated a generative AI-powered agent-based workflow using Step Functions.
The workflow uses multiple tools to make decisions and adapt content for different social media platforms.
Leveraged the Step Functions "Supervisor" pattern, where Lambda functions are used in a limited capacity to orchestrate the overall workflow.
Key Takeaways
JSONata can significantly simplify orchestrations by eliminating the need for separate Lambda functions for data transformation.
Optimized service integrations and direct SDK calls within Step Functions can reduce latency, cost, and maintenance overhead.
Step Functions can be used to build agent-based workflows that leverage generative AI capabilities, with Lambda functions playing a more limited "Supervisor" role.
The simplified approach to Step Functions orchestrations can be applied to a wide range of automation and integration use cases, from ETL to security/IT workflows to microservices orchestration.
These cookies are used to collect information about how you interact with this website and allow us to remember you. We use this information to improve and customize your browsing experience, as well as for analytics.
If you decline, your information won’t be tracked when you visit this website. A single cookie will be used in your browser to remember your preference.
