Integrating applications with Amazon EventBridge Pipes (API218)
Event Bridge Pipes: Gluing AWS Services Together without Glue Code
Introduction
The presenter introduces the concept of "glue code" and asks the audience to boo loudly whenever it is mentioned.
The presenter explains that glue code is a disgusting and vile thing that wakes you up at 2 AM when things go wrong.
The presenter introduces himself as a senior technical product manager for the Event Bridge team, with a focus on pipes.
AWS Services and Composing Applications
AWS offers more than 200 different services that can be used as building blocks to compose a complete application.
Using these services allows you to innovate faster by focusing on what differentiates your application, rather than worrying about undifferentiated heavy lifting like provisioning infrastructure or running data centers.
The services are incredibly powerful and enable you to use the right tool for the right job.
Assembling these building blocks together results in a solution that is scalable, reliable, and resilient, and can evolve as your needs change.
Glue Code and its Challenges
Connecting these building blocks together may seem easy, like just drawing a line between the boxes.
However, the code required to transfer data between these services, known as "integration code" or "glue code", can be challenging and time-consuming to write and maintain.
Glue code requires understanding the nuances of each technology, such as Kinesis data streams, SQS, and more. This includes handling things like error handling, retries, authentication, rate control, performance testing, and deployment.
Glue code can also be a source of bugs that can cause production issues and wake you up in the middle of the night.
Event Bridge Pipes
Event Bridge Pipes is a service that connects AWS services together without the need for glue code.
On the left-hand side, you have a variety of event sources, and on the right-hand side, you have a variety of event targets.
In the middle, you have optional but powerful features like filtering, enrichment, and transformation, which allow you to customize how events are processed and delivered.
The presenter explains that the demo will show how to use Event Bridge Pipes to move data from a large Kinesis stream to a smaller Kinesis stream, and then from SQS to a Step Function, with enrichment from Zendesk.
Demo 1: Kinesis to Kinesis
The presenter sets up a scenario where there is a large Kinesis stream containing click stream data, and the goal is to filter out only the events related to the sales section of the website and send them to a smaller Kinesis stream.
The presenter demonstrates creating an Event Bridge Pipe to accomplish this task, showing how to set up the filtering step to only include the relevant events.
Due to a technical issue, the presenter is unable to fully complete the demo, but they explain the expected outcome.
Demo 2: SQS to Step Function with Enrichment
In the second scenario, the presenter is working in the support organization, where there is a need to process support tickets.
The presenter demonstrates creating an Event Bridge Pipe that takes events from an SQS queue, enriches the events with additional data from the Zendesk API, and then sends the enriched events to a Step Function for processing.
The presenter explains the enrichment step, which uses an API destination to fetch the full ticket information based on the ticket ID in the SQS event.
The presenter then shows the Step Function receiving the enriched events, demonstrating the successful completion of the pipe.
Conclusion
The presenter summarizes that Event Bridge Pipes allows you to connect AWS services together without the need for glue code, which can be time-consuming, error-prone, and a source of production issues.
The presenter encourages the audience to explore Event Bridge Pipes and take advantage of its powerful features like filtering, enrichment, and transformation to simplify the process of building complex applications.
These cookies are used to collect information about how you interact with this website and allow us to remember you. We use this information to improve and customize your browsing experience, as well as for analytics.
If you decline, your information won’t be tracked when you visit this website. A single cookie will be used in your browser to remember your preference.