Real-world applications of Amazon Q in data science and development (AIM243)

Key Takeaways

Slalom's Experience with Amazon Q

• Slalom, a software engineering company, has been rolling out generative AI applications, including Amazon Q, across various phases of the software development lifecycle.
• They conducted an internal A/B test with four teams of developers, two using Amazon Q and two not. The teams with access to Amazon Q were 20-50% faster in completing their tasks, with junior developers seeing the greatest increase in velocity.
• Developers also reported higher levels of satisfaction and engagement when using the tool, as it allowed them to spend less time on mundane tasks and more time in the "flow" of interesting work.

How Slalom is Leveraging Amazon Q

• Slalom has integrated Amazon Q across various interfaces, including the IDE, AWS console, documentation, and chat interfaces like Slack and Teams, allowing developers to get assistance wherever they are without losing context.
• The tool has been beneficial in various phases of the software development lifecycle, such as:
  • Learning about new technologies
  • Planning features and generating roadmaps
  • Suggesting and refining code
  • Reviewing and refactoring code
  • Generating unit tests and finding security vulnerabilities
  • Maintaining and modernizing codebases

Productivity and Quality Improvements

• Slalom has seen productivity increases of around 22% when rolling out Amazon Q to customers.
• The tool also helps improve code quality by generating tests and documentation, leading to better test coverage and easier maintenance over time.

Challenges and Recommendations

• The most significant challenge is teaching developers how to effectively interact with the large language model through prompt engineering.
• Slalom recommends:
  • Measuring and tracking the desired benefits, such as speed and quality
  • Targeting specific pain points, such as maintenance, modernization, and testing
  • Involving the developer community in the feedback and adoption process
  • Leveraging senior developers first to set best practices and guard rails for junior developers

Leveraging Amazon Q for a Data Science Platform

• Slalom used Amazon Q to help build a data science platform for a large agricultural company, allowing API and infrastructure engineers to flexibly contribute to various parts of the platform.
• The platform includes a multilingual SDK that abstracts away complex processes for data scientists, allowing them to focus on model development.
• Amazon Q helped the team autogenerate documentation, examples, and user guides, improving the overall user experience of the platform.

Conclusion

Slalom's experience demonstrates the significant productivity and quality benefits that can be achieved by integrating generative AI tools like Amazon Q throughout the software development lifecycle. However, successful adoption requires addressing challenges around prompt engineering and involving the developer community in the process.