How an analytics engineer uses AWS in a financial company (DEV212)

Summary of the Video Transcription

What is Analytics Engineering?

• Analytics Engineering is a new data career that bridges the gap between data production (software engineers, data engineers) and data consumption (data scientists, data analysts).
• Analytics Engineers help companies extract value from data by organizing and making data more accessible for other data roles.

Analytics Engineering Skill Set

Hard Skills:

• Big Data
• Programming Languages (SQL, Python, Spark)
• Cloud Computing (AWS Services)
• Statistics
• Data Visualization and Storytelling
• Business Knowledge

Soft Skills:

• Communication
• Conflict Solving
• Creativity
• Critical Thinking

Analytics Engineers in a Real-World Financial Company

• The company is one of the largest banks in Brazil, a strong AWS partner, and has adopted a data mesh architecture.
• Analytics Engineers sit between data engineers and data scientists, helping to:
  • Understand business problems and translate them into technical solutions.
  • Prepare high-quality data for data science teams to build better machine learning models.
  • Facilitate the deployment of features and models into production.

Technical Example: Specializing Data with Analytics Engineers

• Analytics Engineers can build solutions using AWS services (e.g., AWS Step Functions, AWS Glue, AWS EventBridge) to automate data processing, measure data quality, and make specialized data available to the organization.

The Future of Analytics Engineering

• The rise of new AI services (e.g., Amazon Comprehend, Amazon Bedrock) will likely lead to Analytics Engineers using AI tools to impact business with data.
• Analytics Engineering may become one of the hottest data careers in the coming years as companies recognize the need to bridge the gap between data production and consumption.