Here is a detailed summary of the video transcription in Markdown format:
Introduction
- Diego, the presenter, is responsible for the AI strategy between AWS and Intel.
- The session is about "Unleashing the Power of AI from Edge to the Cloud with Intel Technology".
- The agenda covers how Intel provides choice when it comes to AI, with perspectives from various partners and customers.
Intel's AI Strategy
- AI is transforming how we work and live every day, with AI being omnipresent in our daily activities and enterprise workloads.
- Intel's AI strategy involves a bottom-up approach, first developing the hardware and then the software.
- Key market data points:
- 58% of CEOs will actively invest in AI, and over $300 billion will be spent on AI by 2026.
- More than 75% of enterprise data will be created and processed outside the data center and cloud by 2025.
Intel and AWS Collaboration
- Intel and AWS have a 18-year-old relationship, with over 400 instances powering the AWS regions and services.
- Intel's strategy is an open-source software toolkit approach, such as PyTorch extensions and OpenVINO, to optimize AI performance on Intel architecture.
- Intel provides a range of solutions, from scalable Xeon CPUs for traditional machine learning and inference, to Habana Gaudi GPUs for large-scale training on AWS.
Accenture's Generative AI Platform
- Accenture has a decade-long collaboration with Intel, focusing on co-innovation, enablement, and go-to-market.
- Accenture has built a generative AI platform with reference architectures, blueprints, and accelerators to help clients leverage open-source and commercially available foundation models.
- The platform utilizes Intel compute options, such as CPUs, to run small language models efficiently and cost-effectively.
- A case study showcases how the platform was used to analyze network switch test data using graph databases and generative AI.
AI Inference on Intel-Powered AWS Instances
- There is a myth that AI workloads require GPUs, but Intel CPUs can often outperform GPUs in certain workloads.
- Intel and AWS have introduced AMX instructions in Intel CPUs, enabling AI inference on a wide range of EC2 and SageMaker instances.
Storm Reply's AI-Powered DevOps Solution
- Storm Reply, a global technology consultancy, has developed a generative AI solution called IDIA to enhance their DevOps process.
- IDIA is integrated into the CI/CD pipeline, capturing errors and code, and using a large language model to provide suggestions for remediation, resulting in over 30% reduction in operational costs.
- The solution leverages Intel-powered instances on AWS to achieve cost-effective and performant AI inferencing.
AI at the Edge with Manchester Airport Group
- Manchester Airport Group (MAG) wanted to monitor aircraft turnaround processes to understand and predict delays.
- MAG worked with AWS Professional Services and Intel to build a computer vision-based solution at the edge using Intel's Panorama and Carbon devices.
- The edge-based approach allowed them to leverage existing CCTV infrastructure and security/access controls, while providing high-performance and cost-effective AI inferencing.
- The solution was deployed in 24 weeks, going from AI amateurs to experts, and is now being expanded to over 300 stands across MAG's airports.
Large Language Models with TII
- The Technology Innovation Institute (TII) is an R&D center in Abu Dhabi, UAE, working on various AI domains, including generative AI and large language models.
- TII has developed the Falcon series of large language models, with a focus on optimizing the models to run efficiently on CPU-based infrastructure, in collaboration with Intel and AWS.
- TII has achieved significant performance improvements in terms of tokens per second when running their large language models on Intel-powered CPU instances, making these models more accessible to end-users.
Key Takeaways
- Intel's 3rd Gen Xeon Scalable processors with built-in AI accelerators (AMX) are a great fit for large language model inference on AWS.
- Intel provides open-source software toolkits, such as OpenVINO, to optimize AI performance on Intel architecture in the cloud.
- Intel also provides open-source tools to optimize and deploy AI solutions at the edge, enabling edge-based AI use cases like the one showcased by Manchester Airport Group.