S&P Global: Scaling data insights with fine-tuned foundational models (FSI313)

Leveraging Fine-Tuned Foundation Models to Implement Scalable AI Solutions at S&P Global

Introduction

  • S&P Global is a leading provider of credit ratings and financial intelligence, dealing with large volumes of data and complex business problems.
  • The company faced challenges in assessing credit risk for small and medium enterprises (SMEs) that lack publicly available financial information.
  • To address this, S&P Global decided to leverage fine-tuned foundation models and a distributed architecture on AWS to build a scalable AI solution.

Business Problem and Challenges

  • S&P Global's credit rating business is over 150 years old, with 1 million outstanding credit ratings on $46 trillion of debt.
  • However, for private companies and SMEs, the lack of publicly available financial information made it difficult to assess credit risk.
  • Key challenges included:
    • Need for "out-of-the-box" credit assessments for large portfolios of SMEs.
    • Lack of timely financial information, as private company data can be months or years old.
    • Covering a large target of over 60-70 million companies without financial information.

Approach to Building a Scalable AI Solution

  1. Scraping Textual Footprint from the Web:

    • Created a domain graph to map child URLs related to a company and identify relevant pages.
    • Utilized a heterogeneous graph structure to classify HTML content and extract relevant risk signals.
    • Leveraged an orchestration pipeline using Amazon Managed Airflow, Redis, and Amazon EKS to scale the web scraping process.
  2. Leveraging Fine-Tuned Foundation Models:

    • Explored parameter-efficient fine-tuning techniques, such as LoRA, to update a small portion of the model parameters while preserving the knowledge in the pre-trained model.
    • Employed a self-grading approach to build a quality training dataset for fine-tuning the models.
    • Fine-tuned the models using Amazon SageMaker, focusing on prompt engineering and hyperparameter tuning.
  3. Deploying and Operating the Solution at Scale:

    • Converted the fine-tuned models to run on CPU for cost-effective inference, leveraging the Llama C++ library.
    • Deployed the classification and extraction tasks on Amazon EKS, with worker pods listening to a Redis-based queue.
    • Achieved high scalability, running up to 400 concurrent pods to handle 6 million inferences per week.

Key Learnings and Insights

  1. Start Simple and Iterate:

    • Begin with a smaller foundation model and fine-tune a small subset of the parameters.
    • Gradually increase complexity as needed, considering factors like overfitting and catastrophic forgetting.
  2. Leverage Parameter-Efficient Fine-Tuning:

    • Techniques like LoRA can significantly reduce the fine-tuning cost and complexity.
    • Allows maintaining a single base model and updating small adapter modules for different tasks.
  3. Optimize for Cost-Effective Inference:

    • Evaluate the trade-off between GPU and CPU instances for inference, considering the overall cost and performance requirements.
    • Utilize tools like Llama C++ to efficiently convert models from GPU to CPU.
  4. Adopt a Self-Grading Approach for Training Data:

    • When dealing with limited resources, a self-grading approach can help build a quality training dataset.
    • Break down complex tasks into smaller pieces and leverage the knowledge in the pre-trained model.
  5. Embrace a Distributed Architecture:

    • Leverage managed services like Amazon Managed Airflow, Redis, and Amazon EKS to build a scalable and resilient infrastructure.
    • Separate heavy and light scraping tasks, and scale worker pods to handle the processing load.

Conclusion

S&P Global's journey demonstrates the power of fine-tuned foundation models and a distributed architecture in building scalable AI solutions to address complex business problems. By leveraging AWS services and adopting best practices, the company was able to create a highly efficient and cost-effective system that handles millions of inferences per week.

Your Digital Journey deserves a great story.

Build one with us.

Cookies Icon

These cookies are used to collect information about how you interact with this website and allow us to remember you. We use this information to improve and customize your browsing experience, as well as for analytics.

If you decline, your information won’t be tracked when you visit this website. A single cookie will be used in your browser to remember your preference.

Talk to us