Practical application of generative AI chatbots in the public sector (WPS206)

Embracing the Future of Public Sector Services with Generative AI

Challenges Faced by Public Sector Agencies

  • Budgets and Funding: Public sector agencies often face budget constraints and struggles to secure funding for modernization efforts.
  • Skills and Legacy Systems: Agencies lack the technical expertise to convert their systems to a more modern state, and they have to deal with a lot of legacy and antiquated systems.
  • Regulatory Compliance, Security, and Privacy: Public sector agencies have to navigate a complex web of regulations, security, and privacy concerns that can be hurdles to modernizing their systems.
  • Data Complexity and Governance: Most public sector agencies have their data spread across multiple silos and islands of technology, making it difficult to access, govern, and share data effectively.

AWS Responsible AI Framework

To address these challenges, AWS has developed the Responsible AI Framework, which aims to deliver AI and generative AI services in a manner that is fair, transparent, and secure. The framework consists of eight key elements:

  1. Fairness
  2. Explainability
  3. Privacy and Security
  4. Safety
  5. Controllability
  6. Veracity and Robustness
  7. Governance
  8. Transparency

Generative AI Services at AWS

  • Amazon Bedrock: A managed service that provides access to multiple large language models through a simple API, with built-in guardrails to handle issues like hallucination, harmful data, and inaccuracy.
  • SageMaker Clarify: Helps identify, detect, and mitigate biases in machine learning models.
  • SageMaker Model Monitoring: Alerts you if a model is providing information that may not be in line with expectations.
  • SageMaker Ground Truth: Allows for human intervention in the event of anomalies.

The U.S. Department of State's Journey

  • The Department of State faced the challenge of making sense of their massive 20-volume, 10,000+ page Foreign Affairs Manual and Handbooks.
  • They started with basic Python scripts and phrase-catching, but quickly realized this was more of a language problem.
  • They experimented with a Mechanical Turk-based approach, but found it to be labor-intensive and not scalable.
  • Ultimately, they turned to a Retrieval Augmented Generation (RAG) approach, leveraging Amazon Bedrock and its guardrails to build a conversational interface that could answer complex questions in seconds.
  • This journey involved trial and error, close collaboration with customers, and a focus on ensuring explainability, transparency, and control over the system's outputs.

Architectural Patterns for Generative AI Chatbots

  1. Fully Managed RAG using Amazon Bedrock Knowledge Bases:

    • Ingests data sources into Amazon Bedrock Knowledge Bases
    • Utilizes Amazon Bedrock Guardrails to address responsible AI concerns
    • Chatbot application securely connects to Amazon Bedrock Knowledge Bases and Large Language Model to generate responses
  2. Building Your Own RAG Experience:

    • Ingests data sources into your own custom ingestion pipeline
    • Converts data into a backdoor representation and stores it in a database
    • Chatbot application performs semantic search on the backdoor database and uses a Large Language Model to generate responses

Service Catalog Demo

The demo showcased a chatbot application that leverages Amazon Bedrock Knowledge Bases and a Large Language Model to provide interactive, conversational responses to user queries about employee benefits, travel, and expense policies. This demonstrates how generative AI can significantly improve the customer experience by providing instant, accurate, and contextual information without the need for users to navigate complex systems and documentation.

The key takeaways from this detailed presentation are:

  • Public sector agencies face various challenges in modernizing their systems and delivering on the promise of AI and generative AI.
  • AWS has developed the Responsible AI Framework to address these challenges and ensure fair, transparent, and secure delivery of AI services.
  • The U.S. Department of State's journey showcases the practical application of generative AI, the architectural decisions, and the lessons learned.
  • AWS provides various architectural patterns, including fully managed and custom-built RAG experiences, to help organizations build effective generative AI chatbots.
  • The service catalog demo demonstrates how generative AI can significantly improve the customer experience by providing instant, accurate, and contextual information.

Your Digital Journey deserves a great story.

Build one with us.

Cookies Icon

These cookies are used to collect information about how you interact with this website and allow us to remember you. We use this information to improve and customize your browsing experience, as well as for analytics.

If you decline, your information won’t be tracked when you visit this website. A single cookie will be used in your browser to remember your preference.

Talk to us