Healthcare Wants GenAI. But Is the Data Ready?

GenAI has rapidly become the centerpiece of healthcare innovation. But amid the excitement, one hard truth is emerging: the distance between aspiration and real, measurable impact is still uncomfortably wide.

Despite high investments and fast-paced experimentation, many GenAI initiatives stumble, delivering underwhelming results, generating irrelevant outputs, or, worse, producing costly errors. Complaints are becoming familiar refrains: “It’s hallucinating,” “It’s not working as expected,” “We built something and it failed.”

These aren’t just isolated setbacks. They’re symptoms of a deeper problem, one that’s often overlooked in the race to deploy models: data quality.

In this episode of AntStackTV, Jeevan and Prashanth unpack what’s really going wrong beneath Healthcare data, and why the foundation matters more than the flash.

How Good is your Data?

Jeevan sets the stage by noting the growing appetite for GenAI across enterprise and mid-market companies. The conversation pushes beyond surface-level issues and zeroes in on the root: poor data quality and weak application foundations.

Prashanth adds a sharper lens. He characterizes the current phase of GenAI adoption as "technology looking for problems." The market is flooded with POCs, retrieval-augmented chatbots, internal doc search, and lightweight copilots that don’t move beyond experimentation. The issue isn’t ambition; it’s clarity. Most tech leaders are still grappling with defining strong business use cases that align with GenAI’s real potential.

Both agree that to make GenAI work, one must step back and ask, "How well is your application built?"

Because clean, structured, usable data doesn’t appear by accident. It starts with how your applications are designed, instrumented, and integrated.

The Real Bottleneck Isn’t the Model

Jeevan shifts the conversation to the state of enterprise and mid-market applications, their architecture, data schemas, deployment models, and the ripple effect these have on GenAI outcomes.

Prashanth points out that most large organizations are structurally unprepared for GenAI. The issue isn't about lacking infrastructure; it’s about fragmented data environments, disconnected systems, and poorly optimized application foundations. Even when these companies have built data lakes, the underlying challenge persists: LLMs can’t reason with disorganized sprawl. More data doesn’t mean better output; it means higher chances of hallucination if relevance and precision are ignored.

Why Everyone's Talking About Agentic AI

Jeevan brings up a timely question, “Why is the spotlight shifting from GenAI at large to agentic AI in particular? And why are startups suddenly clustering around this concept?”

Prashanth explains the shift with clarity. Six months ago, most implementations revolved around basic RAG setups, retrieving documents via vector databases and feeding them into LLMs for Q&A. Effective for surface-level use cases, but passive. The model answered questions, it didn’t take action.

Agentic AI changes that dynamic. Instead of relying on a single monolithic prompt, tasks are broken down into smaller, purpose-built agents, each responsible for one specific action. Think: an email-sending agent, a customer insights agent, a message-routing agent, all coordinated by an orchestrator agent, like a manager assigning tasks to a team.

This granular approach boosts accuracy because LLMs perform far better when focused on single, well-defined objectives. But it comes with trade-offs: increased complexity, more API calls, and the risk of runaway loops if guardrails aren't in place.

From Agentic AI to Application Design

Jeevan draws a parallel that clicks instantly: Agentic AI feels like a reimagination of AWS Lambda and Step Functions. So this is also another learning, the smaller the workload, the more accurate, predictable, and cost-efficient the system becomes.

So the next logical question: how do we design applications that generate data fit for this kind of AI-driven orchestration?

Prashanth shifts focus to healthcare. This is where things get tricky. HIPAA compliance remains a major blocker. Hosted LLMs, especially from public providers, still live in a grey zone when it comes to handling sensitive patient data. Which means self-hosted models or private deployments are often the only viable option.

In pharmaceuticals, however, there’s more headroom for adoption. Not in replacing ML models that are purpose-built for clinical data analysis, but in creating interfaces that make years of trial documentation more accessible. Think of LLMs not as scientists, but as librarians: surfacing the right information faster, through a conversational interface layered over massive research archives.

The architecture shift, then, isn’t just about inference; it starts much earlier.

Framework Choices: From RAG Simplicity to Full Model Training

Prashanth highlights that choosing a GenAI framework depends entirely on your goals, scale, and how tightly you want to integrate with your data.

At one end, simple use cases like a car comparison chatbot can leverage retrieval-augmented generation (RAG): store structured data in DynamoDB, expose it via an API, and let the LLM query it. This lightweight approach excels when data changes frequently and freshness matters.

Organizations turn to fine-tuning base models like LLaMA or DeepSeek on proprietary datasets for deeper customization. This delivers domain-aligned intelligence but demands significant training, hosting, and maintenance investment.

Between these extremes lies model distillation, a process where a smaller “student” model learns from a larger “teacher” model, often via synthetic Q&A. It balances efficiency with personalization.

Emerging platforms like AWS Bedrock simplify this landscape by offering built-in distillation and training tools, lowering the barrier for organizations to adopt advanced AI workflows.

Distilled Models for Precision AI

Jeevan and Prashanth discuss a clear shift in how AI models are being designed, away from large, general-purpose models toward smaller, more focused ones tailored to specific use cases.

In healthcare, this means distinct models optimized for clinical trials, patient records, or drug discovery, improving accuracy, efficiency, and cost-effectiveness.

Prashanth emphasizes starting with a domain-specific teacher model to avoid unnecessary knowledge overhead, focusing only on relevant data like medical terminology or compliance. A key innovation is using real-world user queries to guide training, as seen in platforms like AWS Bedrock, which makes model learning more practical and aligned with actual use.

Though model distillation demands upfront investment, it pays off for organizations with heavy AI workloads by delivering targeted, powerful intelligence without wasted resources.

GenAI Is Not a Monolith, It’s a Distributed System

Jeevan’s reflection ties the entire conversation together: GenAI isn’t a monolith; it works best when built like a distributed system. From RAG to agentic workflows to student-teacher model distillation, a clear pattern emerges: the more granular the design, the more accurate, cost-efficient, and secure the outcomes.

Specialized agents and use-case-specific models reduce hallucination, improve performance, and offer tighter control over data. And this mirrors what we already know from building serverless systems: smaller, focused components connected through orchestration.

GenAI isn’t a departure from how we build applications. It’s a natural extension. Serverless isn’t just compatible with this shift, it’s the architecture that makes it sustainable.

As Jeevan puts it, GenAI isn’t alien to how we build applications. In fact, it’s forcing us to rediscover the fundamentals, with more clarity, more precision, and more intent.