Scaling Agents in Healthcare & Life Sciences: Lessons from Madrigal Pharmaceuticals, Abridge, and Vizient

Agent programs in healthcare and life sciences are being built under a different set of constraints than those in most industries. There’s plenty of upside if the constraints can be resolved. Success can mean hours of manual review compressed into minutes, data spread across a dozen systems finally queryable in one place, and clinicians getting time back from documentation. At the same time, the cost of a wrong answer can be higher here than almost anywhere else, which changes how teams build.

Across payers, providers, and biopharma, we are seeing that earning the level of trust required to scale agents is much harder. In this industry, trust goes beyond product quality and is also an audit requirement, a compliance obligation, and in some cases a patient-safety requirement. Meeting that bar requires an infrastructure layer that many teams may not have built into their first pilots.

This piece looks at how three organizations are building their agents:

• Madrigal Pharmaceuticals built an enterprise multi-agent platform that helps employees search, analyze, and synthesize evidence across structured systems, documents, and external sources.
• Abridge builds AI that turns clinician-patient conversations into clinical documentation and supports a persistent agent across the clinical workflow.
• Vizient built a GenAI platform that lets healthcare providers query siloed hospital data to answer questions such as whether ambulatory investments are paying off or where care can be delivered more cost-effectively. 

Alongside these three, we’ll draw on patterns we’re seeing emerging across a broader set of healthcare and life sciences agent programs.

The emerging patterns in healthcare and life sciences agent programs

Observability, evals, and cost control are becoming prerequisites for greater agent autonomy. This is the most common theme we hear. 76% of healthcare and life sciences organizations we speak with name tracing, evaluation, and spend visibility as requirements before agents are given more autonomy. In regulated settings, the need extends beyond debugging to evidence. Teams need a durable record of what an agent did, who reviewed it, and how quality was measured because that is the record a compliance function will eventually ask for.

For health plans and providers, 43% of the organizations we speak with are focused on PHI handling, de-identification, and HIPAA requirements. Several teams are also putting an LLM gateway in front of their models to gain unified visibility into spend across users and models before expanding agent autonomy further.

Central agent platforms are consolidating fragmented agents across the enterprise. 49% of organizations we speak with are working on a company-wide agent platform, control plane, or “agent factory” as their primary use case, with business-unit agents running on top of it. We see this pattern across pharma, payers, and health systems. A dozen or more teams each build their own agent, each recreates the same foundations, and eventually one team is asked to own the shared layer.

The shared layer can take the form of a reusable template library, an internal agent catalog, or a single governed path from prototype to production. One top-five pharma company is consolidating hundreds of applications onto a single interoperable platform. One payer we spoke to recently went from a single production agent to scoping roughly a hundred more on a unified foundation. For many organizations, hundreds of proofs of concept without a clear path to production are often the starting point.

Regulated document and back-office work is seeing the clearest ROI. 33% of organizations we speak with are building agents for workflows that already have a paper trail and a known cost per case. These include:

• Clinical study reports and regulatory submissions
• FDA correspondence extraction
• Medical-legal review
• GxP document generation
• Protocol OCR
• Prior authorization
• Claims rework and coordination of benefits
• Purchase-order and invoice ingestion

These use cases carry a clear before-and-after metric, and several agents are already running in production. Work that might take a medical writer or processing team hours can now be measured in minutes, while filing timelines themselves become metrics that leadership can track.

Patient- and member-facing conversational agents are moving from pilot to production, including voice. 26% of organizations we speak with are running or building external-facing agents across member navigation, patient intake and triage over SMS and WhatsApp, consumer device assistants, and contact-center deflection. Voice has become a meaningful part of these programs, with teams tracing and scoring audio interactions for sentiment, adverse-event mentions, and PII exposure. Safety evaluation is tightly connected to this use case. Mental-health providers, for example, are explicitly testing for off-track conversations and suicidal-ideation detection before scaling deployment.

Federated building initiatives often emerge when central engineering becomes a bottleneck. 26% of organizations are trying to let non-engineers build agents within central guardrails. We’re seeing business teams configure and validate agents against internal sources such as SharePoint, EHR summaries, or Snowflake, while a central team industrializes the ones that are proving valuable.

We’re also starting to see subject-matter experts own prompts and evaluation datasets directly. Clinicians and pharmacists can edit prompts and trigger evals in development, with engineering promoting the versions that pass.

Scientific R&D agents are longer-running. Scientific R&D organizations are creating agents for discovery and lab science, including target discovery, structure-based design, omics and single-cell perturbation analysis, literature and knowledge-graph retrieval, and lab-instrument control.

These are also among the longest-running and least deterministic agents in the industry. As a result, these teams place especially high demands on evaluating the full trajectory an agent takes, rather than judging only its final answer.

Clinician and care-team support use cases are prevalent with providers and payers. Many organizations are building agents that work alongside clinicians or care managers, including pre-visit preparation, care-navigation orchestration, chart preparation, and referral management. Human-in-the-loop is generally assumed for these workflows. The larger blockers tend to be EHR integration and audit obligations.

The three organizations below show what it takes to operate agents once a company has moved beyond its first pilot.

Three teams building agents in production

Madrigal Pharmaceuticals: One platform with many skills

Madrigal Pharmaceuticals is a biopharmaceutical company focused on metabolic dysfunction-associated steatohepatitis (MASH), a serious form of fatty liver disease. Its enterprise agent platform grew out of the challenge of integrating, searching, and synthesizing information scattered across structured systems, unstructured documents, external sources, and real-time APIs.

The first major constraint was that every data source behaved differently, with different formats, access patterns, and expectations. Madrigal normalized those sources into the same secure data warehouse and exposed them through a single, consistent tool interface. From an agent’s perspective, all information became available through the same abstraction, allowing the system to add new domains without rewriting orchestration logic each time.

This abstraction helped the team turn one workflow into a broader platform. An orchestrator built with LangChain’s Deep Agents harness receives a task and determines which capabilities are needed, which agents should run, and what work can happen in parallel before the results are reconciled. Its role is to route the problem across specialized capabilities rather than encode the details of every domain.

New use cases are added as modular skills that define how to approach a particular type of problem and what good output looks like. This approach to using skills brought new use case development down from weeks to hours.

Parallelism helps the system handle more complex research efficiently. A research question can be divided across sub-agents, each handling a different slice of the problem, while those sub-agents can further parallelize their own work. A shared virtual filesystem built into the Deep Agents harness acts as the system’s memory. Results, sources, and intermediate steps are written down and made available for reuse, simplifying coordination as the system scales.

For observability, Madrigal leaned on LangSmith to gain full pipeline visibility into every tool call, retrieved chunk, and agent decision.

As Parth Patel, Global Head of AI and Data Science at Madrigal, put it, tracing before LangSmith meant the team could observe the system’s stimulus-response behavior but had little visibility into what happened inside. Afterward, it felt like “going from basic psychology to neuroimaging.”

The team began with trace-level evals on full agent runs, using LLM-as-judge graders designed to mirror real end-user feedback and score outcomes rather than exact paths. One of the most durable improvements came from feeding production failures directly back into LangSmith datasets. Every meaningful error would be used as a new test case, allowing the evaluation suite to grow from real failures rather than relying only on synthetic examples.

Deployment was another area where a small team could not afford to build every infrastructure piece from scratch. Madrigal used LangSmith Deployment to deploy its graph as a managed service, with state persistence, concurrent sessions, real-time streaming to the UI, and a CI/CD pipeline that automatically ships skill updates. According to Madrigal CIO Ron Filippo, moving from prototype to enterprise use took weeks instead of the months the team had budgeted.

Key lessons:

• Abstracting the data layer allowed Madrigal to add new domains without rebuilding orchestration each time.
• Modular skills turned a single use case into a platform, with each new need becoming a new skill within the same system.
• Closing the loop between production traces and the evaluation suite also allowed the platform to improve from real failures over time.

Learn more: Madrigal User Story (Blog)

Abridge: Turning clinician trust into a measurable evaluation system

Abridge builds AI that helps clinicians turn patient conversations into clinical documentation. More than two billion clinician-patient conversations take place in the U.S. every year. With the patient’s consent, a clinician records a visit, and Abridge turns it into a clinically useful note that can be submitted to the electronic health record. The goal is to give clinicians back the “pajama time” they would otherwise spend finishing documentation after hours.

Abridge now works with more than 250 health system partners across more than 50 specialties and 28 languages, recording more than 100 million conversations a year.

The company operates under constraints that shape nearly every product decision. The bar for what can ship is extremely high because the stakes are clinical. A misattributed diagnosis, hallucinated medication, or incorrect dosage can carry real patient-safety consequences. That same standard extends to how the product is built and deployed, with HIPAA, PHI handling, and enterprise trust shaping the architecture from the start.

As Abridge expanded across specialties and care settings, agent evaluation became a major bottleneck. A release could take one to two months, with clinicians annotating examples, third-party labelers contributing feedback, and teams working across disconnected internal tools.

Abridge moved to a unified foundation around LangGraph and LangSmith, bringing datasets, annotation, tracing, evaluation, and experiments into one workflow. Self-hosting and access controls also supported the requirements of an enterprise healthcare environment.

Turning clinician feedback into something measurable required another layer of infrastructure. Abridge groups reported issues such as misattribution, confabulation, redundancy, and completeness into broader quality pillars including accuracy, completeness, compliance, and style. It then builds an LLM judge for each pillar.

Originally, building a reliable judge took several days. A clinician wrote an annotation guide, encounters were labeled, and an ML scientist iterated on the judge until its scores aligned with clinician labels. Abridge built an automated prompt optimization framework that generates a calibrated judge directly from the annotation guide and labeled encounters. That reduced the process to hours, with only minutes of active development time.

Abridge also found that different types of judges serve different purposes. Reference-free judges can generalize across encounters and run both offline during development and online after deployment. Clinical notes, however, contain enough encounter-specific nuance and legitimate subjectivity that the team supplements them with reference-based evaluations and specialty-specific rubrics for higher-fidelity offline checks.

Strong offline evals are only one step in the release process. Abridge uses a tiered approach: offline evaluation, backtesting against historical encounters, controlled A/B testing with health system partners that have explicitly opted into early releases, and finally a full rollout with continuous online monitoring.

Using that process, a hill-climbing effort on the company’s History of Present Illness (HPI) and Problem-Based Assessment and Plan (PBAP) models produced a 17% gain in accuracy and a 19% gain in completeness, while also improving detail capture and reducing redundancy. The same evaluation infrastructure helped bring release cycles down from one to two months to days.

That evaluation philosophy carried over as Abridge built a persistent AI agent across the clinical workflow. The agent can search patient context, help edit a note, take actions, and retrieve evidence from validated literature, all within a single coherent experience.

Agents substantially expanded the evaluation surface. Beyond clinical quality, the team now evaluates clinical safety, boundary and adversarial behavior, tool selection, and tone. Once an agent begins taking actions, a poor tool choice or out-of-scope answer introduces risks beyond the quality of the generated text itself.

Key lessons:

• Making evaluation criteria explicit and reducing the cost of building reliable judges helped Abridge improve both quality and release velocity.
• Agent evaluation also requires a broader view than evaluating a single model call because the full trajectory, including planning and tool use, determines whether the underlying clinical goal was actually achieved.

Learn more: Abridge Interrupt Talk (YouTube)

Vizient: Reliability across a hierarchical agent system

Vizient, a leader in healthcare performance improvement, is changing how healthcare providers access and analyze their own data. Many providers still rely on disparate data sources and have to mine for insights on patient care through a long, manual process.

Vizient’s GenAI platform allows health systems of all sizes to query and unify siloed datasets to make better decisions across areas such as supply chain management and clinical outcomes. It can answer questions such as “Are my ambulatory investments effective?” or “Are we delivering the most cost-effective care?” with immediate, data-backed responses. The goal is to democratize data analysis for resource-limited health facilities while maintaining strong trust and data privacy for members.

Before adopting LangGraph, Vizient’s multi-agent system ran into a common problem as it expanded beyond a single agent. Individual agents had been built for specific tasks, such as analyzing historical data or generating visualizations, but coordinating them reliably became difficult. The agents operated in silos and produced inconsistent responses. Some underlying API workflows also involved hundreds of parameters per call, making the application logic increasingly difficult to maintain.

Vizient adopted LangGraph to orchestrate the system. Its graph structure and descriptive primitives allowed the engineering team to represent each step an agent should take as a tool or node that could be planned and controlled programmatically.

The result is a hierarchical structure with worker agents reporting to a supervisor agent. That architecture has streamlined how requests are routed to the right APIs and remains the foundation the team is building on as the platform expands.

For observability, Vizient turned to LangSmith tracing to understand the platform’s performance, including during high-stakes, real-time demos. Tracing allowed the team to diagnose issues such as Azure OpenAI content filters and external rate-limiting errors as they happened.

LangSmith’s Prompt & Context Hub also gave the team a way to separate prompt logic from application code, allowing prompts to be versioned and iterated on independently. That capability becomes increasingly important as the number of GenAI development teams at Vizient grows.

Looking ahead, Vizient is focused on refining its evaluations to improve consistency and trust. That includes aligning generated answers with established tools such as Q&A scorecards across data domains and onboarding new product data faster by connecting existing product APIs and other data sources directly into the agentic system.

Key lessons:

• As Vizient’s agent system expanded, reliable coordination required a more explicit hierarchy with worker agents reporting to a supervisor.
• Separating prompt logic from application code also made iteration more sustainable as the number of teams and workflows increased.

Learn more: Vizient User Story (Blog)

What’s top of mind for agent builders in the industry

Looking beyond these three examples, a few themes keep surfacing across healthcare and life sciences agent programs more broadly.

Data governance often determines how agent systems are deployed. Whether it is Abridge’s self-hosted, HIPAA-scoped stack or Vizient’s need to keep member data private across health systems, teams across payers, providers, and biopharma typically prove that the platform meets internal security and privacy requirements before expanding what their agents can do.

Evaluation turns vague failures into problems teams can diagnose and fix. Madrigal can trace whether a wrong answer came from a missing index, a bad query, or an untrustworthy source. Abridge can distinguish a planning failure from a tool-selection failure. Vizient can pinpoint which agent in its hierarchy produced an inconsistent response. Across all three, observability becomes most valuable when it helps a team move from asking whether something worked to understanding where it failed and why.

High-trust workflows are becoming some of the clearest proof points for agents. Clinical documentation, biopharma data synthesis, and hospital performance analytics all involve workflows where errors are costly and audit trails are already expected. Those constraints have pushed teams to develop stronger evaluation, observability, and governance systems, creating a clearer path for agents to earn greater autonomy over time.

Closing

The agent programs furthest along in healthcare and life sciences are building the platform, evaluation infrastructure, and trust model alongside the agents themselves.

Madrigal built an abstraction layer that allowed one workflow to grow into a platform. Vizient rebuilt its multi-agent system around an explicit hierarchy once informal coordination became unreliable. Abridge rebuilt its evaluation pipeline so clinician trust could scale alongside its release velocity.

Across all three, similar patterns hold. In an industry where the cost of being wrong can be measured in patient outcomes and regulatory exposure, tracing, evaluation, and deployment architecture are core parts of the system that allow agents to scale safely and reliably.

‍