Constitutional Oversight: A Disease-Agnostic Supervisory Agent for Clinical AI Safety

The multi-party team led by UCSD, including Anthropic and Harvard Medical School, will design and build the disease-agnostic supervisory AI system that continuously monitors the ADVOCATE TA1 clinical agents after their successful deployment. This includes detecting unsafe recommendations, and out-of-distribution behavior in real time, providing an ongoing assurance layer that the U.S. Food and Drug Administration (FDA), health systems, patients, and clinicians can trust.

Anthropic will support the build of the supervisory system and clinical decision support interface that spawns adversarial defender and refuter agents against a swappable clinician-authored constitution, with anti-sycophancy rewards and adaptive council sizing. UCSD will ground the system in clinical deployment, Harvard Medical School will lead evaluation and clinical validation. The team's objective is real-time flagging of unsafe or out-of-distribution recommendations, each with a transparent, inspectable rationale.

Back to Award Directory