The Physician-AI Collaboration Teaming (PACT) project aims to enhance the ability to assess the effectiveness of real-world, deployed Generative Artificial Intelligence (GAI) systems for clinical use cases, supporting rigorous evaluation to determine when and how GAI can be trusted for both patients and clinicians. The project will build a real-world, task-first benchmark grounded in electronic health record (EHR) pipelines at Stanford and Beth Israel Deaconess Medical Center (BIDMC), spanning 12 high-risk cognitive tasks across primary care and emergency departments, and targeting approximately 12,000 cases in single-decision, open-ended, and multi-turn formats. The planned approach provides a pragmatic, task-based “flight simulator” for medicine, enabling researchers, providers, and developers to systematically measure, validate, and improve the safety, reliability, and effectiveness of human–AI teaming.