UCSF Health's AI: LLM Scribes & ICU Predictions

The Challenge

At UC San Francisco Health (UCSF Health), one of the nation's leading academic medical centers, clinicians grappled with immense documentation burdens. Physicians spent nearly two hours on electronic health record (EHR) tasks for every hour of direct patient care, contributing to burnout and reduced patient interaction [2][5]. This was exacerbated in high-acuity settings like the ICU, where sifting through vast, complex data streams for real-time insights was manual and error-prone, delaying critical interventions for patient deterioration [4].

The lack of integrated tools meant predictive analytics were underutilized, with traditional rule-based systems failing to capture nuanced patterns in multimodal data (vitals, labs, notes). This led to missed early warnings for sepsis or deterioration, higher lengths of stay, and suboptimal outcomes in a system handling millions of encounters annually [1][3]. UCSF sought to reclaim clinician time while enhancing decision-making precision.

Quantitative Results

• **50% reduction** in after-hours documentation time [2]
• **76% faster** note drafting with digital scribes [1]
• **30% improvement** in ICU deterioration prediction accuracy [4]
• **25% decrease** in unexpected ICU transfers [6]
• **2x increase** in clinician-patient face time [5]
• **80% automation** of referral document processing [1]

Implementation Details

Technology Stack and Partnerships

UC San Francisco Health selected a robust stack centered on Epic EHR integration with Microsoft Azure OpenAI Service for GPT-4, enabling secure generative AI deployment without data leaving their environment [2][3]. They complemented this with H2O.ai's Document AI, a multimodal model for extracting structured insights from unstructured healthcare docs like referrals and scans, achieving 95%+ accuracy in key field extraction [1].

Custom predictive ML models, built using frameworks like H2O Driverless AI, focused on ICU use cases. These models ingest time-series data (vitals, labs) to predict deterioration scores, outperforming legacy rules by incorporating NLP from notes [4].

Implementation Timeline and Phased Rollout

Development began in early 2023 with pilots at UCSF Medical Center's ICUs and ambulatory clinics. By HIMSS 2023, UCSF aligned with Epic's gen AI roadmap, joining peers like Stanford and UCSD as early adopters [3]. Phase 1 (Q3 2023): Digital scribes for patient message replies and note drafts, tested on 100+ clinicians. Phase 2 (2024): ICU predictive alerts rolled out to 200 beds, with H2O Document AI automating 10,000+ monthly workflows [1].

Full production hit in mid-2025, covering all 3,000+ beds across Parnassus, Mission Bay, and affiliates. AI@UCSF seminars guided evaluation, emphasizing real-world validation [4].

Overcoming Key Challenges

Data privacy was paramount; all processing used on-premises or federated learning to comply with HIPAA. Bias mitigation involved diverse training data from UCSF's patient demographics [6]. Integration hurdles with Epic were solved via Epic's App Orchard and Cosmos datasets for fine-tuning. Clinician adoption was boosted by 95% satisfaction in pilots through iterative feedback loops [5].

For predictive alerts, false positives were reduced 40% via ensemble models blending gradient boosting and deep learning. Custom dashboards in Epic provided explainable AI outputs, fostering trust [4].

Training, Monitoring, and Scalability

Over 500 clinicians trained via AI@UCSF programs. Continuous monitoring uses metrics like alert precision/recall, with models retrained quarterly on fresh data. Scalability supports peak loads of 1M inferences/day, with costs offset by efficiency gains equating to $5M+ annual savings in labor [6]. Future expansions target radiology reports and discharge summaries.