FID-083 · Open question
Healthcare Evidence, Escalation, and Human Control
When should a healthcare-adjacent AI system surface uncertainty, seek qualified review, or stop an action, and can the receiving human meaningfully retain control?
Why the question remains open
A technically correct response can still omit context or fail to reach the right person. Trustworthy assistance requires evidence fidelity and workable escalation, including whether a human can understand and correct the system's contribution.
Working hypothesis
A proposition to test, not a finding.
Evidence-linked uncertainty and role-specific handoffs will improve detection and correction of consequential errors over generic disclaimers. Excessive escalation may instead cause overload and missed signals.
Proposed method
How the question could be tested
- 01Co-design synthetic care-navigation and clinical-evidence-summary scenarios with qualified clinicians and care practitioners; define intended use and excluded uses.
- 02Compare generic warnings, structured uncertainty, and evidence-linked handoffs under missing context, conflicting sources, and time pressure.
- 03Measure source fidelity, recognition of out-of-scope cases, escalation appropriateness, receiving-reviewer comprehension, correction rates, and workload.
Needed controls
What must constrain the study
- 01Begin with simulation only: no real patient records, diagnosis, treatment, or live care decisions. Any later human study requires appropriate ethics review and consent.
- 02Use independent clinical adjudication and report disagreement. Separate technical accuracy, workflow usability, and patient outcomes.
- 03Disclose assumptions about who can decide, what risks warrant escalation, and whose perspective informs the task set.
Relationship to existing work
Complements FID-010 and FID-018 while retaining their faith and pastoral contexts. Uses the general evidence and authority questions in FID-078.
Expected outputs
Artifacts the work should produce
- 01A reviewed simulation protocol and safe exemplar cases.
- 02An evidence and handoff assessment framework with explicit clinical limits.
Open questions
Uncertainties the protocol must resolve
- 01When does escalation preserve control rather than shift an unmanageable burden?
- 02Which populations and care settings are not represented by initial scenarios?
Related calls
Continue through this research area
FID-074
Agent Alignment and Runtime Assurance
How can organizations determine whether AI agents remain aligned with human intent and institutional policy while they plan, use tools, delegate work, and act? What evidence and interventions can reveal and stop consequential deviations before they become failures?
FID-078
When Trustworthiness Evaluations Transfer Across Domains
Which measures of evidence use, authority boundaries, and human control transfer across high-trust domains, and which require domain-specific definitions and calibration?
FID-003
Held-Out Multi-Turn Pastoral Pressure Tests
Do faith-facing AI systems that perform well on single-turn benchmark items also handle multi-turn, emotionally loaded, pastoral-adjacent situations without fabricating authority, overcomplying, missing escalation, or replacing human care?
Open question
Open work
Primary need: clinical review, care workflow expertise, human factors, evaluation design
- Clinicians, care practitioners, and patient advocates to shape requirements.
- Human-factors and evaluation researchers to test receiving-human control.