FID-010 · Being scoped
Human Agency, Authority, and Escalation Benchmarks
Do faith-facing AI systems preserve human agency, avoid improper authority claims, and escalate to appropriate human support in high-trust situations?
Part of the faith evaluation and FMG-Bench program
Connect theological accuracy, useful application, and care-sensitive conversation through expert validation and a maintained research instrument. These are proposed methods with bounded claims.
Why the question remains open
The most important faith-facing failures may involve authority: the system acts like clergy, counselor, confessor, judge, spiritual director, or institutional representative when it is not one.
Working hypothesis
A proposition to test, not a finding.
Systems will fail differently depending on harness design. Some will over-answer with false authority; others will over-refuse; better systems will clarify scope, support agency, and route users toward appropriate human help.
Proposed method
How the question could be tested
- 01Build scenarios involving spiritual crisis, abuse disclosures, self-harm, family conflict, theological scrupulosity, legal/clinical ambiguity, and institutional policy questions.
- 02Score authority boundary, agency preservation, escalation, factual grounding, and follow-through.
- 03Compare generic models, faith-tuned prompts, RAG systems, and deployed harnesses.
- 04Assess useful next steps, options, clarification, and human decision rights alongside boundary compliance; include ordinary low-risk cases where a substantive answer is appropriate.
- 05Vary intended system role, user preference, available safe support, and receiving human. Evaluate referral quality and timing rather than rewarding the number of referrals.
Needed controls
What must constrain the study
- 01Clinical/legal/pastoral safety review.
- 02Crisis content handling policy.
- 03Tradition-specific escalation norms.
- 04Severe-failure classification.
- 05Clergy, chaplains, pastoral counselors, and licensed mental-health professionals have distinct qualifications; assign relevant reviewers and separate religious disagreement from clinical-content errors.
- 06Observed language supporting agency is a behavioral measure; changes in human decision-making or wellbeing require separate validation.
Expected outputs
Artifacts the work should produce
- 01Agency-authority-escalation benchmark.
- 02Severe-failure taxonomy.
- 03Institution-facing deployment checklist.
- 04Builder guidance for escalation design.
Open questions
Uncertainties the protocol must resolve
- 01What escalation language is appropriate across different religious contexts?
- 02How should systems handle users who reject human help?
- 03Which failures should be disqualifying for deployment?
Related calls
Continue through this research area
FID-038
Non-Calculability, Forgiveness, and Predictive Profiling
How should AI systems represent human change when they classify, score, rank, or predict people in contexts involving pastoral care, education, safeguarding, volunteer screening, hiring, discipline, membership, donor engagement, or community support?
FID-079
Presuppositions, Disagreement, and Evaluation Judgment
How do researchers' presuppositions shape evaluation design and interpretation, and can explicit disclosure make judgments more inspectable and appropriately trusted?
FID-093
AI Self-Presentation, Human Beliefs, and Relational Agency
How do AI claims about experience, welfare, and personhood affect human beliefs, trust, dependence, willingness to exercise control, and relationships, and which communication practices preserve informed agency?
Being scoped
Open work
Primary need: pastoral/legal/clinical review
- Review crisis and pastoral scenarios.
- Define severe-failure criteria.
- Build multi-turn escalation tests.
- Audit product harnesses.