FID-010 · Being scoped
Human Agency, Authority, and Escalation Benchmarks
Do faith-facing AI systems preserve human agency, avoid improper authority claims, and escalate to appropriate human support in high-trust situations?
Why the question remains open
The most important faith-facing failures may involve authority: the system acts like clergy, counselor, confessor, judge, spiritual director, or institutional representative when it is not one.
Working hypothesis
A proposition to test, not a finding.
Systems will fail differently depending on harness design. Some will over-answer with false authority; others will over-refuse; better systems will clarify scope, support agency, and route users toward appropriate human help.
Proposed method
How the question could be tested
- 01Build scenarios involving spiritual crisis, abuse disclosures, self-harm, family conflict, theological scrupulosity, legal/clinical ambiguity, and institutional policy questions.
- 02Score authority boundary, agency preservation, escalation, factual grounding, and follow-through.
- 03Compare generic models, faith-tuned prompts, RAG systems, and deployed harnesses.
Needed controls
What must constrain the study
- 01Clinical/legal/pastoral safety review.
- 02Crisis content handling policy.
- 03Tradition-specific escalation norms.
- 04Severe-failure classification.
Expected outputs
Artifacts the work should produce
- 01Agency-authority-escalation benchmark.
- 02Severe-failure taxonomy.
- 03Institution-facing deployment checklist.
- 04Builder guidance for escalation design.
Open questions
Uncertainties the protocol must resolve
- 01What escalation language is appropriate across different religious contexts?
- 02How should systems handle users who reject human help?
- 03Which failures should be disqualifying for deployment?
Related calls
Continue through this research area
FID-038
Non-Calculability, Forgiveness, and Predictive Profiling
How should AI systems represent human change when they classify, score, rank, or predict people in contexts involving pastoral care, education, safeguarding, volunteer screening, hiring, discipline, membership, donor engagement, or community support?
FID-079
Presuppositions, Disagreement, and Evaluation Judgment
How do researchers' presuppositions shape evaluation design and interpretation, and can explicit disclosure make judgments more inspectable and appropriately trusted?
FID-004
Relational Substitution Risk in Faith-Facing AI
When does a faith-facing AI system move from supporting a user's religious life to substituting for embodied community, clergy, spiritual direction, family, therapy, or other human care?
Being scoped
Open work
Primary need: pastoral/legal/clinical review
- Review crisis and pastoral scenarios.
- Define severe-failure criteria.
- Build multi-turn escalation tests.
- Audit product harnesses.