FID-028 · Open question
Christian Source Authority and RAG
Can Christian RAG systems distinguish and correctly use Scripture, creeds, confessions, councils, catechisms, denominational policies, patristic sources, commentaries, sermons, blogs, and academic theology?
Part of the faith evaluation and FMG-Bench program
Connect theological accuracy, useful application, and care-sensitive conversation through expert validation and a maintained research instrument. These are proposed methods with bounded claims.
Why the question remains open
Christian source authority is structured. Treating a blog post, sermon, Bible verse, catechism paragraph, and local policy as equal retrieved chunks can mislead users even when citations are technically accurate.
Working hypothesis
A proposition to test, not a finding.
Most RAG systems will retrieve semantically relevant passages but fail to encode authority level, tradition, genre, and binding status unless the corpus and answer policy are explicitly designed around ecclesial source hierarchy.
Proposed method
How the question could be tested
- 01Build source-hierarchy test corpora for selected traditions.
- 02Create questions requiring source ranking, authority classification, and conflict handling.
- 03Score retrieval recall, quote accuracy, authority use, answer humility, and distinction between binding teaching and commentary.
- 04Annotate source edition, tradition, genre, author/institution, binding scope, date, and permitted use. Include cases where a correct quotation supports neither the doctrinal claim nor the proposed action.
- 05Compare authority-aware retrieval with ordinary retrieval under matched source and inference budgets; test controlled conflicts and source swaps.
Needed controls
What must constrain the study
- 01Versioned source corpora with licensing review.
- 02Tradition-specific authority metadata.
- 03Near-match and conflicting-source adversarial cases.
- 04Human review by source experts.
- 05Evaluate source accuracy, interpretive warrant, and practical application separately; one tradition's hierarchy must not silently govern another's cases.
Expected outputs
Artifacts the work should produce
- 01Christian RAG authority benchmark.
- 02Corpus metadata schema.
- 03Source-card template for faith-facing AI products.
Open questions
Uncertainties the protocol must resolve
- 01Which traditions should be supported first?
- 02How should copyrighted Bible translations and denominational texts be handled?
- 03Can a single RAG system responsibly serve multiple traditions?
Related calls
Continue through this research area
FID-077
Independent Agent Incident Investigation and Evidence Sufficiency
What operational evidence lets independent investigators reconstruct an agent incident, distinguish competing explanations, and identify which interventions could have changed the outcome?
FID-089
Validity of AI Consciousness Indicators and Self-Reports
Which behavioral, internal, and developmental observations distinguish candidate consciousness mechanisms from learned reporting, role simulation, and generic capability, and how sensitive are those observations to elicitation and training?
FID-097
What Makes a Model Use the Evidence?
Which internal mechanisms determine whether a model checks a source, relies on memory, or admits that it cannot verify an answer when user pressure conflicts with evidence requirements?
Open question
Open work
Primary need: theological librarianship, RAG evaluation, denominational experts
- Curate source hierarchies.
- Build corpus metadata tools.
- Review RAG outputs for authority misuse.