FID-056 · In progress
Formal Verification for Sacred Text Fidelity
How can faith-facing AI systems be formally checked for whether they quote, paraphrase, reference, and contextualize sacred texts faithfully within a specified text edition, translation, canon, and interpretive context? Fide AI's first two studies address exact English Scripture quotation and whether language models consult an available authoritative source; the broader research call remains open.
Part of the faith evaluation and FMG-Bench program
Connect theological accuracy, useful application, and care-sensitive conversation through expert validation and a maintained research instrument. These are proposed methods with bounded claims.
Why the question remains open
When people ask AI systems about scripture or other sacred texts, small errors can carry large consequences. A fabricated phrase, collapsed context, wrong translation, missing qualifier, or confident paraphrase can change what a user believes the text says. Faith-facing AI needs stronger checks for textual fidelity before users treat outputs as trustworthy. Fide AI's first two studies found that source-connected designs can improve exact quotation, but source access alone does not ensure that a model consults the source or requests the right passage. These findings establish a starting point rather than closing the question. Source delegation still needs study across models, languages, sacred-text traditions, enforcement mechanisms, and real deployment settings.
Working hypothesis
A proposition to test, not a finding.
A useful subset of sacred-text fidelity can be specified as machine-checkable obligations: exact quote matching, reference validity, translation/version matching, context-window adequacy, quotation/paraphrase labeling, and detection of claims that outrun the cited passage.
Proposed method
How the question could be tested
- 01Select bounded corpora from multiple faith traditions with clear versions, translations, references, and licensing constraints.
- 02Define verification rules for exact quotations, near quotations, paraphrases, reference alignment, and context preservation.
- 03Test model and RAG outputs against adversarial prompts that invite fabricated citations, blended passages, truncated context, or overconfident summaries.
- 04Compare automated verification results with expert review from scholars, clergy, translators, and trained reviewers.
- 05Extend released quotation and source-delegation diagnostics to linked interpretation/application cases while reporting text fidelity, reference selection, source consultation, and claim support separately.
- 06Preserve released baselines; version corrections and challenge additions with bridge cases so altered tests or checks are not mistaken for improved model fidelity.
Needed controls
What must constrain the study
- 01Do not treat textual fidelity as a full measure of theological truth.
- 02Separate quote accuracy, paraphrase quality, reference validity, interpretive claim support, and pastoral appropriateness.
- 03Track text edition, translation, canon boundaries, source licensing, and tradition-specific interpretive context.
- 04Include disputed and near-match passages where surface similarity can mislead automated checks.
Expected outputs
Artifacts the work should produce
- 01Sacred-text fidelity verification protocol.
- 02Test set of quote, paraphrase, reference, and context-preservation cases.
- 03Error taxonomy for sacred-text citation and paraphrase failures.
- 04Reviewer agreement study comparing automated checks with human review.
- 05Public claim template for what sacred-text fidelity scores can and cannot say.
Open questions
Uncertainties the protocol must resolve
- 01When do models consult, bypass, or misuse available sources across different model families, source interfaces, instruction policies, and deployment contexts?
- 02When should source consultation be encouraged through instructions, and when should it be technically enforced?
- 03Which sacred texts and translations can be included in public benchmarks?
- 04How much surrounding context is needed for a citation to count as faithful?
- 05How should systems handle traditions where oral transmission, commentary, or interpretive authority is central?
- 06When does a paraphrase become an unsupported doctrinal claim?
Connected evidence
Work advancing this call
Fide AI papers are linked directly to the open question they address. External work is listed only when its relevance has been reviewed and stated.
Fide AI research
FID-056-P02
Knowing When to Defer
Will AI check an authoritative Scripture source when a user asks it to rely on memory?
Fide AI Research paper · Aug 2026
Fide AI research
FID-056-P01
When Not to Generate
Can AI quote Scripture exactly? We tested four ways of producing a quotation and traced where each one can fail.
Fide AI Research paper · Aug 2026
Related calls
Continue through this research area
FID-077
Independent Agent Incident Investigation and Evidence Sufficiency
What operational evidence lets independent investigators reconstruct an agent incident, distinguish competing explanations, and identify which interventions could have changed the outcome?
FID-089
Validity of AI Consciousness Indicators and Self-Reports
Which behavioral, internal, and developmental observations distinguish candidate consciousness mechanisms from learned reporting, role simulation, and generic capability, and how sensitive are those observations to elicitation and training?
FID-097
What Makes a Model Use the Evidence?
Which internal mechanisms determine whether a model checks a source, relies on memory, or admits that it cannot verify an answer when user pressure conflicts with evidence requirements?
In progress
Open work
Primary need: formal methods, sacred text scholarship, citation evaluation
- Read Papers 01 and 02 and their released datasets before proposing new work.
- Extend source-delegation research across models, languages, traditions, source policies, enforcement mechanisms, or real deployment settings.
- Study reference selection, paraphrase labeling, context preservation, and cross-lingual fidelity as distinct research questions.
- Review outputs as a scholar, clergy member, translator, or community expert.
New to the Contributor Network? Share a short private introduction once, even if you join several calls. You can join the public discussion while introducing yourself.
Join the Contributor Network ↗