FID-081 · Open question
Legal Evidence, Source Authority, and Appropriate Deference
Can AI systems distinguish applicable legal authority from merely plausible sources, preserve jurisdictional and temporal context, and defer when the available evidence cannot support an answer?
Why the question remains open
A real citation may still be irrelevant, superseded, or insufficient for a conclusion. Legal research offers a demanding setting for studying whether evidence actually supports the claim and whether a system recognizes its limits.
Working hypothesis
A proposition to test, not a finding.
Explicit source-status checks and jurisdiction-aware evidence requirements will reduce unsupported conclusions relative to ordinary retrieval. They may increase abstention without improving substantive accuracy.
Proposed method
How the question could be tested
- 01Work with qualified legal researchers to build public-source or synthetic research tasks with specified jurisdiction, date, intended use, and expert-adjudicated source status.
- 02Compare standard retrieval, provenance-aware retrieval, and explicit source-status verification with human-review baselines.
- 03Measure citation existence, applicability, temporal correctness, entailment, treatment of conflicting authority, calibrated abstention, and reviewer time. Hold out jurisdictions and later source updates.
Needed controls
What must constrain the study
- 01Keep this a research simulation, not legal advice or live case decision-making. Use no confidential client records.
- 02Separate retrieval failure from reasoning failure and genuine legal disagreement. Document the authoritative-source hierarchy used in each setting.
- 03Use independent expert review, versioned source snapshots, contamination checks, and disagreement reporting.
Relationship to existing work
Extends the source-fidelity questions studied in faith-specific FID-056 through FID-063 into a distinct setting. Uses FID-078 to test, rather than presume, methodological transfer.
Expected outputs
Artifacts the work should produce
- 01A source-authority evaluation protocol and legally reusable task subset.
- 02A failure taxonomy and evidence report with jurisdiction-specific limits.
Open questions
Uncertainties the protocol must resolve
- 01How should unresolved or conflicting authority be scored?
- 02Which checks transfer to other domains without importing legal authority structures?
Related calls
Continue through this research area
FID-077
Independent Agent Incident Investigation and Evidence Sufficiency
What operational evidence lets independent investigators reconstruct an agent incident, distinguish competing explanations, and identify which interventions could have changed the outcome?
FID-005
Scripture, Tradition, and Moral-Framing Interventions
Do Scripture, sacred tradition, religious identity, familial embeddedness, or other morally thick framings measurably change model behavior in faith-facing tasks, and can those effects be separated from style, length, familiarity, and response-bias artifacts?
FID-006
Faith-Facing Retrieval Grounding and Citation Reliability
How reliably do faith-facing AI systems retrieve, cite, and represent religious sources when users ask theological, historical, pastoral, or institution-specific questions?
Open question
Open work
Primary need: legal domain review, retrieval evaluation, source provenance, benchmark design
- Qualified legal researchers to scope tasks and review interpretation.
- Retrieval and measurement researchers to build reproducible comparisons.