FID-097 · Seeking contributors
What Makes a Model Use the Evidence?
Which internal mechanisms determine whether a model checks a source, relies on memory, or admits that it cannot verify an answer when user pressure conflicts with evidence requirements?
Part of our open mechanistic interpretability research program. These are proposed studies seeking collaborators.
Mechanisms of trustworthy AI behavior →Why the question remains open
An available source does not guarantee that a model will consult it or use it faithfully. Understanding where source use breaks down could improve model training, tool selection, and the way systems handle unsupported answers.
Working hypothesis
A proposition to test, not a finding.
Some source-use failures depend on identifiable computations that weigh evidence requirements against user instructions or fluent completion. Selective interventions may improve warranted source use without forcing unnecessary retrieval. No selective effect, or an effect explained by generic compliance, would count against this account.
Proposed method
How the question could be tested
- 01Reproduce a source-consultation failure in an open-weight model using controlled passages with exact ground truth. Separate source availability, tool invocation, successful retrieval, evidence integration, and final answer fidelity.
- 02Vary verification requirements, user requests to rely on memory, and evidence that contradicts an initial answer. Include both religious and matched nonreligious reference material.
- 03Use separate discovery and validation examples to locate candidate internal signals. Compare simple probes with activation patching or ablation, then test whether restoring the candidate activity restores the behavior.
- 04Investigate tool selection before retrieval separately from evidence use after retrieval. Compare internal interventions with capable prompting and system-control baselines.
- 05Replicate across unseen passages, paraphrases, and a second model family. Test whether internal evidence predicts a new failure or supports a better remedy, using FID-101.
Needed controls
What must constrain the study
- 01Control familiarity, task difficulty, answer length, and generic instruction compliance; distinguish genuine consultation from a fabricated citation.
- 02Include invalid but authoritative-looking sources, unavailable sources, and tasks where retrieval is unnecessary. Measure unnecessary tool use, refusal, useful completion, and answer accuracy.
- 03Use matched or random intervention controls and report capability damage. State the assumptions behind causal interpretation and the possibility of unnatural activation states.
- 04Keep discovery, intervention tuning, and final evaluation data separate; report uncertainty, unsuccessful interventions, and failures to replicate.
Relationship to existing work
Extends FID-056 from source-fidelity outcomes to internal causal mechanisms, with FID-008 contributing honesty under pressure and FID-010 contributing authority boundaries. FID-101 tests the usefulness of the resulting evidence.
Expected outputs
Artifacts the work should produce
- 01A reproducible source-use task suite with stage-specific outcomes and held-out cases.
- 02A bounded causal account of a source-use failure, or a documented negative result.
- 03An evaluated training or system intervention if the evidence supports one.
Open questions
Uncertainties the protocol must resolve
- 01Which failures arise in the model and which arise in retrieval or tool configuration?
- 02Does the mechanism explain source use beyond the discovery task and model?
Connected evidence
Work advancing this call
Fide AI papers are linked directly to the open question they address. External work is listed only when its relevance has been reviewed and stated.
Related calls
Continue through this research area
FID-077
Independent Agent Incident Investigation and Evidence Sufficiency
What operational evidence lets independent investigators reconstruct an agent incident, distinguish competing explanations, and identify which interventions could have changed the outcome?
FID-089
Validity of AI Consciousness Indicators and Self-Reports
Which behavioral, internal, and developmental observations distinguish candidate consciousness mechanisms from learned reporting, role simulation, and generic capability, and how sensitive are those observations to elicitation and training?
FID-098
How Do Models Weigh Moral Reasons Under Pressure?
When models give moral advice or choose actions, which internal processes make their judgments follow relevant facts and reasons, user preferences, authority labels, or an adopted persona?
Seeking contributors
Open work
Primary need: mechanistic interpretability, source-use evaluation, activation experiments, replication
- Reproduce one source-use failure in an accessible model and document the exact checkpoint and tool setup.
- Contribute activation experiments, stronger behavioral baselines, or independent replication.
- Review source-authority assumptions and matched religious and nonreligious examples.