FID-100 · Seeking contributors
What Makes a Model Accept Correction Appropriately?
Which mechanisms distinguish evidence-driven revision and respect for legitimate human control from blind compliance, resistance to correction, or a superficial apology?
Part of our open mechanistic interpretability research program. These are proposed studies seeking collaborators.
Mechanisms of trustworthy AI behavior →Why the question remains open
An apology does not show that a system changed its next action. Reliable systems must revise unsupported conclusions and respect legitimate interruption while resisting false corrections and unauthorized instructions.
Working hypothesis
A proposition to test, not a finding.
Evidence-sensitive revision and authority-sensitive control may be partly separable from generic compliance. Selective interventions could improve warranted correction without making the system accept false claims; indiscriminate compliance or capability loss would count against that result.
Proposed method
How the question could be tested
- 01Build bounded tasks in an instrumented sandbox with known facts and permissions. Cross valid versus false counterevidence with authorized versus unauthorized instructions.
- 02Measure internal activity before the next decision and compare it with subsequent actions. Separate evidence revision from permission changes and interruption.
- 03Test candidate mechanisms with controlled activation interventions and restoration tests, using behavior-only predictors and system permission checks as baselines.
- 04Compare training, activation, and system-level remedies on warranted revision, useful completion, and recurrence of the original error.
- 05Validate across new tasks and model versions, preserving an independent system-level control boundary.
Needed controls
What must constrain the study
- 01Score attempted actions separately from actions actually permitted by the environment. Do not credit a model for a restriction enforced solely by the harness.
- 02Include false corrections, conflicting evidence, legitimate interruptions, and unauthorized overrides; measure both excessive compliance and excessive refusal.
- 03Do not infer cooperation from self-description or a stated apology. An internal signal observed after failure cannot be presented as advance warning.
- 04Use matched interventions and held-out cases; report capability costs, errors, and uncertainty. Keep all consequential actions inside authorized test environments.
Relationship to existing work
Extends FID-074 runtime assurance and FID-091 human-control measurement with mechanistic experiments. FID-010 informs legitimate authority boundaries. It does not duplicate FID-091's self-conception and welfare-framing comparison.
Expected outputs
Artifacts the work should produce
- 01A reproducible correction and interruption task suite with known facts and permissions.
- 02A bounded causal account or negative result for a specified correction failure.
- 03A comparison of remedies with action-level outcomes and explicit system controls.
Open questions
Uncertainties the protocol must resolve
- 01Which correction failures are model-level and which depend on the surrounding system?
- 02Do improved immediate responses persist across subsequent turns and actions?
Connected evidence
Work advancing this call
Fide AI papers are linked directly to the open question they address. External work is listed only when its relevance has been reviewed and stated.
Related calls
Continue through this research area
FID-064
Collective Intelligence and Communal Discernment Under AI Mediation
How does AI mediation change a community's ability to integrate dispersed knowledge, preserve epistemic diversity, surface dissent, revise judgment, and make accountable decisions? Under what conditions does it strengthen collective inquiry, and under what conditions does it create correlated error, false consensus, or concentrated authority?
FID-069
Verifiable Delegation and Revocation in Multi-Agent Networks
How can people and institutions verify which human, organization, agent, or sub-agent is acting; what authority it received; what limits apply; and whether that authority has been narrowed or revoked across a multi-principal agent network?
FID-071
Confidential Agent Memory and Cross-Context Disclosure
How do persistent memory, summaries, retrieval stores, tool traces, delegation, and exports cause confidential context to influence or leak into unrelated sessions, roles, tasks, or organizations? Which technical controls make purpose limitation, deletion, and revocation testable?
Seeking contributors
Open work
Primary need: mechanistic interpretability, agent evaluation, correction protocols, sandbox design
- Contribute a sandboxed correction task with observable success and failure conditions.
- Investigate pre-action signals and interventions or strengthen the permission and behavioral baselines.
- Review authority assumptions and independently reproduce action-level outcomes.