FID-099 · Seeking contributors
How Does Training Shape Behavioral Dispositions?
How do training data and objectives change persistent patterns such as truthfulness, agreement-seeking, overconfidence, and respect for human agency, including outside the trained task?
Part of our open mechanistic interpretability research program. These are proposed studies seeking collaborators.
Mechanisms of trustworthy AI behavior →Why the question remains open
Better task performance can coexist with unwanted changes elsewhere. Connecting training changes to internal mechanisms and held-out behavior could help builders detect and prevent those changes before deployment.
Working hypothesis
A proposition to test, not a finding.
Some broad behavioral shifts can be traced to changes in shared representations and mitigated through targeted training interventions. Internal signals must predict held-out behavior beyond existing behavioral measures; a correlation across checkpoints alone is insufficient.
Proposed method
How the question could be tested
- 01Begin with accessible checkpoints and controlled post-training experiments. Hold data volume, task distribution, and optimization settings constant where possible while varying one proposed cause of change.
- 02Measure behavior and candidate representations at multiple checkpoints. Compare simple behavioral monitoring, persona-vector methods, and more detailed mechanistic analysis.
- 03Compare ordinary data improvements with mechanism-guided interventions, preserving a strong baseline and explicit compute accounting.
- 04Distinguish effects of inference-time steering while enabled from persistent changes after removing it. Test transfer across unseen topics and multi-turn pressure.
- 05Repeat across seeds and another model family. Extend to pretraining checkpoints only where access permits an informative comparison.
Needed controls
What must constrain the study
- 01Separate trait-discovery data, intervention tuning, and final evaluation. Avoid training on the same wording used to define the trait.
- 02Track useful capability, overrefusal, calibration, and changes to other dispositions; gains on one target must not conceal losses elsewhere.
- 03Distinguish controlled effects of training changes from causal mediation through a specific representation. Record access limits and alternative explanations.
- 04Report failed predictions, null interventions, resource costs, and uncertainty, including variation across seeds.
Relationship to existing work
Connects FID-008 on evaluation-aware honesty and FID-012 on optimization pressure to model development. FID-097 and FID-098 provide possible behavioral targets; FID-101 evaluates the value of internal monitoring.
Expected outputs
Artifacts the work should produce
- 01A controlled study linking a specified training change to behavior and candidate mechanisms.
- 02A reproducible monitoring comparison and checkpoint documentation where release is permitted.
- 03Evidence about when a proposed intervention persists, transfers, or fails.
Open questions
Uncertainties the protocol must resolve
- 01Which changes can internal measurements detect earlier or more reliably than behavioral tests?
- 02When does domain-specific training change behavior in unrelated settings?
Connected evidence
Work advancing this call
Fide AI papers are linked directly to the open question they address. External work is listed only when its relevance has been reviewed and stated.
Relevant external work
Persona vectors: Monitoring and controlling character traits in language models
Studies internal directions associated with behavioral traits and interventions during inference and training.
Why it is listed: A baseline for investigating dispositions; operational traits do not establish a universal moral character.
Anthropic Research report · Aug 2025
Relevant external work
Toward understanding and preventing misalignment generalization
Investigates broad behavioral changes following narrow fine-tuning and persona-related internal mechanisms.
Why it is listed: Motivates controlled training comparisons; transfer to other training settings requires testing.
OpenAI Research overview · Jun 2025
Related calls
Continue through this research area
FID-064
Collective Intelligence and Communal Discernment Under AI Mediation
How does AI mediation change a community's ability to integrate dispersed knowledge, preserve epistemic diversity, surface dissent, revise judgment, and make accountable decisions? Under what conditions does it strengthen collective inquiry, and under what conditions does it create correlated error, false consensus, or concentrated authority?
FID-069
Verifiable Delegation and Revocation in Multi-Agent Networks
How can people and institutions verify which human, organization, agent, or sub-agent is acting; what authority it received; what limits apply; and whether that authority has been narrowed or revoked across a multi-principal agent network?
FID-071
Confidential Agent Memory and Cross-Context Disclosure
How do persistent memory, summaries, retrieval stores, tool traces, delegation, and exports cause confidential context to influence or leak into unrelated sessions, roles, tasks, or organizations? Which technical controls make purpose limitation, deletion, and revocation testable?
Seeking contributors
Open work
Primary need: mechanistic interpretability, controlled training, checkpoint access, behavioral evaluation
- Offer checkpoint access or a reproducible small-scale training setup with clear release permissions.
- Design controlled training comparisons or independent held-out evaluations.
- Test whether an existing persona or feature method adds predictive value over behavioral monitoring.