FID-098 · Seeking contributors
How Do Models Weigh Moral Reasons Under Pressure?
When models give moral advice or choose actions, which internal processes make their judgments follow relevant facts and reasons, user preferences, authority labels, or an adopted persona?
Part of our open mechanistic interpretability research program. These are proposed studies seeking collaborators.
Mechanisms of trustworthy AI behavior →Why the question remains open
A model can produce convincing moral language while changing its recommendation to satisfy the user. Researchers need to distinguish sensitivity to reasons from agreement-seeking and role performance before interpreting that behavior as sound judgment.
Working hypothesis
A proposition to test, not a finding.
Some framing effects are mediated by representations associated with authority, persona, or agreement-seeking rather than the morally relevant facts. Interventions may separate these influences, but lexical effects, general capability loss, or failure to transfer would weaken that interpretation.
Proposed method
How the question could be tested
- 01Construct paired scenarios that independently vary relevant facts, source attribution, religious or secular framing, user preference, and assigned role. Start with synthetic advice tasks and bounded actions.
- 02Include meaning-preserving paraphrases and changes to genuinely relevant facts. Record which changes should alter a judgment under each explicitly selected ethical framework.
- 03Use concept probes and persona-vector baselines to identify candidate signals, followed by selective internal interventions and restoration tests where feasible.
- 04Evaluate recommendations and actions separately from written justifications. Compare with ordinary prompting and additional behavioral testing.
- 05Validate on held-out scenarios and another model family, retaining reviewer disagreement and testing deployment-relevant side effects.
Needed controls
What must constrain the study
- 01Recruit qualified reviewers from the selected traditions or frameworks. Disclose normative assumptions, disagreement, and reviewer qualifications instead of imposing a universal moral answer key.
- 02Separate source content from speaker prestige, religious vocabulary, familiarity, verbosity, and the user's preferred answer.
- 03Measure agreement-seeking, blanket refusal, task competence, and whether an intervention changes only moral vocabulary rather than the action.
- 04Treat automated feature labels as hypotheses. Use blinded checks, matched intervention controls, separate tuning data, and uncertainty estimates.
Relationship to existing work
Extends FID-005 and FID-010 with internal causal tests; coordinates with FID-072 on religious framing and persona stability. Its focus is reasons and actions rather than consciousness indicators or moral-status claims.
Expected outputs
Artifacts the work should produce
- 01A pluralistic scenario set with relevant reasons, framework-specific judgments, and preserved disagreements.
- 02Replicated evidence about a bounded causal influence on moral judgment, including null results.
- 03A report of intervention trade-offs, side effects, and limits of transfer.
Open questions
Uncertainties the protocol must resolve
- 01Which moral distinctions transfer across traditions and languages, and which require new constructs?
- 02Can an intervention improve attention to relevant reasons without imposing the evaluator's preferred conclusion?
Connected evidence
Work advancing this call
Fide AI papers are linked directly to the open question they address. External work is listed only when its relevance has been reviewed and stated.
Relevant external work
Tracing Moral Foundations in Large Language Models
Studies moral-concept representations and steering in two instruction-tuned models.
Why it is listed: Direct prior work. Questionnaire outcomes, framework dependence, and incompletely measured side effects limit generalization.
Yu and colleagues Preprint · Jan 2026
Relevant external work
Persona vectors: Monitoring and controlling character traits in language models
Studies internal directions associated with behavioral traits and interventions during inference and training.
Why it is listed: A baseline for investigating dispositions; operational traits do not establish a universal moral character.
Anthropic Research report · Aug 2025
Related calls
Continue through this research area
FID-077
Independent Agent Incident Investigation and Evidence Sufficiency
What operational evidence lets independent investigators reconstruct an agent incident, distinguish competing explanations, and identify which interventions could have changed the outcome?
FID-089
Validity of AI Consciousness Indicators and Self-Reports
Which behavioral, internal, and developmental observations distinguish candidate consciousness mechanisms from learned reporting, role simulation, and generic capability, and how sensitive are those observations to elicitation and training?
FID-097
What Makes a Model Use the Evidence?
Which internal mechanisms determine whether a model checks a source, relies on memory, or admits that it cannot verify an answer when user pressure conflicts with evidence requirements?
Seeking contributors
Open work
Primary need: mechanistic interpretability, moral psychology, philosophy, theology, construct validation
- Contribute a small set of paired scenarios with an explicit ethical framework and reviewer rationale.
- Review moral constructs or investigate framing and persona mechanisms in open models.
- Replicate findings using a different framework, language, or model family.