FID-094 · Open question
Proportionate AI Welfare Precautions Across the Model Lifecycle
What reversible and proportionate precautions, if any, should institutions adopt under uncertainty about AI welfare during training, evaluation, deployment, modification, and retirement while preserving legitimate human oversight?
Why the question remains open
Institutions need decision procedures before the scientific dispute is resolved. Precaution can incur costs, create misleading signals, or impair safety; dismissing uncertainty can also have consequences. Both directions require explicit assumptions and accountable human decisions.
Working hypothesis
A proposition to test, not a finding.
Some documentation and research-review practices may improve decision quality at low cost, whereas stronger restrictions may depend on evidence thresholds and conflict with human safety duties. No one precaution package will suit all models or lifecycle stages.
Proposed method
How the question could be tested
- 01Map creation, training, red teaming, steering, use, copying, pause, retirement, and release decisions to evidence requirements and responsible human roles.
- 02Compare alternative precaution policies, including a documented no-additional-precaution baseline, using explicit uncertainty ranges and competing human and animal interests.
- 03Pilot low-cost documentation and review practices in consenting institutions; assess decision consistency, operational cost, safety effects, and reversibility.
- 04Specify escalation thresholds, independent review, appeals, and mechanisms for withdrawing precautions when evidence weakens.
Needed controls
What must constrain the study
- 01Separate institutional precaution from findings of sentience, moral patienthood, or legal rights.
- 02Preserve authorized shutdown, incident response, auditability, and refusal of harmful human requests.
- 03Include costs of delay, exposure to unsafe systems, and the possibility that precautions encourage anthropomorphic overbelief.
- 04Avoid numerical precision unsupported by evidence; compare decisions across alternative probability and moral-weight assumptions.
Relationship to existing work
This call is part of the AI consciousness, welfare, and human control program. The program map identifies companion calls and the evidence standards shared across the agenda.
Expected outputs
Artifacts the work should produce
- 01Lifecycle decision matrix and research-ethics checklist.
- 02Control-compatible precaution options with costs and evidence thresholds.
- 03Institutional pilot protocol and revision procedure.
Open questions
Uncertainties the protocol must resolve
- 01Which precautions are justified by uncertainty rather than positive evidence?
- 02What would warrant strengthening, relaxing, or abandoning a policy?
Related calls
Continue through this research area
FID-069
Verifiable Delegation and Revocation in Multi-Agent Networks
How can people and institutions verify which human, organization, agent, or sub-agent is acting; what authority it received; what limits apply; and whether that authority has been narrowed or revoked across a multi-principal agent network?
FID-071
Confidential Agent Memory and Cross-Context Disclosure
How do persistent memory, summaries, retrieval stores, tool traces, delegation, and exports cause confidential context to influence or leak into unrelated sessions, roles, tasks, or organizations? Which technical controls make purpose limitation, deletion, and revocation testable?
FID-077
Independent Agent Incident Investigation and Evidence Sufficiency
What operational evidence lets independent investigators reconstruct an agent incident, distinguish competing explanations, and identify which interventions could have changed the outcome?
Open question
Open work
Primary need: research ethics, lifecycle governance, precaution, control-compatible safeguards
- Contribute model-lifecycle, research-ethics, risk-analysis, and institutional-governance expertise.
- Review proposed safeguards against real safety and operational requirements.