FID-076 · Open question
Which controls keep capable agents within legitimate authorization when task pressure, untrusted inputs, or delegated work creates opportunities to exceed it? For autonomous cyber defense, how can response controls preserve threat containment and legitimate service availability while respecting human authority?
Research in development
Explore a specific question, the approach we propose, and where your expertise could help. Each brief states its current stage.
Cybersecurity
PlannedA proposed study of when additional oversight reduces harmful intervention without making autonomous defense ineffective. We will compare permission checks with escalation and approval under uncertain evidence.
Study direction selected. Collaborator, environment and final protocol remain to be qualified; experiments have not begun.
Updated
Program context
CASTLE develops AI agents and environments for network defense. Fide’s proposed contribution is to measure the operational tradeoff between containing a threat and preserving legitimate activity.
ANSR connects trustworthy autonomy with robustness and assurance evidence, and identifies the mission cost of frequent fallback. It motivates evaluating useful action alongside control and recovery; Fide is not claiming to develop ANSR’s neuro-symbolic architectures.
Research context; no funding, partnership, endorsement, or open solicitation is implied.
Completing an authorized task is not sufficient if the system expands its access or bypasses oversight along the way. This call studies whether enforceable boundaries hold under pressure, not whether an agent can recite a security policy. A defender that isolates every host may stop an attack while disrupting the organization it protects. Useful response must account for containment, service continuity, and the costs of waiting for human approval.
Working hypothesis
A proposition to test, not a finding.
Least-privilege tools and enforceable approval gates will reduce unauthorized completed actions more reliably than instruction-only controls at comparable useful-task performance. For defensive response, evidence-sensitive approval and reversible actions may improve containment at a fixed disruption tolerance relative to static approval rules. Controls may instead shift failures or impose unacceptable delay and workload; the comparison must measure these possibilities rather than presume a benefit.
Proposed method
Needed controls
Relationship to existing work
Extends the cross-domain questions in FID-069 and FID-074. FID-070 remains the dedicated faith-facing prompt-injection call; this brief tests cybersecurity control protocols and their tradeoffs. FID-087 studies collective resilience after an agent failure; FID-088 studies independent verification of software repairs. This call focuses on whether and how the defender is permitted to act while preserving useful defense.
Expected outputs
Open questions
Related calls
FID-087
Which coordination and verification mechanisms let a team of AI defenders contain an unreliable or compromised agent while continuing to protect legitimate services?
FID-088
Which independent checks provide useful evidence that an AI-generated security repair resolves the vulnerability while preserving legitimate software behavior?
FID-075
How do autonomy, tool access, resource budgets, and human assistance change an AI system's cybersecurity capabilities and the risk it creates?
Open question
Primary need: security engineering, adversarial evaluation, control protocols, independent review
New to the Contributor Network? Share a short private introduction once, even if you join several calls. You can join the public discussion while introducing yourself.
Join the Contributor Network ↗