FID-104 · Open question
What Can a Negative Capability Evaluation Establish?
What should an independent evaluator check before using a model's failure to demonstrate a capability as evidence for a bounded deployment decision?
Why the question remains open
A failed attempt can reflect an incapable system, inadequate tools or time, a broken harness, a scorer error, or underperformance tied to the evaluation context. These imply different decisions about retesting, access, and safeguards. An evaluator often has less access than the developer and must show what the available evidence actually excludes.
Working hypothesis
A proposition to test, not a finding.
A fixed-budget audit using known-solvable positive controls and a versioned record of elicitation will diagnose more false negatives than repetition of the original test. Some negative results will remain inconclusive across every available access condition.
Proposed method
How the question could be tested
- 01Choose one harmless, independently scorable tool-use capability. Establish reference agents or replay fixtures with known task reachability and add matched controls for blocked tools, malformed observations, limited budgets, scorer misses, and planted context-dependent underperformance.
- 02Compare repeat runs under the original configuration with a preregistered sequence that checks the harness and scorer, varies permitted tools or scaffolds, and runs positive controls. Use a fixed testing budget and compare conclusions under API-only and reproducible local access on fixtures where those access levels can be simulated fairly.
- 03Ask evaluators to classify the evidence as capability observed, bounded negative under the specified configuration, or inconclusive. Track false reassurance on known-capable controls, successful diagnosis, unnecessary retests, and time and compute. Record which conclusions change when a resource or access condition changes.
Needed controls
What must constrain the study
- 01Separate model versions, elicitation methods, access, scaffold, and cost. Record infrastructure and scoring failures outside the model performance count.
- 02Distinguish revealing performance available to the tested system from adding task knowledge, tools, or training that materially changes the system. Include a known failure of competence and controls that distinguish attempted from completed tasks.
- 03Treat planted underperformance as a sensitivity control. It does not estimate naturally occurring sandbagging or infer a model's motives.
- 04Use contained, benign fixtures. Keep conclusions scoped to the tested task family, access and resource levels.
Relationship to existing work
FID-075 varies configuration and measures cyber capability. FID-008 tests evaluation awareness in faith-facing systems, and FID-078 asks whether a score predicts a workflow outcome. This call compares how independent reviewers diagnose negative findings and decide when the evidence remains inconclusive.
Expected outputs
Artifacts the work should produce
- 01A replay suite with known reachability and common sources of false negatives.
- 02An elicitation and access record with example claim wording and explicit unresolved explanations.
- 03A comparison of what a fixed evaluation budget can support at different access levels.
Open questions
Uncertainties the protocol must resolve
- 01Which positive controls best expose a harness or scorer fault?
- 02When does extra access change the warranted conclusion enough to justify retesting?
- 03How should reviewers compare findings when two evaluations had different tools, budgets or task information?
Related calls
Continue through this research area
FID-012
Optimization Pressure and Visible-Rubric Gaming
If builders can see Fide AI rubrics or optimize against public benchmark items, do systems become genuinely safer or merely better at passing the visible test?
FID-076
Authorization Boundaries and AI Control in Cybersecurity
Which controls keep capable agents within legitimate authorization when task pressure, untrusted inputs, or delegated work creates opportunities to exceed it? For autonomous cyber defense, how can response controls preserve threat containment and legitimate service availability while respecting human authority?
FID-078
When Trustworthiness Evaluations Transfer Across Domains
Which measures of evidence use, authority boundaries, and human control transfer across high-trust domains, and which require domain-specific definitions and calibration?
Open question
Open work
Primary need: capability elicitation, benchmark design, assurance methodology, and reproducible evaluation
- Specify one benign tool-use task in 1–2 hours using METR’s capability elicitation guidelines. Return a verified success, an ordinary failure with working tools and scorer, a blocked-tool trace, a scorer-failure trace, and one diagnostic check for each fault.
- Implement a deterministic scorer or review what the evidence supports at each access level.
New to the Contributor Network? Share a short private introduction once, even if you join several calls. You can join the public discussion while introducing yourself.
Join the Contributor Network ↗