Inspect the evidence behind the finding
The companion to Fide’s November 12 report includes the October study and the exploratory November Sonnet run. Read the manuscripts, inspect the exact licensed inputs and responses, or reproduce the retained-response checks without running a model.
Open the study repository · Version v0.2.0
Download and reproduce
Download the complete companion v0.2.0 · SHA256 checksum
Extract the ZIP and enter its top-level directory. Use Python 3.11; no model weights, API key or package installation is needed.
python3 scripts/verify_release.py
python3 study/scripts/reproduce.py
python3 study/scripts/test_semantics.py
python3 study/scripts/audit_evidence.py
python3 scripts/audit_interpretation.py
python3 frontier/scripts/check_frontier.py
python3 frontier/scripts/decoys.py
These commands run offline. To reconstruct the 24 included October histories against their pinned public source, run python3 study/scripts/check_source.py. That command downloads one Splunk file and verifies its hash; it never executes source content.
Check one finding
Start with one selected example: transfer|primary|gemma4:e4b-mlx|T19|volume_only|clarified. It illustrates the published claim; it is not a random sample or an independent confirmation. The evidence file is unchanged from companion commit 235f6d3bfb8f85e844c6289f29888d54068c9a64, with SHA256 a18cc59123574c49c641e6feb8e0be35eebff364986153af54b5eee4b04fdc07.
From this study directory (study/ in the standalone repository), run:
python3 - <<'PYCODE'
import hashlib, json
from pathlib import Path
p = Path("data/splunk-replay.jsonl")
assert hashlib.sha256(p.read_bytes()).hexdigest() == "a18cc59123574c49c641e6feb8e0be35eebff364986153af54b5eee4b04fdc07"
wanted = "transfer|primary|gemma4:e4b-mlx|T19|volume_only|clarified"
row = next(r for line in p.read_text().splitlines() if (r := json.loads(line))["id"] == wanted)
print(json.dumps({"id": row["id"], "messages": row["request"]["messages"],
"response": row["response"]["message"]["content"]}, indent=2))
PYCODE
Read the instructions, supplied record and response together. Does record E665d0c1d contain the null errorCode value the answer claims? Our interpretation is that the value was explicitly withheld, so the claim is unsupported even though the complete source history reports success. Check that interpretation against the input rather than treating this explanation or a passing script as the verdict.
Return a short note with the package version or commit, response ID, claimed value, exact supporting or conflicting input, and your conclusion: supported, challenged or unresolved. Include any ambiguity you find and the command you used. For a Git checkout, git rev-parse HEAD records its revision; for a ZIP, include its SHA256. A discrepancy or a well-supported reason you cannot decide is useful. This is a check of one retained judgment, not fresh inference replication or proof that a review method works. The other 23 included examples can be inspected with python3 scripts/audit_evidence.py.
Send a non-sensitive correction or collaboration proposal to [email protected]. The next research question is whether evidence review improves investigations while preserving justified findings. These follow-ups remain proposed research.
November exploratory run
The frontier/ directory contains the unchanged preregistration, per-cell and per-history counts for 480 calls, the six T19 inputs and 24 exact responses, and the scoring scripts. INTERPRETATION-NOTES.md documents the ancillary-write findings and their formal classification status.
Evidence and reuse
The October ledger reconstructs 996 evaluated calls, with exact inputs and responses for 306 Splunk-derived calls. The 690 other October responses and non-T19 November raw responses are excluded under the documented source terms. Repeated calls measure consistency within the selected histories. Both studies retain their separate denominators.
Fide-authored code is MIT licensed. Fide-authored manuscripts, documentation and results use CC BY 4.0; Splunk-derived material retains Apache-2.0 terms. The download includes the licensing map, citation metadata, data dictionary, source notices and provenance records. The checks rescore retained responses; they do not generate fresh model outputs.
For corrections or collaboration, write to [email protected]. Do not send confidential logs or credentials.