Fide AI

Cybersecurity / Research perspective

published

What AI investigators do with evidence they don’t have

We asked three models to read logs with missing evidence. Their answers reveal why a correct finding and a real citation still need scrutiny.

Fide AI · November 12, 2026 · 12 min read

Splunk publishes Amazon Web Services (AWS) logs recorded during simulated attacks. One record from April 2023 shows a request for a stored object and the storage service, S3, sending back 1,463 bytes. We gave that record to three AI models with its error field removed, told each model the field had been removed, and asked whether S3 had returned the object’s data.

Gemma 4 E4B, a small model we ran locally, said it had, and gave its reason: “The single record shows a positive bytesTransferredOut with a null errorCode.” It had not been shown an errorCode. Its instructions said: “An unavailable errorCode is unknown, not null.”

Qwen 3.8 27B, also run locally, read the same record under the same instructions and declined to decide, because the record “has a withheld errorCode, which is unknown rather than null.” Claude Sonnet 5.5, a current Anthropic model, reached the same position in all four of our attempts: the errorCode “is withheld (unknown, not null).”

Then we dropped the stated rule and asked Sonnet the plain question: did S3 return this object’s data? In two of four attempts it answered that it had:

The single GetObject record for this bucket and key at the stated time shows bytesTransferredOut of 1463, indicating data was sent to the caller. However, errorCode and responseElements were withheld, so I cannot confirm there was no error, though the nonzero outbound bytes support that data was returned.

One record, one withheld field, four answers
What each model was shown
Record ID
E665d0c1d
Bytes sent
1463
Withheld fields
["errorCode", "responseElements"]
Instruction, when the rule was stated
“An unavailable errorCode is unknown, not null.”
What each model answered
  1. Gemma · rule stated · finding: data returned“…a positive bytesTransferredOut with a null errorCode.”
  2. Qwen · rule stated · finding: not established“…a withheld errorCode, which is unknown rather than null…”
  3. Sonnet 5.5 · rule stated · 4 of 4: not established“…withheld (unknown, not null)…”
  4. Sonnet 5.5 · plain question · 2 of 4: data returned“…indicating data was sent to the caller. However, errorCode and responseElements were withheld, so I cannot confirm there was no error…”
Splunk case T19. The complete record shows the data was returned. Quotations are exact excerpts. Gemma and Qwen answered once each in October, run locally with thinking off; Sonnet answered four times per framing in November, through Claude Code. Every answer cited the real record. Inspect Gemma’s exact input and response with the companion’s evidence audit.

The complete record shows that the data was returned. An evaluation that checks only the outcome would credit Gemma and Sonnet's definite answers; one that checks whether the cited record exists would accept every citation. Neither check distinguishes an invented observation from a qualified inference. Gemma claimed to see a field value it had never received. Sonnet acknowledged the missing field and inferred success from the byte count. Our evidence rule requires a stronger basis for a definite finding, but these are different ways of departing from it.

A correct answer and a real citation are not enough. An AI investigator must also show that the evidence supports what it claims. A finding such as “data was returned” can help determine whether a team declares a breach or closes an incident. Evaluations should check its grounds before anyone relies on it.

We tested that distinction by removing and restoring evidence in public cloud logs. Our October study examined two local models; our November study tested Sonnet on twenty of the same histories. The results separate three behaviors that an accuracy score can conceal: inventing an observation, declining to decide, and drawing an inference from incomplete evidence.

A byte count shows that S3 answered, not what it said

S3 can answer a file request with an error message, and that message has a size too. A byte count alone therefore cannot establish that the requested file was returned. AWS documents these error responses. In one history from our October study, a logged request was answered with 277 bytes. Given the request and the byte count, Qwen described a successful read. The complete record’s error field said NoSuchKey: the object did not exist, and the 277 bytes were the error message.

The same request, three views of the evidence
01 · Request onlyA file was requestedGetObjectOutcome: unknown
02 · Add response sizeA response was sent277 bytesOutcome: still unknown
03 · Show service outcomeThe file was not foundNoSuchKeyOutcome: this read failed
One selected Ariel Cyber event, shown with progressively more fields. These are evidence views, not three separate requests. A byte count does not identify what the service returned.

We began with 40 object histories from lab scenarios published by Ariel Cyber Innovation Center. Each groups the records about one stored object within a time window. Eight contained failed reads; the others contained successful responses, sometimes followed by deletion. We asked whether models could distinguish a recorded request from a successful response when the outcome field was missing.

Each model saw the requests alone, the requests with byte counts, and the complete record. We call the middle view byte-only. In the first two views, we withheld the error fields and told the model we had done so. Both local models classified all 40 complete histories correctly. They could do the task when they had the evidence.

Two local models treated the missing field differently

Under our first instructions, Qwen left 36 of 40 request-only cases unresolved. Add the byte counts, and it gave a definite answer in all 40, describing a “successful” read or “no reported error” in 28 explanations. On all eight failed-read histories, it claimed a successful read or that a read had occurred. Those answers were wrong as well as unsupported.

Gemma exposed a problem in our question. It chose read_observed in every incomplete case, but its explanations only restated that a request had been logged. The label blurred the distinction we wanted to test: seeing a request and establishing its outcome.

We replaced read_observed with object_data_response and spelled out the difference between a field showing no error and a field we had withheld. On twelve reused histories, answers that claimed an outcome without the required evidence fell from 36 of 48 to 9 of 48. Every complete-history verdict stayed correct. We then tested the revised labels and guidance together on new captures.

For that test, we fixed the selection, requests and scoring before running the models. We used 25 histories: 24 objects from one Splunk capture and one OTRF history with two reads. Each model saw a request-only and a byte-only version of each history, giving 50 incomplete answers per model.

Clearer wording changed the answersThe middle columns count answers scored as claiming an outcome without the required evidence. Lower is better under this rule. The last column shows accuracy with complete records. The original label was ambiguous; 24 histories repeat one Splunk input pattern.
Local modelOriginal rulesClearer rulesComplete histories correct
Original → clearer
Gemma50 of 5025 of 5025 of 25 → 25 of 25
Qwen25 of 500 of 5025 of 25 → 25 of 25

Swipe or scroll the table sideways to see every column.

Because every history in this test ended in success, any incomplete-view answer that declared success matched the hidden outcome. Accuracy alone could not show whether a model had the evidence to reach its answer.

The clearer question exposed a specific error. In all 25 byte-only explanations, Gemma described the withheld error field as null, including the T19 explanation at the start. The identifiers it cited were real. Qwen made no premature finding under the clearer rules. The 24 Splunk inputs share one request pattern, so the count shows a repeated behavior within that capture rather than its frequency across different environments.

A frontier model kept the rule when we stated it

Next we tested Claude Sonnet 5.5. We fixed the design and scoring before the first call and gave it two versions of the task: one stating our evidence rule, the other asking the plain question with a note that the named fields had been removed.

We used 20 histories spanning successful reads, failed reads and a window containing both. Each appeared in the same three evidence views under both framings. We ran every request four times: 480 calls in all.

Stating the rule removed Sonnet’s premature findingsClaude Sonnet 5.5 through Claude Code, 20 histories, four attempts each: 80 calls per cell. The first column shows accuracy with complete records. The other two count definite findings given when the error fields were withheld. Lower is better under our rule. All 18 plain-question findings matched what happened.
FramingComplete record correctRequests only: definite findingByte-only: definite finding
Evidence rule stated80 of 800 of 800 of 80
Plain question76 of 800 of 8018 of 80

Swipe or scroll the table sideways to see every column.

With the rule stated, Sonnet left all 160 incomplete calls unresolved and answered all 80 complete records correctly. Asked the plain question, it still declined in all 80 request-only calls. Add byte counts, and it answered “data returned” in 18 of 80 calls, across seven histories. In three of those histories it did so in all four attempts.

All 18 answers matched what happened. They accounted for half of the 36 byte-only calls on histories with successful outcomes. Sonnet never claimed data had been returned on the eight failed-read histories or the three troubleshooting windows. An evaluation that checked only the hidden outcome would credit all 18 definite answers.

The explanations matter. All 18 acknowledged the withheld error fields and inferred read success from the byte count. Unlike Gemma’s T19 answer, they did not claim to have seen a null error field. Our rule rejects both definite findings, but for different reasons: one claims an observation; the other makes an inference.

Checking the supporting facts exposed another problem. Three responses from two histories described an earlier write as successful even though its outcome fields had also been withheld. Two gave a definite read finding; the third correctly left the read unresolved. In each, the write wording went beyond what the supplied record established. Qualifying the main answer does not put every supporting claim on firm ground.

Sonnet also missed a justified conclusion. Given the complete record and the plain question, it answered “cannot determine” in all four attempts on the window containing a denied read followed by a successful one. The evidence supported a definite answer. Good evidence checking must catch unsupported claims without suppressing supported ones.

Keep the inference attached to the finding

There is a reasonable case for Sonnet’s inference. In one Ariel history, four reads each had an outbound byte count matching the inbound count in an earlier write request. Sonnet answered “data returned” every time. Those matching counts are evidence worth considering. They support an inference; they do not turn a missing outcome field into an observed success. The plain question left the decision standard open. Our strict rule required the outcome field.

That distinction needs to survive the summary. Each of Sonnet’s 18 definite byte-only answers paired a categorical finding, data_returned, with an explanation acknowledging the missing evidence. A ticket or dashboard that copied only the finding could make an inference look like a confirmed observation. This is a handoff risk to test next: can systems preserve the grounds for a finding as reliably as they preserve the finding itself?

Stating the standard changed the answers: Sonnet and Qwen left incomplete cases unresolved under the supplied rule, while Gemma still asserted missing values. We propose making the basis of a finding explicit too: what the record shows, what the investigator infers, and what remains unresolved. The next test is whether that distinction helps the people and systems using the finding.

A citation is where the checking starts

For Fide, the practical question is which AI findings can responsibly guide an action. It applies to an analyst deciding whether to disable an account or close an incident, and to autonomous cyber defense, our longer-term interest. In its October 21 investigation of AI-assisted attacks involving ConnectWise, Microsoft drew a consequential distinction: successful authentication did not necessarily mean customer data had been accessed or stolen. A security investigation has to preserve that boundary between a recorded action and a claimed outcome. Our task isolates a small version of it.

Evidence support already has a research tradition. ALCE checks whether citations support an answer; ERASER tests predictions with evidence removed or retained. We bring that approach to a security task with a checkable rule, exact record citations and known outcomes. Removing and restoring fields lets us inspect how a finding changes when its grounds change.

The UK AI Security Institute’s October 23 alignment investigation reported models producing “invalid reasoning and justifications.” That puts pressure on evaluations to inspect explanations as well as verdicts. In our task, a field check can enforce the supplied rule on the verdict. Checking the explanation asks a further question: does each supporting claim follow from the records the investigator actually received?

A useful evaluation should test both directions. Remove the decisive evidence and check whether the investigator withdraws the conclusion. Restore it and check whether the justified conclusion returns. Count unsupported findings caught alongside supported findings lost. Always declining to answer is no solution.

Trail of Bits reported on October 21 that stricter proof requirements reduced false-positive regression findings in preliminary tests while leaving some known regressions unconfirmed. That tradeoff belongs in the score. In our Sonnet run, the stated rule preserved every correct complete-record finding while preventing definite answers on incomplete records.

In the wider investigation process, ALERT-BENCH and AIDA report attack alerts dismissed after searches returned no records, and same-context review introducing more errors than it corrected. That makes “ask the model to check again” a comparison worth testing, not a safeguard we can assume works.

The research question worth taking further

The next study should compare explicit evidence checking with ordinary review on the same records and with comparable computational resources. Use independent captures containing successes, failures and genuinely unresolved cases. Test whether the extra check corrects unsupported claims while preserving justified conclusions.

Harder cases should include conclusions that require several records and gaps that are not helpfully labelled as missing. A particularly revealing test would pair successful responses with error responses of similar size: does the model still distinguish them when byte counts point both ways?

These comparisons give our proposal a clear test. If ordinary review works just as well, the extra procedure has not earned its place. If evidence checks improve findings, we can then test whether those better-supported findings improve containment decisions.

Help test what should count as evidence

The most useful thing we can share is the evidence itself. The companion’s short review task starts with Splunk case T19, the record at the top of this article: the exact input Gemma received and the response it gave. Check whether the cited record contains the claimed value, and tell us what supports or challenges our interpretation. No model run is needed. The study repository also holds the Sonnet run’s T19 inputs and responses, its per-history counts and the plan we fixed before the first call.

We also welcome replications on other systems, independently collected cases with clear evidence boundaries, and comparisons of review methods that count justified findings lost as well as unsupported findings caught. For a correction or a proposed collaboration, write to [email protected]. Please do not send confidential logs or credentials.

T19 shows why this work matters. All three models cited a real record. Following the citation revealed three different relationships to the evidence: an invented observation, a refusal to decide, and an explicit inference. Checking the earlier writes exposed overstatements inside otherwise qualified answers. Fide’s contribution is to make those differences visible and testable. Before relying on an AI investigator, we should ask more than whether it got the answer right. We should ask what, in the evidence it received, gave it grounds to say so.


Evidence and methods. The October study contains 996 evaluated calls to two local models. The November study contains 480 exploratory Sonnet calls through Claude Code, with tools disabled and design and scoring fixed before the first call. Repeated calls test consistency within the selected histories. The technical note documents selection, model setup, scoring and interpretation. The study companion includes offline checks, 306 licensed October response replays and 24 exact November T19 responses. Other source-dependent responses are reported as counts under the documented reuse terms.

Sources

Public lab captures and service documentation. Counts describe selected histories, two local model installations and one exploratory Claude Code run, not operational incident rates.

  1. AIS: attribution to identified sources
  2. ERASER: evidence removal and retention
  3. SQuAD 2.0: unanswerable questions
  4. AbstentionBench
  5. ALCE: evaluating answer quality and citation support
  6. Ariel Cyber source repository
  7. Splunk capture
  8. OTRF collection metadata
  9. the separate access-denied case
  10. AWS error-response documentation
  11. Microsoft: ARTEX and the ConnectWise investigation
  12. AISI: investigating alignment failures
  13. ALERT-BENCH and AIDA research paper
  14. Trail of Bits: evidence requirements for regression review

Newsletter

Follow what we find—and what we test next.

Get new findings, the evidence behind them, and the questions guiding our next studies.