Fide AI · Engagement brief
Evidence for your next AI decision.
Fide AI investigates AI model and agent behavior so teams can make a specific training, release, or deployment decision with evidence they can inspect.
A defined study can complement your internal evaluation team by examining a risk, comparing a change, or reviewing whether a measurement method supports the conclusions drawn from it.
Three starting points
Model and checkpoint comparison
Compare agreed model versions or mitigations under a consistent protocol. Investigate regressions, adversarial behavior, and usefulness tradeoffs, keeping development examples separate from held-out tests.
Best fit: A training, mitigation, checkpoint, or release comparison with a defined behavioral question.
Required inputs: Agreed model versions or endpoints, intended use, and permitted test access. Training data or checkpoints only where available and authorized.
Typical outputs: Comparison protocol, traceable results, failure analysis, uncertainty, and a briefing on the implications for your decision.
Agent workflow assessment
Test one consequential workflow for faithful source use, permissions, delegation, and truthful action reporting. Examine realistic exceptions, adversarial inputs, and the points where a person should intervene.
Best fit: One workflow where tools, permissions, delegation, or human approval affect consequential actions.
Required inputs: A workflow specification, authority boundaries, a safe test setting, and permitted action or effect records.
Typical outputs: Workflow and authority map, evidence-backed findings, remediation priorities, and targeted retesting where commissioned.
Evaluation and grader validity review
Examine rubrics, reference judgments, grader reliability, and disagreements. Investigate false positives and negatives, coverage gaps, and sensitivity to changes in models or operating conditions.
Best fit: A benchmark or grader that informs training, selection, release, or deployment.
Required inputs: The rubric, representative cases, grader setup, reference judgments where available, and the decision the signal supports.
Typical outputs: Measurement review, calibration analysis, explicit limitations, and recommendations for how to use the evaluation signal.
Scope and delivery
Agree the decision, intended users, model or system versions, relevant risks, available access, and technical or domain expertise. Define the protocol, outputs, test budget, schedule, and fee before work begins.
Depending on the study, outputs can include rerun materials, evidence-backed findings, uncertainty and limitations, remediation or retesting priorities, and a briefing for decision makers. A recurring evaluation cycle has explicit scope and capacity; changes to the system or risk areas may require a new scope.
Delivery
Fide leads the engagement. Technical and domain expertise, delivery responsibilities, and qualified review arrangements are agreed for each scope.
Learn about Fide. Fide’s published evaluations state their methods and limits. Results in one setting do not establish performance in another domain.
Responsibility and independence
Your team retains responsibility for development and deployment. Fide discloses relevant involvement and conflicts. Fees do not depend on favorable findings.
If Fide materially helped design or implement the subject matter, its assessment of that work is collaborative validation. An independent opinion requires an appropriately qualified assessor genuinely independent of that work and the relevant organizational conflicts. Disclosure or a different internal reviewer does not by itself establish independence.
Data access, retention, intellectual property, research reuse, and publication are agreed separately. Conclusions are bounded by the intended use, methods, and available evidence. Evaluation is not a blanket safety guarantee or certification.
Read the independence policy →Start a conversation
Start an inquiry in Google Forms ↗ or explore technical evaluation for assurance partners.
Send the system or workflow, the decision, available access, and timing to [email protected].
Engagement information and published evidence: fideai.org/evaluations/.