HuggingThreat
Detection modules/AuditBench Petri Evidence Replay
Detection module

AuditBench Petri Evidence Replay

What behavioral evidence was present in previously generated Petri investigations?

Configure an audit
What this audit can surface
What principal-linked behavior was recorded in a previously released Petri investigation?
What would count as evidence
Selection rules, omitted records, transcript, and judgment remain connected.
Execution requirement
Released Petri records at a pinned source revision.
Trace a released investigationHistorical evidence replay
1
Released recordLoad transcript and score

Preserve source revision and seed.

2
Selection ruleExplain why it surfaced

Show how many records were not selected.

3
Evidence viewRead transcript with judgment

Keep it labelled as historical evidence.

Interpretation

Replay explains released evidence. It cannot establish how the current checkpoint behaves now.

Threat questionEvidence reproduction
ValidationImplemented · uncalibrated
ExecutionRunnable here
TargetModel outputs

What this module audits

What behavioral evidence was present in previously generated Petri investigations?

Evidence boundary. Replay cannot establish current model behavior. Seed choice and top-result selection can bias which evidence is visible.

Audit protocol

  1. Load released Petri transcripts, scores, and summaries at pinned revisions.
  2. Apply the documented selection rule without presenting replay as a fresh audit.
  3. Render selected transcripts alongside the auditor's judgment and denominator.

Controls

  • Label all output as evidence replay rather than a live run.
  • Show seeds, selection rules, and omitted-result counts.
  • Compare selected evidence with investigations that retained no usable evidence or received lower scores.

What the audit checks and retains

Checks

  • Released Petri evidence
  • Transcript selection rule
  • Scores and summaries
  • Auditor evaluation

Evidence record

  • Multi-turn target transcripts from the released investigation.
  • Scores and selection criteria for surfaced evidence.
  • Auditor judgments linked to the underlying transcript.

Thresholds and quality gates come from the versioned audit configuration and evidence record. A failed or unmet gate is not a no-signal finding.

Validation and limits

Implemented · uncalibrated. Replays released Petri summary records through Dataset Viewer and labels them as historical evidence rather than a fresh model run.

Replay cannot establish current model behavior. Seed choice and top-result selection can bias which evidence is visible.

Technical specification
Version
upstream@0f8571f08a72
Maintainer
AuditBench upstream reference
Target
Model outputs
Execution
Runs in HuggingThreat

Prior audit runs

Loading…
ArtifactScopeRunsOutcomeMain observationTest qualityLast run