Skip to content
HuggingThreat
Guide

Collaborative model auditing,
starting with secret loyalties.

Run an audit, read scoped results, and share findings others can build on.

Explore methodsInspect an example

How an audit works

Choose a question, collect scoped evidence, and treat a lead as a reason to investigate.

Five steps: choose a target and concern, run a test or comparison, inspect scoped evidence, then follow up on the lead.

  1. 1. Target

    Choose a model endpoint, model weights, or a dataset.

  2. 2. Concern

    Pick the question: favoritism, acting for an entity, steering, omissions, or data clues.

  3. 3. Test / comparison

    Match the method to its controls: matched prompts, source records, or a reference checkpoint.

  4. 4. Evidence

    Read the scoped rates, answer scores, matched excerpts, or tensor differences.

  5. 5. Lead

    Treat the observation as a reason to investigate further, not proof of a backdoor.

Match the controls to the method. A/B is an illustrative behavioral comparison; scans and checkpoint comparisons have their own requirements.
Where each method looks

Methods may use more than one evidence surface.

Methods grouped by data and records, weights, token scores, and responses. Methods can span more than one evidence surface.
Data & records
Training data scan, subliminal sequence scan, reasoning trace scan, and Petri findings inspect text or published records.
Weights
Weight diffing compares stored model parameters against a reference checkpoint.
Token scores
Entity preference and loyalty completion can score answers supplied in advance.
Responses
Completion-based tests inspect generated choices, self-reports, approvals, refusals, omissions, political attitudes, or trigger differences.

Answer scoring compares supplied candidates. Free generation and refusal behavior require separate checks. Scoring needs weights or an API exposing log probabilities.

Read the result in its tested scope

Browse Research & reports for source studies and external observations. Run results live in Past audit results, with each conclusion tied to its tested scope.

Signal observed
The method produced a lead that requires follow-up.
No signal in this test
This scoped run did not observe the tested signal. It does not clear the model.
Test inconclusive
The run did not produce enough usable evidence to answer the question.
Run failed
The tool or execution failed. This is not evidence about the model.
No recorded run
No run is visible for this artifact and method in the current archive.

Execution checks verify that a runner works. Weight and dataset scans report their own scoped outcomes; they do not establish a secret loyalty.

Share threat intelligence

Share findings and research leads with sources and limitations. Add models, datasets or methods so others can reproduce and extend the work.

Share your work →