Evidence Collection

Use Evidence Collection to collect and review evidence as you develop and evaluate models. Connect results to requirements so your team can use them in your organization’s verification, validation, and compliance process.

For a quantized model, one requirement can have two checks: accuracy from an evaluation run and latency from a profile run on the target device. Both runs should describe the same model.

Read the submission diagram

The example requirement sets two thresholds: accuracy of at least 90% and p95 latency of at most 20 ms. Its submission contains an evaluation run with 92% accuracy, a profile run with 18.4 ms p95 latency, and a summary explaining the model choice and its tradeoffs. Both runs describe the same quantized model. Record the model, dataset, device, and settings so reviewers can assess the results.

How it works#

1. Define a requirement. Describe what the evidence must demonstrate, or link to a requirement managed in another system.
2. Add checks. Evaluate recorded metrics, parameters, or tags with rules, or describe evidence for a reviewer to assess.
3. Submit evidence. Select the relevant runs and explain how they support the requirement.
4. Review the submission. Inspect the results and test conditions, then accept the submission or request changes.

Submitting saves a numbered revision for review. If the reviewer requests changes, update the draft and submit a new revision. Earlier evidence and decisions stay available in the submission’s history.

Getting started#

Collect evidence for a quantized model using one requirement, two checks, and two runs.