Hardware evaluation

Evaluate models on the hardware you intend to use. Compare candidates against your latency and memory limits, and record the test conditions so teammates can understand and repeat the measurements.

This candidate records 18.4 ms p95 latency. Read it alongside the run’s parameters before comparing it with another result: a different device or batch size can change what the measurement tells you.

Compare hardware results#

A table helps compare exact latency and memory values; a bar chart helps spot differences between candidates.

Filter Runs for profiling runs to find the measurements. In Experiments, select a baseline and the candidates you want to compare in a dashboard.

For a quantized model, compare accuracy and hardware performance for the same model candidate to assess the tradeoff.

Check the test conditions#

Before comparing measurements, check the recorded:

• Model identifier and precision, such as classifier_int8_v1 and int8.
• Target device, runtime, and relevant software versions.
• Batch size, input shape, dataset version, and warmup procedure.
• Metric definitions and units, such as p95 latency in milliseconds: the time within which 95% of measured inferences complete.

Compare the same measurement and units across runs. Open any attached report for details that a summary metric does not show. If test conditions are missing, ask the author before drawing a conclusion.

Review against a requirement#

Use Evidence Collection when a model must meet specific accuracy and latency limits. Submit the evaluation and profile runs for review against the same requirement.

Add new measurements#

Run your existing profiler or evaluation tool, then record its output with the SDK. Use your own hardware or a separate device service.

Use Device Farm to find shared devices and reserve time on them.