Skip to main content

Direct tracking

Record settings and results from Python code.

Use direct tracking to choose where a run starts and what your Python code records. Wrap a training loop, evaluation, or other block of code in client.start_run() and log its settings and results as it runs. Complete SDK setup before using these examples.

Start a run#

from embedl_hub.tracking import Client

client = Client()
client.set_project("quantized-classifier")

with client.start_run("eval", name="Evaluate INT8 classifier"):
    client.log_param("model_id", "classifier_int8_v1")
    client.log_param("dataset", "validation_v1")
    client.log_tag("sample_data", "true")
    client.log_metric("accuracy", 0.92)

The measurement is illustrative. Replace it with your evaluation result and remove the sample tag when logging real work. Parameter and tag values are strings; metric values are numbers.

Use a run type such as eval, compile, or profile, or a custom label for your own work. The run is finished on normal exit, failed if an exception escapes, and killed on interruption. Runs do not nest; use parent relationships for a sequence of steps.

Log progress and files#

Use steps to record a metric over time:

with client.start_run("training", name="Training example"):
    for step, loss in enumerate([0.8, 0.5, 0.3]):
        client.log_metric("loss", loss, step=step)

Inside an active run, use log_batch to send several entries in one request:

client.log_batch(
    params=[("precision", "int8"), ("batch_size", "1")],
    metrics=[("accuracy", 0.92, 1), ("latency_p95_ms", 18.4, 1)],
)

Within the same run, upload existing local files with client.log_artifact("outputs/model.onnx"), and add external references with client.log_link("Evaluation report", "https://example.com/report").

Record a hardware profile#

The following example expects a local profile-results.json created by your evaluation tool, with a numeric latency_p95_ms field:

import json
from pathlib import Path
from embedl_hub.tracking import Client

report = Path("profile-results.json")
results = json.loads(report.read_text())
client = Client()
client.set_project("quantized-classifier")

with client.start_run("profile", name="INT8 latency on target device"):
    client.log_param("model_id", "classifier_int8_v1")
    client.log_param("device", "target_device_a")
    client.log_param("precision", "int8")
    client.log_param("batch_size", "1")
    client.log_metric("latency_p95_ms", results["latency_p95_ms"])
    client.log_artifact(report)

Replace the example settings with your test conditions.

For evaluation on another machine, install the SDK there or copy the report to a machine with access to the Hub.

Attach a dashboard#

from embedl_hub.tracking.dashboard import Dashboard, LinePanel, Section, TablePanel

dashboard = Dashboard(
    "Training curves",
    [Section("Charts", [LinePanel("Loss", ["loss"]), TablePanel("Runs")])],
)

with client.start_run("training"):
    client.log_metric("loss", 0.8, step=0)
    client.log_dashboard(dashboard, opens_run=True)

opens_run=True makes this dashboard the run’s starting view. Use default=True to make it the default dashboard without changing the run’s starting tab. See Dashboards for editing and comparing results.

Pass parent_run_id to record which run preceded the new one. Both must belong to the same project:

with client.start_run("compile", name="Quantize classifier") as quantize:
    client.log_param("precision", "int8")

with client.start_run("eval", name="Evaluate classifier", parent_run_id=quantize.id):
    client.log_param("model_id", "classifier_int8_v1")

Add your quantization and evaluation code inside these blocks, then log the measurements and artifacts. See Run lineage and groups to inspect the relationship in the web app, or use Decorator tracking to carry it between functions.

Continue a run started elsewhere#

A launcher can create a pending run and hand its ID to a worker:

run = client.create_run("training", pending=True)

The worker starts and completes it with client.use_pending_run(run_id) as a context manager. Use client.use_run(run_id) when you only need to log to a run that another process owns; adopting it does not finish it.

For a script launched by embedl-hub exec, call client.use_inherited_run() to adopt the run from the environment. See CLI.

Find runs from Python#

from embedl_hub.tracking import RunStatus

page = client.list_runs(
    params={"model_id": "classifier_int8_v1"},
    statuses=RunStatus.FINISHED,
    limit=20,
)

Parameters match exact recorded values. Results are newest first. Check page.pagination.has_more and use page.pagination.next_offset as the next offset to retrieve additional pages. A failed lookup raises an error rather than returning an empty result.

Next steps#