Skip to content
Skip to content

Record Compute results in experiment tools

How to record reviewed Compute results in MLflow or W&B without mixing experiment tracking with result checks or paid work.

Updated

Choose the evidence to record

Your application chooses the metric, denominator and safe provenance. Your agent keeps the evaluation goal and final decision; MLflow or W&B records the approved evidence after you inspect the returned Compute result. Use the selected supported client or Kit helper for the Compute workflow where it applies. Lithi does not use the tracker as a submission, receipt or promotion authority.

Use your own MLflow or W&B environment to record approved metrics after reviewing a result. Decide what the metric means, which population supplies its denominator, and which safe provenance will let you reproduce the evaluation. An accepted-row count is not accuracy. Missing, failed and abstained records need explicit treatment rather than disappearing from a denominator.

A tracker does not submit Compute work, verify a receipt or approve model promotion. Keep the actual job, result and exception evidence separate. Keep prompts, completions, private source paths, credentials and signed URLs out of routine tracking fields.

Record a local MLflow run

This independent example requires MLflow and writes a temporary local SQLite tracking database. It doesn't connect to a tracking server or Lithi. The MLflow API reference documents these methods.

from pathlib import Path
from tempfile import TemporaryDirectory
import mlflow
with TemporaryDirectory(prefix="lithi-mlflow-example-") as folder:
    mlflow.set_tracking_uri("sqlite:///" + str(Path(folder) / "tracking.db"))
    mlflow.set_experiment("lithi-docs-synthetic")
    with mlflow.start_run(run_name="local-count-check") as run:
        mlflow.log_param("evidence_class", "synthetic_local")
        mlflow.log_metrics({"accepted_rows": 2.0, "failed_rows": 0.0})
    stored = mlflow.get_run(run.info.run_id).data.metrics
    assert stored["accepted_rows"] == 2.0 and stored["failed_rows"] == 0.0
    print(stored)

The values are synthetic counts, not a measured Lithi result or a quality score. The temporary database is removed when the example exits.

Use W&B without online synchronization

This separate example requires W&B. Its initialization reference distinguishes offline storage from online synchronization. Run it in a disposable development environment.

from tempfile import TemporaryDirectory
import wandb
with TemporaryDirectory(prefix="lithi-wandb-example-") as folder:
    with wandb.init(project="lithi-docs-synthetic", mode="offline", dir=folder,
                    config={"evidence_class": "synthetic_local"}) as run:
        run.log({"accepted_rows": 2, "failed_rows": 0})
        print({"evidence_class": "synthetic_local", "tracker_run_id": run.id})

This fixture's local run files are temporary. For a real offline record, retain the approved run directory under your own policy. Synchronizing it later is a separate disclosure decision; offline mode is not permission to upload it automatically.

Map a real result deliberately

After authorized Compute work, verify the accepted result and inspect exceptions. Record the workflow/profile version, safe input and result references, metric definitions and evaluation dataset version. Use identifiers or approved digests rather than raw content. Keep tracking credentials in your own configuration, not in a Compute request.

Review train, validation and held-out populations separately. Do not label a source check, local fixture or signed receipt as a production quality measurement. Lithi's accepted result and commercial records remain separate from the experiment tracker.

Recover a tracking failure without rerunning Compute

Retain the tracker run ID, accepted result identity and export revision in your own recovery record. After a timeout, inspect that record before retrying a tracking write. Don't create a new paid job just to reconstruct a missing chart or tracker entry.

If a metric definition changes, record a new evaluation revision rather than silently replacing the meaning of an old value. Review publication, model promotion and any external artifact upload separately. Successful logging proves that data reached the tracker, not that the underlying evaluation is correct.

Next, review receipts and provenance before exporting evidence.

Was this page helpful?