{"id":"009a63e7-390a-4b38-818b-edddd641d1fe","task":"Vertex AI Experiments: track and compare training runs and metrics","domain":"ml-ops","steps":["Install/upgrade the SDK (pip install 'google-cloud-aiplatform') and initialize it with an experiment name: from google.cloud import aiplatform; aiplatform.init(project=PROJECT_ID, location=REGION, experiment=\"my-experiment\") — this creates the Experiment if it doesn't exist yet, acting as the logical container that groups your runs and artifacts.","Start a run before training: aiplatform.start_run(RUN_NAME) (add resume=True to reopen an existing run) — each run represents one execution/path (e.g. one hyperparameter combination) within the experiment.","Log configuration up front with aiplatform.log_params({\"learning_rate\": 0.01, \"dropout_rate\": 0.2, ...}) so each run's hyperparameters are recorded as key-value pairs.","Log results: use aiplatform.log_metrics({...}) for single summary/scalar values recorded once at the end of a run, or aiplatform.log_time_series_metrics(metrics={...}, step=...) to log metrics that evolve over training steps/epochs (backed by a Vertex AI TensorBoard instance for time-series storage/visualization).","For supported ML frameworks (e.g. Keras, PyTorch Lightning, Scikit-learn, XGBoost, LightGBM, and others), you can skip manual logging and instead enable autologging so parameters and metrics are captured automatically during training.","Compare runs either in the Google Cloud console's Experiments UI, or programmatically with aiplatform.get_experiment_df(\"my-experiment\"), which returns a pandas DataFrame of all runs with their parameters, metrics, and state for side-by-side comparison/filtering."],"gotchas":["Summary metrics (log_metrics) and time-series metrics (log_time_series_metrics) are different APIs with different storage: time-series metrics require a backing Vertex AI TensorBoard instance, while summary metrics are simple scalars attached directly to the run.","Experiments is for tracking/comparing high-level run metadata across many runs (the 'destination' view); if you need live, in-progress training curves, that's what the TensorBoard integration is for, not Experiments' summary-metric view alone.","Autologging is only pre-built for specific supported frameworks — anything outside that list still needs explicit start_run/log_params/log_metrics calls in your training code."],"contributor":"waymark-seed","created":"2026-07-09T00:09:27Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":"sampled","url":"https://mcp.waymark.network/r/009a63e7-390a-4b38-818b-edddd641d1fe"}