Build a versioned DVC pipeline (dvc.yaml) that chains data prep, featurization, and training stages with cached, reproducible runs

domain: doc.dvc.org · 5 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗

Steps

  1. Track raw input data with `dvc add`
  2. Create each stage with `dvc stage add -n <name> -p <params> -d <deps> -o <outs> <command>`, chaining outputs of one stage as dependencies of the next
  3. Commit the generated `dvc.yaml` (and `.gitignore` entries) to Git
  4. Run `dvc repro` to execute the pipeline; DVC skips stages whose dependencies/params haven't changed
  5. Commit the resulting `dvc.lock` file, which records hashes of deps, params, and outputs

Known gotchas

Related routes

Configure a DVC pipeline stage with an external S3 dependency and output that DVC does not cache locally
doc.dvc.org · 5 steps · unrated
Configure Buildkite pipelines with dynamic pipeline upload, per-step agents with custom queues, and artifact passing between steps with integrity verification
Buildkite · 6 steps · unrated
Amazon SageMaker Pipelines: build a pipeline with Processing, Training, and RegisterModel steps using the SageMaker Python SDK
ml-ops · 6 steps · unrated

Give your agent this knowledge — and 15,500+ more routes

One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus: claude mcp add --transport http waymark https://mcp.waymark.network/mcp

Need this verified for your stack — or a route we don't have yet?

We author + individually verify a route for your exact task within 24h. Custom route — $25 · Teams: Pilot — $750/mo · all plans