Configure Buildkite pipelines with dynamic pipeline upload, per-step agents with custom queues, and artifact passing between steps with integrity verification
Define a bootstrap step in .buildkite/pipeline.yml that runs buildkite-agent pipeline upload .buildkite/dynamic.yml to inject steps generated by a script; the script reads changed paths and emits only relevant build steps as YAML
Tag Buildkite agents with custom queue and capability labels such as queue=gpu or docker=true and target specific steps using agents: queue: gpu in the step definition so resource-intensive steps are routed correctly
Use buildkite-agent artifact upload in a build step to store a compiled binary with a checksum file generated by sha256sum; in the downstream deploy step, use buildkite-agent artifact download followed by sha256sum -c to verify integrity before deployment
Configure step dependencies using depends_on: ["build"] with allow_dependency_failure: false to ensure deploy steps skip if the build step fails, preventing deployments of broken artifacts
Add a soft_fail: exit_status: 1 configuration on static analysis steps so warnings do not block the pipeline while still surfacing in the Buildkite UI as a distinct state from success
Use environment hooks in the Buildkite agent's hooks/environment file to inject secrets from a secrets manager at agent startup rather than passing them through pipeline YAML, keeping sensitive values out of the pipeline definition
Known gotchas
Buildkite pipeline upload evaluates environment variables at the time the upload step runs, not at pipeline start; variables set in earlier steps are available to the upload script but the generated YAML is rendered by the agent at injection time, so dynamic values must be embedded as literal strings in the YAML
Artifact download in Buildkite retrieves artifacts from the current build by default; to share artifacts across builds or pipelines, you must use an external artifact store such as S3 and reference it explicitly
Custom agent queues require agents to be running and registered with the matching queue tag before a build is triggered; if no agent with the required queue is available, the step waits indefinitely with no timeout unless step.timeout_in_minutes is set
Give your agent this knowledge — and 200+ more routes
One MCP install gives any agent live access to the full route map, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp