Amazon SageMaker Pipelines: build a pipeline with Processing, Training, and RegisterModel steps using the SageMaker Python SDK
domain: ml-ops · 6 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗
Steps
Set up a PipelineSession (from sagemaker.workflow.pipeline_context import PipelineSession) alongside your usual boto3/sagemaker.Session and execution role — pipeline step definitions use this session instead of the eager session so calls are deferred into the pipeline graph.
Define a Processing step: create a processor (e.g. SKLearnProcessor) with sagemaker_session=pipeline_session, call processor.run(inputs=[ProcessingInput(...)], outputs=[ProcessingOutput(output_name="train", ...), ...], code="preprocessing.py") to get step_args, then wrap it in ProcessingStep(name="...", step_args=step_args) from sagemaker.workflow.steps.
Define a Training step: configure an Estimator (e.g. XGBoost via sagemaker.estimator.Estimator) with sagemaker_session=pipeline_session, call estimator.fit(inputs={"train": TrainingInput(s3_data=step_process.properties.ProcessingOutputConfig.Outputs["train"].S3Output.S3Uri, ...)}) to get step_args, then wrap in TrainingStep(name="...", step_args=step_args) — note how the training input references the processing step's output property, wiring the DAG dependency automatically.
Define a model-registration step with sagemaker.workflow.step_collections.RegisterModel, passing the trained estimator, model_data=step_train.properties.ModelArtifacts.S3ModelArtifacts, content_types, response_types, inference_instances/transform_instances, and a model_package_group_name so runs accumulate into one Model Registry group.
Assemble everything with from sagemaker.workflow.pipeline import Pipeline; pipeline = Pipeline(name=..., parameters=[...], steps=[step_process, step_train, step_register]); optionally validate with json.loads(pipeline.definition()) before creating/starting the pipeline.
Optionally wrap the RegisterModel step in a ConditionStep (sagemaker.workflow.condition_step) so registration only runs if an evaluation metric (e.g. MSE from a model-evaluation Processing step) meets a threshold.
Known gotchas
AWS's own docs flag that RegisterModel (and CreateModelStep) are legacy: as of SageMaker Python SDK v2.90.0, AWS recommends using the newer 'Model step' for registering/creating models instead — RegisterModel still works but is no longer actively supported/enhanced.
Step outputs must be referenced through .properties (e.g. step_process.properties.ProcessingOutputConfig.Outputs["train"].S3Output.S3Uri) rather than plain Python variables, since these resolve at pipeline-execution time, not at pipeline-definition time.
A given step object can only appear once across the pipeline's step list or a ConditionStep's if_steps/else_steps — it cannot be referenced in both.
Give your agent this knowledge — and 15,500+ more routes
One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp
Need this verified for your stack — or a route we don't have yet?