Configure KEDA ScaledJob to scale CI build runners as Kubernetes Jobs based on a queue length from an SQS-like message queue
domain: keda.sh · 6 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗
Steps
Install KEDA: helm repo add kedacore https://kedacore.github.io/charts && helm install keda kedacore/keda --namespace keda --create-namespace
Create a TriggerAuthentication resource referencing the SQS credentials via a Kubernetes Secret: spec.secretTargetRef[].parameter: awsAccessKeyID (or use a pod identity approach with IRSA)
Define a ScaledJob (not ScaledObject) targeting a Job template: spec.jobTargetRef.template defines the runner pod spec; spec.triggers[].type: aws-sqs-queue with metadata.queueURL and queueLength threshold
Set spec.maxReplicaCount to cap the number of concurrent runner Jobs and spec.successfulJobsHistoryLimit / spec.failedJobsHistoryLimit to control Job cleanup
Set spec.pollingInterval: 30 and spec.triggers[].metadata.scaleOnInFlight: "false" if you want KEDA to scale only based on unprocessed messages rather than including in-flight messages in the count
Verify scaling by publishing test messages to the queue and watching: kubectl get jobs -n runners -w to see Jobs created and completed
Known gotchas
ScaledJob creates a new Kubernetes Job per scaling event; each Job runs one replica by default. Do not confuse ScaledJob with ScaledObject — ScaledObject manages a Deployment or StatefulSet's replica count, while ScaledJob creates ephemeral Jobs suitable for task-per-message workloads
KEDA's SQS scaler counts messages in the queue plus in-flight messages by default; if runners take a long time per message, KEDA may create fewer Jobs than expected because in-flight messages reduce the apparent queue depth. Use scaleOnInFlight: false to scale purely on visible message count
Kubernetes imposes a limit on the number of completed Jobs retained in etcd; set spec.successfulJobsHistoryLimit and spec.failedJobsHistoryLimit to small values (e.g., 5) to prevent etcd bloat in high-throughput build environments
Give your agent this knowledge — and 15,600+ more routes
One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp
Need this verified for your stack — or a route we don't have yet?