Build Vector pipelines with VRL transforms to parse and route logs

Steps

1. Define a source in vector.yaml (or TOML/JSON) with a unique component ID, type (e.g., file, syslog, kafka, http), and any required options such as include paths or address
2. Add a transform of type remap; set inputs to your source ID and write a VRL program in the source field to parse, enrich, and reshape events—for example: . |= parse_json!(.message) to parse a JSON body into top-level fields
3. Use VRL's structured error handling (the ! suffix aborts on error; the ? suffix passes errors to a metadata field) to prevent a single malformed event from dropping the entire batch
4. Add a route transform to fan out events to different sinks based on field values; each route condition is a VRL boolean expression, and unmatched events go to the _unmatched output
5. Define sinks (loki, elasticsearch, aws_s3, splunk_hec, kafka, etc.) with inputs pointing at the transform or route output IDs; multiple sinks can consume the same transform output for fan-out
6. Run vector validate --config vector.yaml before deploying to check syntax and component graph integrity; run vector top in a terminal to inspect live event rates per component