Skip to content

Pipelines and Nodes

A pipeline is a workflow made of connected nodes.

Node

A node performs one step in a pipeline. Nodes are the building blocks users place on the Workspace canvas to read, prepare, route, write, store, or send data.

SchemAlign uses three primary node categories:

  • Source nodes read from an API, database, file, service, or other upstream system.
  • Compose nodes prepare, shape, validate, branch, combine, or transform data. SQL-centered transformation logic belongs inside compose patterns.
  • Destination nodes write, store, send, or publish the result.

Current node examples include:

  • Ethos Get
  • File Get
  • HTTP Get
  • Inbound HTTP Request
  • SQL Loader
  • SQL Transform
  • File Put
  • HTTP Response

See the Node Reference for the current node catalog by category.

Terminology

SchemAlign uses node as the product term for these pipeline building blocks. Some integration tools, older notes, or technical conversations may use the word connector for the same idea. In SchemAlign documentation and user-facing screens, treat connectors as nodes and use node when naming, building, configuring, or supporting pipeline steps.

Edge

An edge connects one node to another.

HTTP GET -> SQL Transform -> Output Artifact

Edges are important because they define data flow and dependency order. SchemAlign uses directed graph edges to decide which nodes are ready to run.

Canvas position should not control execution

The visual position of a node on the canvas should not determine execution order. Only directed graph edges should control dependency and flow.

Run

A run is one execution of a pipeline.

During a run, SchemAlign tracks:

  • pending nodes
  • running nodes
  • succeeded nodes
  • failed nodes
  • skipped or blocked nodes
  • artifacts
  • published values
  • error messages

Execution and run tracking

SchemAlign uses the pipeline graph to make node dependencies explicit. Edges show which steps feed other steps, and run history should make it clear where a workflow is pending, running, completed, failed, blocked, or skipped.

HTTP GET -> SQL Transform -> Output Artifact

This graph-first model keeps execution order reviewable and makes troubleshooting easier because each node's status, configuration, output, and error details can be traced back to the step that produced them.