Compose Nodes¶
Compose nodes prepare, load, shape, and transform data after it enters a pipeline. In SchemAlign, SQL-centered work belongs in compose patterns so integration logic remains readable and reviewable.
Use compose nodes between source and destination nodes when data must be loaded, filtered, joined, normalized, enriched, or otherwise prepared before delivery.
Compose node catalog¶
| Node | Use when | Common pipeline pattern |
|---|---|---|
| SQL Loader | Source stream data needs to become SQL-accessible working data. | File Get -> SQL Loader -> SQL Transform |
| SQL Transform | Data needs to be shaped, joined, filtered, or prepared with SQL logic. | SQL Loader -> SQL Transform -> File Put |
Choosing a compose node¶
| Compose node | Best fit |
|---|---|
| SQL Loader | Loading streamed file-like input from a source node into a working table that later SQL steps can query. |
| SQL Transform | Running SELECT or CTE-based SQL against available working data and publishing the query output for downstream nodes. |