SQL Loader¶

Description¶

SQL Loader is a compose node that consumes a stream produced by an upstream node and loads the data into a SQL-accessible working table. Use it when a source node retrieves file-like content that needs to be staged before SQL Transform, validation, enrichment, or delivery.

SQL Loader expects an upstream stream payload. It does not call an external system itself; it converts incoming stream bytes into a table inside the pipeline execution context.

Field reference¶

Field	Description
Input producer node (optional)	Selects which upstream producer to consume when the SQL Loader has more than one upstream input. Leave blank when there is only one upstream producer. Accepted values include graph node id formats such as `5`, `node_5`, or `node:5`.
Target table	Required. The working table name that the incoming stream should be loaded into. Downstream SQL Transform nodes can query this table after the load succeeds.
If exists	Controls what happens when the target table already exists. `replace` recreates or overwrites the table contents, `append` adds rows to the existing table, and `fail` stops the node if the table already exists.
Create table if missing	When enabled, SchemAlign creates the target table if it does not already exist. Disable this when the table must already be defined.
Column mode	Controls how columns are handled during load. `replace` allows the load to align the table structure to the incoming data. `preserve` keeps the existing table structure when possible.
Input format	Determines how the incoming stream should be parsed. `auto` infers the format from upstream metadata or filename; supported explicit values are `csv`, `json`, and `xml`.
CSV delimiter	Delimiter used when parsing CSV-like input. Common values include comma, tab, pipe, and semicolon. Escaped values such as `\t` are normalized before loading.
CSV header row	Indicates whether the first row of CSV-like input contains column headers.
JSON mode	Controls how JSON input is interpreted. `records` is for row-oriented JSON data; `raw` keeps the JSON structure more directly when supported by the loader.
JSON root (optional)	Optional root path or key used to locate row data inside a larger JSON payload.
NDJSON	Treats JSON input as newline-delimited JSON records when enabled.
XML row XPath (optional)	Optional XPath-style row selector used to identify repeating row elements in XML input.

When to use SQL Loader¶

Use SQL Loader when a source node produces data that should become queryable working data. Common patterns include:

File Get -> SQL Loader -> SQL Transform
HTTP Get -> SQL Loader -> SQL Transform
Inbound HTTP Request -> SQL Loader -> SQL Transform

Skip SQL Loader when the upstream node already publishes a database table handle that SQL Transform can query directly.

Inputs and outputs¶

SQL Loader requires one upstream stream payload. If the node has multiple upstream producers, configure Input producer node (optional) so the loader knows which stream to consume.

When the load succeeds, SQL Loader reports the working schema, target table, loaded table reference, detected or selected format, rows loaded, column mode, and upstream stream metadata. Downstream compose nodes can use the loaded table in SQL logic.

Format handling¶

When Input format is set to auto, SchemAlign first checks the upstream stream format metadata. If that is not available, it falls back to the upstream filename extension. CSV-like extensions are treated as CSV, JSON and NDJSON extensions are treated as JSON, and XML extensions are treated as XML.

Use an explicit format when the upstream filename or metadata is ambiguous.

Validation and common errors¶

Message or condition	What it means
`sql_loader has multiple upstream producers and requires config.input_producer_id`	More than one upstream node can feed this loader. Set Input producer node (optional) to the node id that should be consumed.
`sql_loader expected upstream stream bytes but received no payload`	The selected upstream node did not provide stream bytes. Check the upstream node output and graph connection.
`sql_loader requires config.target_table`	Target table is required.
Pipeline execution missing tenant context	The node did not receive the execution role or data schema required for tenant-bound loading.

Security notes¶

SQL Loader writes to tenant-bound working data using the execution context assigned to the pipeline. Use clear table names and avoid loading sensitive data into broadly reused tables unless the downstream workflow requires it.