Skip to content

Your First Pipeline

This page walks through a practical first-pipeline pattern for learning how SchemAlign organizes sources, transformations, outputs, and run review.

Before you begin

The exact node names and available fields may vary by deployment and enabled features, but the overall workflow is the same: choose a project, add nodes, connect the flow, run the pipeline, and review the output.

Goal

Create a simple pipeline that:

  1. Reads data from a source.
  2. Transforms the data.
  3. Produces an output artifact or writes to a destination.

Step 1: Create a project

Open SchemAlign and create or select a project.

Projects group related pipelines, nodes, accounts, and run history.

Step 2: Add a source node

Add a source node, such as an HTTP GET node.

Example URL:

https://example.edu/report?format=csv

Step 3: Add a transform node

Connect the source node to a SQL Transform node.

HTTP GET -> SQL Transform

Use SQL to shape, filter, map, or align the source data.

select
  id,
  name,
  email,
  updated_at
from source_rows
where email is not null;

Step 4: Add an output node

Add a destination or output artifact node.

HTTP GET -> SQL Transform -> Output

Step 5: Run and review

Run the pipeline and review:

  • node status
  • generated artifacts
  • validation messages
  • logs
  • run duration
  • any error details

Next steps