Publishing Data

SAL can publish the same built data product in two ways:

• sal push packages .sal/data as an OCI artifact.
• sal upload deploys .sal/data to an object store bucket.

Build locally first with sal build.

Push to an OCI registry

Use sal push when you want a portable artifact that records the source Git repository and commit hash:

sal push ghcr.io/my-org/my-data-product --username "$OCI_USERNAME" --password "$OCI_PASSWORD"

The pushed artifact is tagged as latest and with the current Git commit hash. SAL adds OCI annotations that connect the artifact back to the source repository and commit.

Consumers can use sal clone to clone the source repository, check out the pinned commit, initialize SAL, and restore the data product into .sal/data.

Upload to object storage

Use sal upload when readers should query the Iceberg table directly from an object store:

sal upload --bucket gs://my-bucket/sal/triples

Supported object-store URL forms include gs://, s3://, azblob://, and Google Cloud Storage HTTPS URLs such as:

sal upload --bucket https://storage.googleapis.com/my-bucket/sal/triples

During upload, SAL stages a temporary copy of .sal/data, rewrites Iceberg metadata paths to the destination, and uploads the staged files. The local .sal/data directory is left unchanged.

Query a deployed table

After uploading to object storage, query the table with DuckDB’s Iceberg extension:

SET enable_object_cache = false;

SELECT subject, predicate, object
FROM iceberg_scan('gs://my-bucket/sal/triples')
LIMIT 5;

For more detail about where SAL writes generated files, see Data Layout.