Using SAL Modules

A SAL Module is a git repository with a Dockerfile in its root whose container implements the SAL Module command line interface. Referencing one from your project’s RDF lets sal build run it and fold the RDF it produces into your data product.

Reference a module

A module is referenced through the salmodule:// protocol scheme. Declare it as a prefix and type an instance with one of the classes the module’s ontology defines:

@prefix salmodule: <https://w3id.org/sal/cgs-earth/sal/ontology/salmodule#> .
@prefix states: <salmodule://github.com/cgs-earth/python-geoconnex/> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

<GeoconnexStates> a states:GeoconnexReferenceFeatureStates ;
    a salmodule:NodeProcessor ;
    states:maxRetries "5"^^xsd:integer .

The prefix takes the form salmodule://[HOST/]OWNER/REPO/. HOST is optional and defaults to github.com.

The prefix must end in a slash

A module’s ontology declares its classes as relative IRIs, such as "@id": "GeoconnexReferenceFeatureStates". SAL resolves them against the prefix you declared, and only a slash-terminated prefix places them underneath the module:

# correct
@prefix states: <salmodule://github.com/cgs-earth/python-geoconnex/> .
# states:GeoconnexReferenceFeatureStates and the ontology's own term both become
#   salmodule://github.com/cgs-earth/python-geoconnex/GeoconnexReferenceFeatureStates

# wrong
@prefix states: <salmodule://github.com/cgs-earth/python-geoconnex#> .
# relative IRI resolution drops the last path segment, so the ontology's term becomes
#   salmodule://github.com/cgs-earth/GeoconnexReferenceFeatureStates
# which no longer matches the term the prefix expands to

A prefix that does not end in a slash reports every module term as undefined even though the module and its ontology are fine, so reach for the slash-terminated form whenever a module’s terms unexpectedly fail to validate.

Configure the task

A task is configured in RDF, with the properties the module’s own vocabulary defines. states:maxRetries above is one of them, so the module decides what a valid task instance looks like and sal validate reports a typo in a configuration property the same way it reports a typo in a class.

SAL serializes those properties into the JSON-LD node object that salmodule:runCommand is invoked with. The instance above is passed as:

{"@id":"https://example.org/GeoconnexStates","@type":"GeoconnexReferenceFeatureStates","maxRetries":{"@value":"5","@type":"xsd:integer"}}

The rules SAL follows when writing it:

• Only properties declared as vocabulary in the module’s own ontology become keys. Everything else the project says about the instance, such as an rdfs:label or a schema:name, is left out, since only the module knows what configures it.
• Keys and IRIs are written the way the module’s ontology writes them: a term of the module’s vocabulary becomes its relative name, and any other IRI is shortened with whichever prefix the ontology’s @context binds it to.
• A typed literal becomes a {"@value": ..., "@type": ...} object and a language-tagged literal becomes a {"@value": ..., "@language": ...} object. A plain string is written as a string.
• A property used more than once becomes an array, ordered by the encoding of its values, since RDF gives repeated properties no order.
• An RDF list, written ( ... ) in Turtle, becomes a JSON array in the order written, with blank nodes in it inlined as objects and lists in those inlined in turn. Use one wherever order matters, such as a list of sources to run in sequence or the arguments of a command; a module declares such a term with "@container": "@list" in its @context. An empty list becomes [].
• A blank node is inlined as a nested object, so structured configuration can be expressed with the usual turtle [ ... ] syntax. A named node stays a {"@id": ...} reference.

Which ontology property describes what

Two properties of the SAL Module ontology are involved, and it is worth keeping them apart:

• salmodule:runCommand describes the invocation the task instance is an input to, and so is the property that documents the JSON-LD node object above. Its skos:note is what says the value must be a JSON-LD representation of the salmodule:Task subclass instance being invoked.
• salmodule:taskInstanceEnvVar only defines a constant: the name of the environment variable that value is carried in. A module declares it on its own ontology, and it defaults to SALMODULE_TASK_INSTANCE.

Caution

Neither property is written in a project’s RDF. Earlier versions of SAL took the task instance from a JSON-LD literal attached to the instance with salmodule:taskInstanceEnvVar; SAL now warns that such a value is ignored and builds the task instance from the instance’s own properties instead.

What SAL does with it

Validation dereferences the salmodule:// prefix so that the module’s terms are checked like any other vocabulary. SAL clones the repository, builds the Dockerfile in its root, and runs docker run IMAGE salmodule ontology to read the module’s vocabulary. A typo in a module term is reported the same way a typo in a schema: term is:

data/module.ttl:5: undefined term states:GeoconnexReferenceFeatureStatess

sal build then runs each task instance with docker run -e SALMODULE_TASK_INSTANCE=... IMAGE salmodule run and commits what it produces in the same snapshot as the sources. sal build --without-sal-modules commits the task’s configuration into the data product without running anything, and sal run is a debugging command that runs the tasks again on top of the last build. An instance is only run when it is typed as a SAL Module task, either because the project’s RDF types it as a salmodule:NodeProcessor, salmodule:NodeProducer, salmodule:NodeConsumer, or salmodule:Task, or because the module’s own ontology declares its class as a subclass of one of those.

A task writes newline delimited JSON to stdout. SAL injects the module ontology’s @context into each node so that its keys resolve to the module’s vocabulary, converts the result to RDF, and commits it alongside the sources. Those triples land in the Iceberg table alongside the triples from your source files.

A relative IRI in a task’s output, such as "@id": "person/bob", resolves against your project’s namespace, not against the module’s. What a task produces is instance data belonging to the project that configured and ran it; the module’s salmodule:// namespace names its vocabulary, not the data it emits. A module that needs its output under a particular namespace, for instance the authoritative source the data was pulled from, writes absolute IRIs, which are kept as written.

If a task emits salmodule:Error nodes instead of data, the run fails with the messages the module reported.

Validating a task’s output

A module can declare what its task’s output must look like by annotating the task class in its ontology with salmodule:stdoutShape, whose value is a SHACL sh:NodeShape:

{
  "@id": "StationFetcher",
  "@type": "owl:Class",
  "rdfs:subClassOf": {"@id": "salmodule:Task"},
  "salmodule:stdoutShape": {
    "@type": "sh:NodeShape",
    "sh:targetClass": {"@id": "schema:Place"},
    "sh:property": [
      {"sh:path": {"@id": "schema:name"}, "sh:minCount": 1, "sh:message": "A station must have a name"}
    ]
  }
}

When the class, or a superclass of it the module’s ontology declares, carries such a shape, sal build validates every line the task writes as it arrives, with the module’s @context injected the same way it is before the output is converted to RDF, so the shape sees each node as it will be committed. The first line that fails stops the container and fails the build with the shape’s sh:message and the node it is about:

run: SAL module salmodule://github.com/owner/stations wrote a node on output line 3 that does not conform to the salmodule:stdoutShape of salmodule://github.com/owner/stations/StationFetcher: <https://example.test/project/station/7> A station must have a name

Nothing the task wrote is committed. A result whose severity the shape lowers to sh:Warning or sh:Info is logged rather than fatal. A class without a salmodule:stdoutShape is run unchecked, and the check costs nothing then.

Each line is validated on its own, so a shape can only describe the nodes of one line; the line’s @id is resolved against your project’s namespace, the same way it is when the output is committed. Standard SHACL targeting applies, so a shape needs a target, typically sh:targetClass, to apply to anything. Any SHACL Core constraint can be used except sh:languageIn, sh:uniqueLang, sh:disjoint, sh:qualifiedValueShapesDisjoint, and sh:sparql; a shape using one of those fails the run before the task is started.

Files a task produces

A task can hand over files as well as triples. Any IRI in its output written with the file:/// scheme names a file inside the container, and sal build copies that file out of the container while the task is still running, the way docker cp would:

{"@id": "https://example.org/dataset/2024", "schema:hasPart": {"@id": "file:///tmp/test.txt"}}

The copy lands in .sal/data/blobs named by the SHA-256 digest of its contents, so it is content addressable and stored once however many tasks produce it. Every triple that named the file, as its subject or its object, is rewritten to refer to the copy, and the copy is described the same way a pinned vocabulary is: the file:/// IRI the task wrote as rdfs:label, when it was written inside the container as dcterms:modified, its digest as owl:versionIRI, and where it sits under .sal/data/blobs as dcterms:identifier. The example above commits:

<https://example.org/dataset/2024> schema:hasPart <urn:sha256:9f86d0…> .
<urn:sha256:9f86d0…> rdfs:label "file:///tmp/test.txt" ;
    dcterms:modified "2026-09-10T14:26:05Z"^^xsd:dateTime ;
    dcterms:identifier "9f86d0…" ;
    owl:versionIRI <urn:sha256:9f86d0…> .

sal serve serves the copy at /blobs/{sha256}, the same as a pinned vocabulary document.

Directories

Some outputs are only useful as a whole directory: a Zarr store, a STAC catalog whose href links name the files beside it. A file:/// IRI ending in a slash names a directory, and sal build copies the whole directory out of the container instead of one file:

{"@id": "https://example.org/dataset/2024", "schema:hasPart": {"@id": "file:///out/catalog/"}}
{"@id": "file:///out/catalog/", "schema:name": "The 2024 STAC catalog"}

A directory is stored under the digest of its contents, .sal/data/blobs/3f9a…/ here, with the files inside it left exactly as the task wrote them, so that a Zarr or STAC client pointed at that directory, on disk, at /blobs/3f9a…/ under sal serve, or in the bucket sal upload copies .sal/data to, reads it with no rewriting or lookup, and so that two modules writing the same container path, /out/catalog/ say, never overwrite each other. The digest is the SHA-256 of the directory’s contents (one line per file, sorted by path, of the relative path, a NUL, and the file’s own SHA-256), so a directory copied again unchanged hashes the same. Every copy, file or directory, is recorded in .sal/data/blobs/prov.jsonld, one node per copy in the shape .sal/config.jsonld records a pinned vocabulary in, with the module that produced it as dcterms:source:

{
  "@context": {
    "dcterms": "http://purl.org/dc/terms/",
    "owl": "http://www.w3.org/2002/07/owl#",
    "rdfs": "http://www.w3.org/2000/01/rdf-schema#",
    "xsd": "http://www.w3.org/2001/XMLSchema#"
  },
  "@graph": [
    {
      "@id": "urn:sha256:3f9a…",
      "rdfs:label": "file:///out/catalog/",
      "owl:versionIRI": { "@id": "urn:sha256:3f9a…" },
      "dcterms:identifier": "3f9a…/",
      "dcterms:source": { "@id": "salmodule://github.com/example/stac-module" },
      "dcterms:modified": { "@value": "2026-09-10T14:26:05Z", "@type": "xsd:dateTime" },
      "rdfs:comment": "Represents the directory file:///out/catalog/ a task of the SAL module salmodule://github.com/example/stac-module copied out of its container as of 2026-09-10T14:26:05Z. urn:sha256:3f9a… is the SHA-256 of its contents, and the directory is served whole as a zip archive under that digest."
    }
  ]
}

The node’s @id is the digest IRI the graph refers to the copy by, its rdfs:label is the file:/// IRI the task named it with, and its dcterms:identifier is where it sits under the blob store. The graph refers to the directory as urn:sha256:3f9a… the same way it does a file, described with that IRI, modification time (the newest of any file in it), digest, path, and source module, together with whatever else the task said about it:

<https://example.org/dataset/2024> schema:hasPart <urn:sha256:3f9a…> .
<urn:sha256:3f9a…> rdfs:label "file:///out/catalog/" ;
    dcterms:modified "2026-09-10T14:26:05Z"^^xsd:dateTime ;
    dcterms:identifier "3f9a…/" ;
    dcterms:source <salmodule://github.com/example/stac-module> ;
    owl:versionIRI <urn:sha256:3f9a…> ;
    schema:name "The 2024 STAC catalog" .

sal serve serves each file in the directory at /blobs/3f9a…/{file} and the directory whole as a zip archive, named after the file:/// IRI the task gave it, at /blobs/3f9a… or /blobs/urn:sha256:3f9a…. A directory whose catalog.json is a STAC Catalog or Collection is also linked as a child of the data product’s own STAC catalog, so a STAC client that opens .sal/data/stac/catalog.json finds the module’s catalog beneath the triples table.

prov.jsonld and the blob store are a log. A run appends the copies it made and never removes what an earlier run left, so a directory produced again with different contents lands under a new digest beside the old one, every urn:sha256: a past build committed to the table still resolves, and every STAC catalog ever handed over stays linked from the data product’s catalog. Clearing old copies is up to the user: sal clean --wipe removes every recorded copy and prov.jsonld with the data product, or delete a copy’s directory and its node by hand.

Files are copied concurrently with the task writing its output, so a module must finish writing a file, or every file in a directory, before it prints the line that names it; one named before it is complete is copied incomplete. Each path is copied once: a second reference to the same path is warned about and skipped, and both references point at the same copy. The container is kept until every copy has finished, and the build fails if a named path does not exist, if a path without a trailing slash is a directory, if a path with one is a file, or if a directory holds anything other than regular files and directories, such as a symbolic link. A file:/// path written as a plain string rather than an IRI is not a file reference; sal build warns about one and discards it. After each task the log says how many files it copied.

Importing a module’s ontology

Referencing a module checks its terms and lets sal build run its tasks, but the vocabulary itself stays outside the data product. Import the module to carry it too:

sal import salmodule://github.com/cgs-earth/python-geoconnex

sal build dereferences the import the same way it dereferences a salmodule:// prefix, by cloning the repository, building the image, and running salmodule ontology, and merges the resulting triples into the table. The module is written without a trailing slash here, since the import names the module rather than the vocabulary base its terms resolve against.

See sal import.

Modules recorded in the table

Every module a build downloaded, whether it was dereferenced for its vocabulary or run as a task, is written to the Iceberg table metadata as a JSON list under the sal.salmodules property:

["salmodule://github.com/cgs-earth/python-geoconnex"]

The list is rewritten on every build and run, so dropping a module from your RDF drops it from the table too. sal query --info properties prints it, and the Modules tab of sal serve --with-ui offers each entry as a chip and as an autocompletion so a module the table was built from can be inspected without typing its URI.

Requirements

• A reachable docker daemon. SAL talks to whatever DOCKER_HOST points at, defaulting to the local socket.
• git on PATH, used to clone the module repository.

Each module is cloned and built at most once per sal invocation, and its image is tagged with the git commit hash of the repository it was built from. A later invocation looks that tag up before doing anything: when the commit the project pins in .sal/config.jsonld still has its image on the docker daemon, the module is neither cloned nor built again, and its container runs from the existing image. Removing the image with docker rmi brings back the clone and build.

A dereferenced module ontology is pinned like any other vocabulary the project resolves against: its JSON-LD is stored under .sal/data and recorded in .sal/config.jsonld, so a later build validates against the same version of the module’s vocabulary without rebuilding the module to ask for it again. --no-cache clones every referenced module again, ignoring the pinned image, and re-pins the ontology each one reports. See Pinned vocabularies.

Materialization is not pinned: sal build runs each task every time it is invoked, since a task’s output is data rather than vocabulary.