Continuous Semantic Validation: SHACL Quality Gates in CI/CD

July 19, 2026 · 12 min read

In Part 1, every Competency Question became a SPARQL assertion: a test that either passes or fails against a populated graph. But for those tests to mean anything, the data in the graph must be well-formed. A SPARQL query that asks “which tier-1 suppliers serve commodity X?” cannot return meaningful results if some supply relationships are missing their tier classification, others have their supplier linked to the wrong class, and the graph contains a mix of partially-loaded records from a failed ETL run.

This is the problem SHACL solves. Before you can validate that the ontology answers its Competency Questions correctly, you need to validate that the data satisfies the structural and operational constraints the model requires. SHACL is the layer that enforces those constraints - and this post is about building it properly, wiring it to your CI pipeline, and making it an active part of the development loop rather than a one-off audit tool.

OWL tells you what is logically possible. SHACL tells you what is operationally required. You need both, because they answer fundamentally different questions about your data.

Why OWL Alone Cannot Validate Your Data

Before writing a single SHACL shape, it is worth understanding precisely why OWL is insufficient for operational validation. This is not a limitation of OWL’s expressiveness: it is a consequence of a deliberate design choice that makes OWL useful for knowledge representation but unsuitable for data enforcement.

The Open World Assumption Explained

OWL operates under the Open World Assumption (OWA). Under OWA, the absence of a triple does not mean that something is false: it means it is unknown. If the graph contains no proc:tier triple for a particular supply relationship, OWL does not conclude “this supply relationship has no tier.” It concludes “we do not know what tier this supply relationship has.”

This is semantically correct for a knowledge representation system. Real-world knowledge is always incomplete. An ontology that represents what is known about a domain should not treat absence of information as evidence of absence. A reasoning system that uses OWA can make sound inferences even over incomplete data.

But operational systems need the opposite assumption. A data pipeline that writes supply relationships to a graph store needs to enforce that every relationship includes a tier classification before it is considered valid for downstream consumers. A compliance system checking tier-1 constraints cannot work with data where some relationships simply have no tier at all. These systems need Closed World semantics: what is not asserted is considered absent, and absence of required data is a violation.

Use OWL axioms to express the domain semantics: the structure of what is true and how concepts relate. Use SHACL shapes to enforce operational constraints: the requirements that data must satisfy to be usable. Neither replaces the other. You need both layers, and they belong in separate files.

What owl:minCardinality Actually Does

It is worth being explicit about this because the mistake is common. The OWL axiom:

proc:SupplyRelationship
    rdfs:subClassOf [
        a owl:Restriction ;
        owl:onProperty proc:tier ;
        owl:minCardinality "1"^^xsd:nonNegativeInteger
    ] .

…does not mean “every SupplyRelationship instance must have a tier value.” Under OWA, what it means to a reasoner is: “if something is a SupplyRelationship and has no explicit tier, the reasoner should infer the existence of some tier value that has not yet been asserted.” This is the opposite of enforcement: it is a reasoning license to assume the data is correct even when it is incomplete.

Compare with SHACL:

proc:SupplyRelationshipShape
    a sh:NodeShape ;
    sh:targetClass proc:SupplyRelationship ;
    sh:property [
        sh:path proc:tier ;
        sh:minCount 1 ;
        sh:message "Every SupplyRelationship must have a tier classification." ;
    ] .

This genuinely fails if proc:tier is absent. The violation is recorded, reported, and - if you wire it to a CI gate - blocks merge.

The SHACL Test Pyramid

Not all shapes are equal in cost or purpose. Borrowing the test pyramid concept from software engineering, a well-designed SHACL validation strategy has three layers, each with a different scope, cost, and trigger frequency.

The cheapest tests run most often. Unit shapes run on every pull request in milliseconds. Business-rule shapes run on merge. Competency Question acceptance tests run against the real graph on release candidates only.

Unit Shapes: The Foundation Layer

Unit shapes are narrow, fast constraints that validate individual properties on individual classes. They should be the most numerous layer in your validation stack: hundreds of shapes, each covering one property on one class, running in milliseconds.

Unit shapes answer questions like: does every proc:Supplier have exactly one proc:supplierCode that matches the pattern SUP-NNNNNN? Does every proc:SupplyRelationship have a proc:tier that is one of the allowed values? Does every proc:commodity value point to a resource that is actually an instance of proc:Commodity?

Because unit shapes are small and fast, you can afford to run all of them on every pull request. A failing unit shape on a PR tells the developer immediately, in their own branch, before anything reaches main.

Cross-entity SPARQL-SHACL: The Business Rules Layer

SPARQL-SHACL shapes use full SPARQL queries embedded in a SHACL constraint. This gives you the power to validate things that span multiple resources: referential integrity between entities, business rules that depend on combinations of properties, constraints that require joins.

These shapes are more expensive to evaluate because each one runs a SPARQL query against the full graph. They are also harder to write, because SPARQL-in-SHACL has a specific syntax and semantic that takes practice. For this reason, this layer should run on merges to main rather than on every PR: it catches architectural-level violations, not typos.

CQ Acceptance Tests: The Specification Layer

At the apex is the test that was defined before the model: the SPARQL assertion for each Competency Question from Part 1. These are the ground truth of the sprint: the thing the ontology was built to pass. They run against the full populated graph with real (or realistic) data, which makes them the slowest and most meaningful layer.

CQ acceptance tests run on release candidates, not on every PR. They validate that the ontology, as a whole, answers the business questions it was specified to answer. A failing CQ acceptance test creates a sprint story, not a hotfix.

Writing Shapes: From Simple to Advanced

Unit Shape Patterns

A basic unit shape for supplier structural validation:

proc:SupplierShape
    a sh:NodeShape ;
    sh:targetClass proc:Supplier ;
    sh:property [
        sh:path proc:supplierCode ;
        sh:datatype xsd:string ;
        sh:minCount 1 ;
        sh:maxCount 1 ;
        sh:pattern "^SUP-[0-9]{6}$" ;
        sh:message "Every Supplier requires exactly one code matching SUP-NNNNNN." ;
        sh:severity sh:Violation ;
    ] ;
    sh:property [
        sh:path proc:country ;
        sh:class schema:Country ;
        sh:minCount 1 ;
        sh:message "Every Supplier must be associated with a country." ;
        sh:severity sh:Violation ;
    ] ;
    sh:property [
        sh:path proc:preferredName ;
        sh:datatype xsd:string ;
        sh:minCount 1 ;
        sh:maxCount 1 ;
        sh:message "Every Supplier must have exactly one preferred name." ;
        sh:severity sh:Warning ;
    ] .

Notice that sh:severity sh:Violation blocks the CI gate, while sh:severity sh:Warning logs and creates a ticket but does not block. This distinction is discussed further below.

Value Range and Enumeration Constraints

Many business rules reduce to “this value must be one of a fixed set” or “this number must be within a range”:

proc:SupplyRelationshipShape
    a sh:NodeShape ;
    sh:targetClass proc:SupplyRelationship ;
    sh:property [
        sh:path proc:tier ;
        sh:in ( proc:Tier1 proc:Tier2 proc:Tier3 ) ;
        sh:minCount 1 ;
        sh:maxCount 1 ;
        sh:message "Tier must be exactly one of Tier1, Tier2, or Tier3." ;
        sh:severity sh:Violation ;
    ] ;
    sh:property [
        sh:path proc:monthlyVolume ;
        sh:datatype xsd:decimal ;
        sh:minInclusive "0"^^xsd:decimal ;
        sh:message "Monthly volume must be a non-negative decimal." ;
        sh:severity sh:Violation ;
    ] ;
    sh:property [
        sh:path proc:leadTimeDays ;
        sh:datatype xsd:integer ;
        sh:minInclusive "1"^^xsd:integer ;
        sh:maxInclusive "365"^^xsd:integer ;
        sh:message "Lead time must be between 1 and 365 days." ;
        sh:severity sh:Warning ;
    ] .

Cross-entity SPARQL-SHACL: Business Rules

When a constraint requires joining across multiple resources, sh:sparql gives you the full SPARQL query language inside a SHACL shape:

proc:Tier1MOQShape
    a sh:NodeShape ;
    sh:targetClass proc:SupplyRelationship ;
    sh:sparql [
        sh:message "Tier-1 supply relationships must specify a monthly volume of at least 1000 units." ;
        sh:severity sh:Violation ;
        sh:select """
            SELECT $this WHERE {
                $this proc:tier proc:Tier1 .
                $this proc:monthlyVolume ?vol .
                FILTER (?vol < 1000)
            }""" ;
    ] .

A more complex example: referential integrity across modules:

proc:CommodityReferenceShape
    a sh:NodeShape ;
    sh:targetClass proc:SupplyRelationship ;
    sh:sparql [
        sh:message "The commodity referenced in a supply relationship must exist as a typed Commodity instance in the graph." ;
        sh:severity sh:Violation ;
        sh:select """
            SELECT $this WHERE {
                $this proc:commodity ?c .
                FILTER NOT EXISTS { ?c a proc:Commodity }
            }""" ;
    ] .

SPARQL-SHACL shapes give you negation, joins, subqueries, and aggregates: anything SPARQL can express. The power is real, but the performance cost scales with graph size, which is why these shapes belong in the on-merge layer rather than the per-PR layer.

Closed Shapes

By default, SHACL shapes are open: they validate the properties you declare but do not complain about extra properties. For sensitive data classes where unrecognized properties could indicate a data integration error, you can close the shape with sh:closed true:

proc:SensitiveSupplierShape
    a sh:NodeShape ;
    sh:targetClass proc:CriticalSupplier ;
    sh:closed true ;
    sh:ignoredProperties ( rdf:type ) ;
    sh:property [ sh:path proc:supplierCode ; sh:minCount 1 ] ;
    sh:property [ sh:path proc:country ; sh:minCount 1 ] ;
    sh:property [ sh:path proc:preferredName ; sh:minCount 1 ] .

Any proc:CriticalSupplier instance with properties not listed above will trigger a violation. Use closed shapes sparingly: they make the validation strict but also make it harder to add new properties incrementally. A good practice is to use closed shapes only for classes where data quality is critical and the schema is genuinely stable.

Severity Levels as Operational Signals

SHACL defines three severity levels, and using them thoughtfully transforms your violation reports from noise into actionable signals.

A practical heuristic: sh:Violation for anything a downstream system will fail on (missing required properties, malformed identifiers, broken referential integrity). sh:Warning for anything that degrades data quality but does not break queries (missing optional labels, deprecated terms in use, unusual but valid values). sh:Info for anything you want visibility into over time (coverage of optional enrichment fields, distribution of values).

The CI Gate Pipeline

The semantic quality gate is a sequential pipeline of five checks, each with a hard failure mode that blocks the next stage. The goal is to catch different categories of problems at the cheapest possible moment.

Gate ①: Syntax Validation

ROBOT validates that the ontology file is valid Turtle (or OWL/XML, depending on your serialization) and that it can be parsed into an OWL ontology object. This takes about thirty seconds and catches: malformed Turtle syntax, missing prefix declarations, invalid IRI syntax, circular owl:imports. No data is needed: this is purely structural.

Gate ②: OWL Consistency

An OWL reasoner checks whether the ontology is logically consistent: that there are no class expressions that entail a contradiction, no owl:disjointWith constraints violated by the class hierarchy, no property chains that create paradoxes.

Use ELK for OWL EL profiles (fast, handles large ontologies with property chains). Use HermiT for OWL DL profiles (complete but slower). The choice depends on which reasoning profile you selected in your ORSD §5 NFRs.

A consistency failure at this gate means you introduced a logical error in the ontology itself: something that makes the formal semantics self-contradictory, regardless of any data. Fix this before proceeding; an inconsistent ontology will silently mis-classify instances in downstream reasoning.

Gate ③: Unit SHACL Shapes

pyshacl runs all unit shapes against the test instance data:

- name: Unit SHACL shapes
  run: |
    python -m pyshacl \
      -s shapes/unit/ \
      -d data/test-instances.ttl \
      --output-format human \
      --inference rdfs

The --inference rdfs flag enables RDFS inference during validation, so rdfs:subClassOf hierarchies are respected by the shapes without requiring a full OWL reasoner. This is appropriate for unit shapes, which target specific classes.

A failing unit shape produces output like:

Constraint Violation in MinCountConstraintComponent (http://www.w3.org/ns/shacl#MinCountConstraintComponent):
	Severity: sh:Violation
	Source Shape: proc:SupplierShape
	Focus Node: <https://example.org/suppliers/SUP-000142>
	Result Path: proc:supplierCode
	Message: Every Supplier requires exactly one code matching SUP-NNNNNN.

This tells you: which shape failed, which node, which property, and the human-readable message you wrote. Good shape messages are the difference between “a validation failed somewhere” and “supplier SUP-000142 is missing its supplier code.”

Gate ④: Cross-entity SPARQL-SHACL

Business rule shapes run on merge to main, against a richer test dataset:

- name: Cross-entity SHACL
  if: github.event_name == 'push' && github.ref == 'refs/heads/main'
  run: |
    python -m pyshacl \
      -s shapes/unit/ \
      -s shapes/business-rules/ \
      -d data/test-instances.ttl \
      --output-format json \
      --inference rdfs

Running both unit and business-rule shapes together on merge ensures the full picture is visible before any change lands on main.

Gate ⑤: CQ Acceptance Tests

The acceptance test runner queries the SPARQL endpoint with the natural-language-to-SPARQL translations from the CQ registry:

- name: CQ acceptance
  if: startsWith(github.ref, 'refs/tags/v')
  run: |
    python scripts/run_cq_tests.py \
      --sparql-endpoint ${{ vars.GRAPH_ENDPOINT }} \
      --cq-registry cqs/registry.yaml

The CQ runner iterates through each CQ, executes the SPARQL assertion, and checks the result against the expected outcome. It exits non-zero if any CQ fails, with a report naming exactly which CQs failed and what the actual result was.

Complete GitHub Actions Workflow

name: Semantic Quality Gate
on:
  push:
    branches: [main]
    tags: ['v*']
  pull_request:
    branches: [main]

jobs:
  semantic-quality-gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install tooling
        run: pip install pyshacl rdflib pyyaml

      - name: ① Syntax validation
        run: robot validate --input ontology.ttl

      - name: ② OWL consistency
        run: robot reason --reasoner ELK --input ontology.ttl

      - name: ③ Unit SHACL shapes
        run: |
          python -m pyshacl \
            -s shapes/unit/ \
            -d data/test-instances.ttl \
            --output-format human \
            --inference rdfs

      - name: ④ Cross-entity SHACL
        if: github.event_name == 'push' && github.ref == 'refs/heads/main'
        run: |
          python -m pyshacl \
            -s shapes/unit/ \
            -s shapes/business-rules/ \
            -d data/test-instances.ttl \
            --output-format human \
            --inference rdfs

      - name: ⑤ CQ acceptance
        if: startsWith(github.ref, 'refs/tags/v')
        run: |
          python scripts/run_cq_tests.py \
            --sparql-endpoint ${{ vars.GRAPH_ENDPOINT }} \
            --cq-registry cqs/registry.yaml

What to Track Over Time

Part 3 covers the next question: once the ontology is validated and the quality gate is passing, how do you version and release it without breaking the pipelines that depend on it?