Skip to content

feat(gen-shacl): translate presence-implies-value rules to SHACL-SPARQL - #19

Open
jdsika wants to merge 22 commits into
mainfrom
feat/shaclgen-presence-implies-value-stacked
Open

jdsika wants to merge 22 commits into
mainfrom
feat/shaclgen-presence-implies-value-stacked

Conversation

@jdsika

@jdsika jdsika commented Jul 10, 2026 •

Copy link
Copy Markdown

Summary

Adds the presence-implies-value pattern to the rules → SHACL-SPARQL
converter: a precondition asserting value_presence: PRESENT on a guard slot
plus a postcondition constraining another slot with equals_string /
equals_string_in becomes a sh:SPARQLConstraint that flags focus nodes where
the guard is present but the target does not hold an allowed value.

It reads as "if the guard slot is present, the target slot must be present and
hold one of the allowed values"
and generalises the existing boolean guard to
arbitrary enum values. Enum permissible values resolve to their meaning IRIs;
values without meaning compare as string literals. $this is pre-bound per
SHACL §5.3.1.

Example from the modeled use case: if sun_altitude is present, daytime must
be day or twilight
— cross-parameter consistency that per-slot SHACL
property shapes cannot express.

Bug fixed along the way

The pre-existing boolean guard was not gated on the target slot's range. A slot
of range string carrying equals_string: "true" was translated as a boolean
comparison, which does not match the string "true" in the data — so conforming
instances were reported as violations. The guard now requires range boolean,
and string-ranged slots fall through to the presence-implies-value pattern and
compare as strings.

Translation contract: skip, never mis-translate

Pattern matching is exact. Each converter requires its conditions to set
precisely the operators it translates. A rule carrying anything further — extra
scalar operators, or expression-level any_of / all_of / none_of /
exactly_one_of — is skipped rather than partially translated, because dropping
a term would either widen the precondition (false positives) or weaken the
postcondition (false negatives).

Slot resolution goes through induced slots, so slot_usage overrides,
slot_uri overrides and alias-form keys all resolve to the same IRI that
sh:path emits.

Stack position

main
└─ #19  presence-implies-value      ← this PR
   └─ #20  compositional fallback (M1–M5)
      └─ #23  documentation

Note

No dependencies. This PR applies directly to main. The rules → SPARQL
framework it extends (_add_rules, _rule_to_sparql, the boolean-guard and
exclusive-value patterns) is already in main via linkml#3451.

Testing

pytest tests/linkml/test_generators/test_shaclgen.py — 105 passed, with
the full generator suite green (1774 passed, 52 skipped, 3 xfailed).
Coverage includes SPARQL syntax validation and pyshacl end-to-end conforming
/ violating round-trips for each pattern, plus negative tests asserting that
unsupported operator combinations are skipped rather than mis-translated.

Review notes

This PR was re-cut from an earlier five-PR stack. The corrections that were
previously separate follow-ups are folded into the feature they correct, so
there is no longer a PR that introduces a defect and another that fixes it.
Two defects found during that re-cut are fixed here: the boolean-guard range
gate above, and a duplicate test_rule_with_elseconditions_warns definition
that silently shadowed the existing test of the same name.

@jdsika

jdsika commented Jul 10, 2026

Copy link
Copy Markdown
Author

Adversarial audit findings (PIV converter, audited at stack tip incl. the hardening PR)

Two substantive findings, both empirically demonstrated with pyshacl end-to-end probes and cross-checked against LinkML's reference rule semantics (gen-json-schema if/then realization):

A1 — real bug: greedy dispatch drops extra pre/postcondition operators → false positives.
_rule_to_sparql dispatches to presence-implies-value whenever the precondition has value_presence: PRESENT and the postcondition has equals_string/equals_string_in — without requiring these to be the only operators set. A precondition {value_presence: PRESENT, minimum_value: 100} loses the threshold: data with temp 50 (precondition unsatisfied, rule vacuously satisfied — LinkML's own JSON-Schema realization accepts it) is flagged as violating. Dual: extra postcondition operators are dropped too (equals_string alongside equals_string_in — the _in list silently wins). This widens/narrows the rule instead of skipping — a mis-translation, not a safe skip.
Fix direction: dispatch only when the pre/post conditions set exactly the pattern's operators; otherwise fall through (the fallback PR's _scalar_filters already does this accounting per-operator — the named patterns need the same exhaustiveness check).

A2 — edge case: boolean-guard shadows PIV for equals_string: "true" on non-boolean slots.
The boolean-guard branch keeps dispatch priority but never checks that the target slot's range is boolean. A rule "if opt present, status (range string) must equal "true"" is hijacked into a boolean comparison; both status "true" (conforming) and status "false" are flagged. Pre-existing in the framework, but this PR codifies the priority order, and PIV is the handler that would translate this rule correctly.
Fix direction: gate the boolean-guard branch on the induced post-slot range being boolean.

Verified clean (attacked, held up): absent-target semantics (violation via !BOUND, matches the JSON-Schema required realization); multivalued-target ∀ semantics (matches items:{const}); multivalued guards; mixed meaning/no-meaning equals_string_in sets (matches rdflib_dumper's IRI-vs-literal convention exactly).

Test gaps (cosmetic): test_presence_implies_value_no_meaning_falls_back_to_literal's "<Manual>" not in query assertion is vacuous (an erroneous emission would be a full IRI, never matching that string); no tests for combined operators (would have caught A1) or equals_string: "true" on a non-boolean slot (would have caught A2).

These findings equally apply to the consolidated branch of #18. Suggest addressing A1/A2 as a follow-up commit on this stack before upstreaming.

@jdsika

jdsika commented Jul 11, 2026

Copy link
Copy Markdown
Author

The substantive audit findings above are resolved in #22 (fix/shaclgen-rule-converter-audit-findings, stacked on this series) — one commit, 19 regression tests, 18 of which fail on the pre-fix source. See the finding→fix table in the #22 description.

@jdsika jdsika self-assigned this Jul 11, 2026
@jdsika
jdsika force-pushed the feat/shaclgen-presence-implies-value-stacked branch from 2e56a36 to 8999ad6 Compare July 11, 2026 10:38
@jdsika
jdsika force-pushed the feat/shaclgen-presence-implies-value-stacked branch from 8999ad6 to e161ce7 Compare September 11, 2026 12:53
@jdsika
jdsika changed the base branch from feat/shaclgen-rules-sparql to main September 11, 2026 12:53
@jdsika jdsika changed the title feat(gen-shacl): add presence-implies-value rule pattern (stacked on #11) feat(gen-shacl): translate presence-implies-value rules to SHACL-SPARQL Sep 11, 2026
@jdsika jdsika closed this Sep 11, 2026
@jdsika jdsika reopened this Sep 11, 2026
@jdsika

jdsika commented Sep 11, 2026

Copy link
Copy Markdown
Author

Mirrored upstream as linkml#3989.

github-actions Bot and others added 9 commits September 25, 2026 16:46
* test(openapigen): parametrize OAS version in templates and fixture

Make the top-level \`openapi\` attribute of the composed OpenAPI templates a
parameter instead of hardcoding '3.0.3'. The OPENAPI_HEADER fragment now takes
an oas_version placeholder, threaded through the openapi_template and
single_endpoint_template composition helpers (defaulting to 3.0.3).

The openapi_spec fixture is parametrized over the OAS_VERSIONS mapping (OpenAPI
version string -> OpenApiGenerator output format name) and wires the chosen
format into the generator via gen_openapi_spec. The TEMPLATE_HEAD constant is
refactored into the template_head() helper so the fixture and the standalone
tests compose the same template. Only 3.0.3 is driven so far; adding a new OAS
version is a one-line extension of OAS_VERSIONS.

Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>

* feat(openapigen): prepare support for multiple openapi version

Generalize the generator so the OpenAPI version is not hard-coded to
v3.0.3. The version is now read from the template's top-level `openapi`
attribute and validated against the list of supported versions, paving
the way for v3.1.0 support.

Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>

* feat(openapigen): add support for openapi v3.1.0

When the template declares OpenAPI v3.1.0, schemas are now generated via
PydanticGenerator instead of JsonSchemaGenerator. Because OpenAPI 3.1.0
is fully aligned with JSON Schema 2020-12, the only post-processing
needed
is rewriting `$defs` references to `components/schemas` and stripping
`linkml_meta` annotations. Schema names invalid under OpenAPI 3.1 are
sanitized and their `$ref`s rewritten.

Adds v3.1.0-specific test fixtures and docs for the two supported
versions.

Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>

* test(openapigen): parametrize version-agnostic tests over both OAS versions

---------

Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>
Co-authored-by: N <13322818+noelmcloughlin@users.noreply.github.com>
@jdsika
jdsika force-pushed the feat/shaclgen-presence-implies-value-stacked branch from e161ce7 to 7622d70 Compare October 2, 2026 11:15
The rules-to-SHACL-SPARQL converter added in linkml#3451 recognised a single
named pattern.  This adds the presence-implies-value pattern: a
precondition asserting `value_presence: PRESENT` on one slot, and a
postcondition constraining another slot with `equals_string` or
`equals_string_in`.  It reads as "if the guard slot is present, the
target slot must be present and hold one of the allowed values", and
generalises the existing boolean guard to arbitrary enum values.

The boolean guard is now gated on the target slot's range actually
being `boolean`.  Without that gate a slot of range `string` carrying
`equals_string: "true"` was translated as a boolean comparison, which
does not match the string `"true"` in the data and so flagged
conforming instances as violations.  String-ranged slots now fall
through to the presence-implies-value pattern and compare as strings.

Pattern matching is exact: each converter requires its conditions to
set precisely the operators it translates.  A rule whose conditions
carry anything further -- extra scalar operators, or expression-level
any_of / all_of / none_of / exactly_one_of -- is skipped rather than
partially translated, since dropping a term would either widen the
precondition (false positives) or weaken the postcondition (false
negatives).  Slot resolution goes through induced slots so that
`slot_usage` overrides, `slot_uri` overrides and alias-form keys
resolve to the same IRI that `sh:path` emits.

Co-authored-by: jdsika <carlo.van-driesten@vdl.digital>
@jdsika
jdsika force-pushed the feat/shaclgen-presence-implies-value-stacked branch from 7622d70 to bae4a43 Compare October 2, 2026 11:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants