chore: feature integration branch for the ENVITED-X pipeline

AGENTS.md

@@ -23,6 +23,8 @@ All commands use `uv run` prefix (e.g., `uv run pytest`).
 * Do not "fix" issues by changing or weakening test conditions. Try harder, or ask questions if a test fails.
 * Avoid try/except blocks, these can mask bugs
 * Failing fast is a good principle
+* Optional dependencies are the documented exception to the two rules above. Guard the import, not the spec: `try: import x` / `except ImportError as exc: raise ImportError("x is required. Install with: pip install 'linkml[extra]'") from exc` - see `generators/bigquerygen.py`. `importlib.util.find_spec` raises under the test fixture below, which hides your message.
+* Testing that an optional dependency is absent is not a mock test: use the `mock_missing_import` fixture in `tests/conftest.py`. Do not build isolated environments in CI for this.
 * Follow the DRY principle
 * Avoid repeating chunks of code, but also avoid premature over-abstraction
 * Declarative principles are favored

docs/generators/json-schema.rst

@@ -378,6 +378,72 @@ will generate:
 LinkML also supports `Structured patterns <https://w3id.org/linkml/structured_pattern>`_, these are
 compiled down to patterns during JSON Schema generation.
 
+Dictionary key constraints (propertyNames)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+A multivalued, inlined slot whose range class has an identifier slot is
+compiled to a JSON object keyed by that identifier (see *Inlining* above).
+When the identifier slot carries string-applicable constraints, they are
+emitted as a `propertyNames <https://json-schema.org/understanding-json-schema/reference/object.html#property-names>`_
+schema on the container object, so the *keys* of the dictionary are validated,
+not just the values:
+
+.. code-block:: yaml
+
+    slots:
+      tags:
+        range: Tag
+        multivalued: true
+        inlined: true
+      uid:
+        identifier: true
+        pattern: "^(0|[1-9][0-9]*)$"
+
+generates on the container:
+
+.. code-block:: json
+
+    "tags": {
+       "additionalProperties": {"$ref": "#/$defs/Tag"},
+       "propertyNames": {"pattern": "^(0|[1-9][0-9]*)$"},
+       "type": "object"
+    }
+
+The constraints carried over from the key slot are the ones applicable to JSON
+Schema strings, because object keys are always strings (`JSON Schema Core
+2019-09, §9.3.2.5 <https://json-schema.org/draft/2019-09/json-schema-core.html#rfc.section.9.3.2.5>`_):
+
+* ``pattern`` -- whether written directly on the slot, resolved from a
+  ``structured_pattern``, or inherited from the slot's ``range`` type (for
+  example an identifier with ``range: ncname``, or a user-defined type that
+  declares a ``pattern``);
+* ``equals_string_in``, emitted as ``enum``;
+* a string ``equals_string``, emitted as ``const``.
+
+The emitted key pattern is always the same one that applies to the identifier
+*inside* the value object, so a key and a redundantly repeated in-object
+identifier are now validated identically.
+
+Numeric constraints -- ``minimum_value``/``maximum_value``, and the numeric
+``const`` produced by ``equals_number`` -- are deliberately **not** carried
+over: they cannot be satisfied by a string key, and a numeric ``const`` would
+reject every key. The ``allOf`` produced by a ``range_expression``, and the
+permissible values of an ``enum``-ranged identifier, are likewise out of scope.
+
+``propertyNames`` composes conjunctively with ``additionalProperties``, so keys
+and values are constrained independently. It is emitted only when the key slot
+actually carries one of the constraints listed above; an unconstrained key slot
+produces exactly the same output as before.
+
+.. note::
+
+   Because type-level patterns are included, an identifier slot whose range is
+   ``ncname`` (or another pattern-bearing type) gains a ``propertyNames``
+   entry even if the slot itself declares no constraint. The generated schema
+   becomes stricter, but only in ways the model already required: data whose
+   keys satisfy the declared identifier type is unaffected.
+
+
 Rules
 ^^^^^
 

docs/generators/jsonld-context.rst

@@ -85,6 +85,31 @@ to gen-prefix-map:
 
    gen-prefix-map --flatprefixes personinfo.yaml > personinfo.prefixmap.json
 
+URIs as IRIs or as literals
+---------------------------
+
+By default a ``uri`` or ``uriorcurie`` slot is coerced to ``xsd:anyURI``, a typed
+literal. ``--xsd-anyuri-as-iri`` coerces it to ``@id`` instead, so the value becomes an
+IRI node, matching the ``sh:nodeKind sh:IRI`` the SHACL generator emits. The OWL
+generator accepts the same flag.
+
+The flag applies to ``uri``, ``uriorcurie`` and the types derived from them. A type
+derived from ``string`` that declares ``uri: xsd:anyURI`` stays a typed literal with or
+without it. Use one for a URI reference that is data rather than a link, such as a file
+path that may be relative: as ``@id`` it would be resolved against the document base.
+
+.. code-block:: yaml
+
+    types:
+      FilePath:
+        typeof: string
+        uri: xsd:anyURI     # always "@type": "xsd:anyURI", sh:datatype xsd:anyURI
+    slots:
+      homepage:
+        range: uri          # "@id" with --xsd-anyuri-as-iri
+      file_path:
+        range: FilePath
+
 
 Docs
 ----

docs/generators/owl.rst

@@ -67,6 +67,61 @@ Mapping
 
 .. note:: The current default settings for ``metaclasses`` and ``type-objects`` may change in the future
 
+Prefix normalization
+^^^^^^^^^^^^^^^^^^^^
+
+Schemas sometimes declare non-standard aliases for well-known namespaces
+(e.g. ``sh1:`` for the SHACL namespace, or a versioned alias for ``skos:``).
+By default these aliases are carried through into the generated artifact.
+
+Use ``--normalize-prefixes`` to remap declared prefixes whose namespace IRI
+matches a well-known vocabulary to that vocabulary's conventional name in the
+output (``owl``, ``rdf``, ``rdfs``, ``skos``, ``sh``, ``xsd``, ...):
+
+.. code:: bash
+
+   gen-owl --normalize-prefixes schema.yaml
+
+The mapping is a static, version-independent table; namespace IRIs that are
+not in the table are left untouched. The option is also available on
+``gen-shacl`` and ``gen-jsonld-context``.
+
+Metadata values that are IRIs
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Schema metadata and ``annotations`` become annotation triples on the ontology, class or
+property. Their values are literals, except for properties whose value is a resource by
+their own specification: the OWL ontology properties ``owl:versionIRI``,
+``owl:priorVersion``, ``owl:backwardCompatibleWith`` and ``owl:incompatibleWith``;
+``rdfs:seeAlso`` and ``rdfs:isDefinedBy``; the DCMI terms that relate a resource to
+another (``dcterms:license``, ``dcterms:conformsTo``, ``dcterms:references``,
+``dcterms:source``, ``dcterms:relation`` and its sub-properties); the DCMI agent terms
+``dcterms:creator``, ``dcterms:contributor``, ``dcterms:publisher`` and
+``dcterms:rightsHolder``; and ``prov:wasDerivedFrom``, ``prov:wasRevisionOf`` and
+``prov:hadPrimarySource``.
+
+A value of one of these is emitted as an IRI when it is an absolute IRI or a CURIE with a
+declared prefix; any other value stays a literal:
+
+.. code-block:: yaml
+
+    id: https://example.org/onto/v2
+    license: https://www.eclipse.org/legal/epl-2.0/
+    annotations:
+      owl:versionInfo: v2
+      owl:priorVersion: ex:onto/v1
+      prov:wasDerivedFrom: https://example.org/releases/tag/v2.0.0
+
+.. code-block:: turtle
+
+    <https://example.org/onto/v2> a owl:Ontology ;
+        owl:versionInfo "v2" ;
+        owl:priorVersion <https://example.org/onto/v1> ;
+        prov:wasDerivedFrom <https://example.org/releases/tag/v2.0.0> ;
+        dcterms:license <https://www.eclipse.org/legal/epl-2.0/> .
+
+``dcterms:identifier`` stays a literal: DCMI gives it the range ``rdfs:Literal``.
+
 Enums and PermissibleValues
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -135,6 +190,31 @@ You can control enum and permissible value representation directly in your schem
             implements:
               - rdfs:Literal
 
+**Typing permissible values with ``instantiates``** - A permissible value can be an
+instance of a class from another schema, for example a named individual that an open
+vocabulary defines the class of. Each ``instantiates`` value becomes an ``rdf:type`` of
+the permissible value's IRI:
+
+.. code-block:: yaml
+
+    enums:
+      LinkCategory:
+        implements:
+          - owl:NamedIndividual
+        permissible_values:
+          isLicense:
+            meaning: ex:isLicense
+            instantiates:
+              - vocab:LicenseCategory
+
+.. code-block:: turtle
+
+    ex:isLicense a owl:NamedIndividual, ex:LinkCategory, vocab:LicenseCategory .
+
+The enum still closes its own class with ``owl:oneOf``; the class it instantiates stays
+open. ``instantiates`` is ignored, with a warning, on a permissible value rendered as a
+literal.
+
 **Using URIs vs. text for permissible values:**
 
 .. code-block:: yaml
@@ -311,6 +391,48 @@ Other examples
   translation of Biolink schema to OWL
 
 
+Deterministic output
+^^^^^^^^^^^^^^^^^^^^
+
+``gen-owl`` output is deterministic by default. The graph is canonicalized with
+`RDFC-1.0 <https://www.w3.org/TR/rdf-canon/>`_ before serialization, so repeated
+runs over the same schema -- and any two isomorphic graphs -- produce
+byte-identical Turtle. No flag is needed, and checked-in artifacts do not churn
+between runs.
+
+RDFC-1.0 numbers blank nodes sequentially (``_:c14n0``, ``_:c14n1``, ...) in
+canonical order. That is stable for a fixed graph, but inserting a single
+statement can shift the numbering of every blank node ordered after it, so an
+unrelated one-line schema edit may rewrite large parts of the file. Pass
+``--diff-stable`` to derive each label from the node's own neighbourhood
+instead, so that only the blank nodes an edit actually touches are renamed:
+
+.. code:: bash
+
+   gen-owl --diff-stable schema.yaml
+
+Both modes are deterministic and yield isomorphic graphs; only the choice of
+label differs. ``--diff-stable`` is off by default because turning it on
+relabels the blank nodes in existing output once.
+
+The same ``--diff-stable/--no-diff-stable`` option is available on ``gen-rdf``,
+``gen-shacl`` and ``gen-shex``.
+
+Graphs that are not standard RDF -- literal predicates, as produced by
+``gen-shacl`` in annotation mode, or relative IRIs such as the metamodel's
+``bibo:status <testing>`` -- cannot be canonicalized under RDFC-1.0. Those fall
+back to plain rdflib serialization, with blank-node labels canonicalized by
+``rdflib.compare.to_canonical_graph``. Those labels are content-derived rather
+than run-local, so the fallback remains reproducible across processes. It emits
+an ``RDFCanonicalizationWarning``, and ``--diff-stable`` has no effect on that
+path -- it warns rather than silently ignoring the request.
+
+Canonicalization itself is implemented by the
+`diffable-rdf <https://github.com/ASCS-eV/diffable-rdf>`_ library;
+``linkml_runtime.utils.rdf_canonicalize.canonicalize_rdf_graph`` is a thin
+adapter that re-emits the library's log warnings as Python warnings.
+
+
 Docs
 ----