Skip to content

Understanding the NMDC Schema

The nmdc-schema is a framework for describing multi-omics microbiome experiments and the data they produce. We aim to answer questions like:

  • What samples were included?
  • Where were they gathered?
  • What qualities do they have?
  • How were they prepared, so that they would be suitable for sequencing, LC-MS proteomics, metabolomics, etc?
  • How were the results of those analyses interpreted?

All metadata gathered and stored by the NMDC community must validate against the NMDC Schema.

The NMDC Schema is expressed in the LinkML modeling language.

LinkML uses structures like classes, slots (for relationships and properties), types and enumerations to create a blueprint of the NMDC data. The schema is used to define the structure of the data, and to ensure that the data is consistent and interoperable. The schema is expressed in a human-readable format, and with the help of LinkML tooling, can be generated in multiple formats, including JSONSchema (used to support validation on ingest to NMDC), YAML (for ease of editing), and OWL. LinkML schemas generally make good use of terminology and concepts already modeled in external, trusted knowledge centers, especially ontologies, and especially those from the OBO Foundry.

Reuse of existing terminologies and knowledge can be found in several places in the NMDC schema, including the use of the MIxS schema, the EnvO ontology, and the CHEBI ontology. In addition, the NMDC schema elements like slots and classes are annotated with mappings (exact, narrow, related scopes) to these external resources.

For example: 

  doi_value:
    description: >-
      A digital object identifier, which is intended to persistantly identify some resource on the web.
    required: true
    aliases:
      - DOI
      - digital object identifier
    range: uriorcurie
    pattern: '^doi:10.\d{2,9}/.*$'
    examples:
      - value: doi:10.46936/10.25585/60000880
        description: The DOI links to an electronic document.
    exact_mappings:
      - OBI:0002110
    narrow_mappings:
      - edam.data:1188

Asserting element identifiers in the schema with URIs and CURIEs

One consequence of this semantic/linked data orientation is that all schema elements are identified by a URI, most often in the compact CURIe form: a prefix and a local identifier. Even if a class isn't decorated with a class_uri annotation, it will always have a key (in a JSON, YAML or Python Obj sense), which is sometimes reiterated as the name. In that case, the class' URI will be <prefix>:<key>. LinkML schemas should have default prefix assertions, but any element can use a different prefix, as long as an expansion is provided.

Prefixes in LinkML schemas

Prefixes used in a LinkML must be associated with an expansion in the schema (which may include imported modules). Ideally, the expanded URI should be web resolvable, but that is not required. The prefixes can be expanded to base URIs owned by a particular resource, or they can be expanded to base URIs owned by some resolving service, like the bioregistry.

Asserting mappings in the NMDC Schema

As mentioned above, URIs are assigned to most elements of a LinkML schema, either explicitly by the schema authors, or implicitly through the default prefix and the element's key. If an external prefix is used, that means the semantics of the element are identical to the external term, unless otherwise refined. Sometimes it is desirable to associate a LinkML schema element with a term from an external resource, without asserting that the semantics are identical. In this case, a variety of mapping terms can be used.

Adding mappings to a schema element is one of the best and most compact ways to clarify the meaning of that element.

Schema contributors are strongly encouraged to use mappings whose prefixes are already defined in the schema. Schema contributors are always responsible for having a holistic understanding of an external term to be mapped into the schema. This means gaining familiarity with the parent and child terms, as well as any other axioms applied to the term. The EBI Ontology Lookup Service is a good place to look for these details.

When it appears necessary to use a mapping whose prefix isn't already defined in the schema, the contributor is responsible for having a holistic understanding of the external namespace (not just the term to be mapped). There are several ways to start assessing an ontology that is being considered as a source of mappings. If the ontology is in the OBO Foundry, one can look at the OBO Foundry Dashboard.

Asserting identifiers in LinkML data

Generally speaking, the smallest atom of LinkML data is an instance of one class. LinkML slots take values, but always in the context of some class (on the right hand side). LinkML data files are frequently collections of instances of one or more classes. The is no requirement that these classes provide a slot whose value uniquely identifies the instances, but LinkML provides a mechanism that is broadly followed: one slot available in each class is annotated with identifier: true. (Or at least, that's what it would look like in a YAML serialization.) That means that the slot is required in all instances of the class, and that any collection of instances from that class must have unique values in that slot. It is also typical to say that the range of the is type uriorcurie.

Mentioning identifiers in LinkML data

A common pattern in the nmdc-schema is asserting that some identifiable process has inputs and outputs. 

pooling_set:
  - id: pooling:1
    inputs:
      - biosample:1
      - biosample:2
      - biosample:3
  - id: pooling:2
    inputs:
      - biosample:4
      - biosample:5
      - biosample:6

Here we have declared the existence of two pooling processes. A CURIe identifier is asserted for both of them, and three inputs are mentioned for each. The example CURIes above don't necessarily follow any nmdc-schema identifier pattern rules. If these were real CURIes, then the pooling and biosample prefixes would have to be defined in the schema files.

In this case, let's assume definitions for the biosample inputs should be defined elsewhere in an NMDC data set. Cases in which biosamples are mentioned without being defined would be considered violations of the referential integrity. Schema validation does not detect them. See Validating a slot that mentions another class's id below for what is checked, what is not, and which tools close the gap.

Another pattern is saying that something defined within a NMDC data set is equivalent to something defined elsewhere.

biosample_set:
    - id: biosample:1
      gold_biosample_identifiers:
        - gold:1
        - gold:2

In this case, the gold prefix must be defined in the schema. When expanded to URIs via the prefix definitions, these gold_biosample_identifiers would all be web-resolvable.

Constraining mentioned identifiers

We can limit the values that go into any slot by using a pattern constraint. We can also use the id_prefixes constraint to limit the prefixes that are used in whatever slot has been declared to be the identifier of a class.

Constraints written in a class's slot_usage do cascade to subclasses through is_a, so there is no need to restate an inherited value defensively. An earlier version of this page described a belt-and-suspenders practice of re-declaring the uriorcurie range for that reason. No class does that, and restating an inherited value is now treated as something to remove: src/scripts/report_redundant_slot_usage.py finds them and tests/test_no_redundant_slot_usage.py fails CI when a new one appears.

Restating costs something real. A class that repeats a value it already inherits stops tracking the parent, so a later edit upstream silently does not reach it.

One inheritance path does drop values, and it is not this one: a slot that inherits from another slot through slot-level is_a loses a falsy value such as minimum_value: 0. That is linkml issue 3845 and it does not affect class slot_usage.

All prefixes used in the id_prefixes constraint must be defined in the schema. We should have a standing practice of reflecting on the declared id_prefixes, and removing prefixes that haven't been used yet and are not likely to eer be used.

Maintenance of the prefix portions of a pattern will generally require more manual checking. We shouldn't be constraining the values of slots to use a prefix that isn't declared, but no checks are automatically applied.

Validating a slot that mentions another class's id

When a slot's range is a class that is not inlined, the value in the data is the id of an instance of that class. Two different things could be checked about that value, and only one of them is enforced by the schema.

The shape of the id is enforced by the schema. A structured_pattern in the class's slot_usage constrains the value to the id pattern of the intended class. For example, Biosample.slot_usage.associated_studies:

associated_studies:
  required: true
  range: Study
  structured_pattern:
    syntax: "{id_nmdc_prefix}:sty-{id_shoulder}-{id_blade}$"
    interpolated: true

The pattern only takes effect after materialization, so validate against nmdc_schema/nmdc_materialized_patterns.yaml rather than the source schema. In the materialized artifact the syntax above becomes ^(nmdc):sty-([0-9][a-z]{0,6}[0-9])-([A-Za-z0-9]{1,})$, so associated_studies: [xyz] fails. This is why so many slot_usage blocks carry a structured_pattern: it is the only in-schema guard against pointing a slot at an id of the wrong class.

The existence of the referent is not checked by the schema. Schema validation is per-document: a value that matches the pattern but names a record that does not exist will pass. Nothing in LinkML or the generated JSON Schema can see the rest of the database.

That check happens in nmdc-runtime, at submission time. /metadata/json:submit and the workflow-execution endpoints call validate_json(..., check_inter_document_references=True), which uses refscan to scan each incoming document's outgoing references and rejects the payload with HTTP 422 if a target is missing. A reference is satisfied if the target already exists in the database or appears in the same payload, so a submission that creates a study and its biosamples together is accepted while a dangling reference is not.

The same check also runs outside the API:

  • make check-references (src/scripts/check_references.py) reports dangling references in this repo's own data files.
  • refscan run directly scans the production database, and refgraph draws the reference graph it derives.

Runtime behavior verified 2026-08-13 against nmdc_runtime/api/endpoints/metadata.py, nmdc_runtime/api/endpoints/workflows.py, and nmdc_runtime/api/db/mongo.py. That is a different repository and can drift, so re-check those files rather than trusting this paragraph if the answer matters.

Two consequences worth knowing when you write a slot_usage. A structured_pattern whose syntax contains | is asserting one pattern per alternative, so {id_nmdc_prefix}:(bsm|procsm)-... accepts both Biosample and ProcessedSample ids. And an any_of range widens rather than narrows: LinkML reads it as a union with the global range, so any_of: [Biosample, ProcessedSample] on a slot whose global range is NamedThing does not exclude other NamedThing subclasses.

The fuller discussion of these limitations, including why pattern-plus-class ranges complicate the OWL and RDF artifacts, is in the v10 to v11 retrospective.

Using URIs supports scoping and self-documentation

Any class could use any slot with any range to "link to" something external. A Person could have a place_of_birth slot, and that could take unconstrained string or enumerated values like "Switzerland". But that doesn't provide much support for people looking up more information about the place_of_birth. You could create wikidata_place_of_birth and dbpedia_place_of_birth slots and add annotations to the slot to aid in external lookups, but that isn't a good practice if supporting several external targets. A better practice is to have one place_of_birth slot, with the uriorcuire range. Then users can provide values like <http://www.wikidata.org/entity/Q39> or <http://dbpedia.org/resource/Switzerland>. Then there is no ambiguity about the target of the link.

Using CURIes makes things more compact and readable

If the schema contains prefix definitions like wd: <http://www.wikidata.org/entity/> and dbpedia: <http://dbpedia.org/resource/>, then the values can be written as wd:Q39 and dbpedia:Switzerland. This is more compact and readable, but it requires that the prefixes be defined in the schema. Then, more constraint can be imposed with a pattern on the place_of_birth slot like '^(wd|dbpedia):.+$'

Where to find a report of NMDC prefixes

  • project/jsonld/nmdc.context.jsonld (autogenerated as the jsonldcontext output from make gen-project)
  • assets/misc/data_prefix_expansions.context.jsonld (curated based on prefixed observed in MongoDB)

PREFIX, id and CURIe notes

  • not using default_curi_maps any more
  • all prefixes must now be explicit
  • prefix definitions have to account for prefixes and base URIs used in the schema and in the data
  • look out forthe presence of http://example.org/UNKNOWN/ and "example." in the schema, the data and any SPARQL results
  • should prefixes be uppercase or lowercase?
  • it must be consistent. look for precedent
  • bogus emsl UUID prefixes handled with --emsl-biosample-uuid-replacement in anyuri-strings-to-iris
  • keep an eye on validation patterns and id_prefixes
  • MetaCyc expansion assumes we're talking about metacyc reactions and not genes, compounds etc. The same may hold for other under-qualified prefixes

see local/lint.log

use bioregistry, not identifiers.org, BUT

warning Schema maps prefix 'CHEMBL.COMPOUND' to namespace 'https://bioregistry.io/chembl.compound:' instead of namespace 'http://identifiers.org/chembl.compound/' (canonical_prefixes)

warning Schema maps prefix 'rdf' to namespace 'http://www.w3.org/1999/02/22-rdf-syntax-ns#' instead of using prefix ' RDF' (canonical_prefixes) warning Schema maps prefix 'rdfs' to namespace 'http://www.w3.org/2000/01/rdf-schema#' instead of using prefix 'RDFS' ( canonical_prefixes)

Typecodes

Not only does NMDC require the use of the nmdc prefix in the primary identifier for data instances, but we also define a pattern that the local portion of the ids must follow. For example, an example id from the NamedThing class is 'nmdc:mgmag-00-x012.1_7_c1'. The text that comes immediately after the nmdc prefix and colon, and before the first hyphen is the typecode, mgmag in this case. Typecodes must correspond 1:1 to a class in the NMDC schema. The typecodes currently in use are available from the nmdc-runtime API: https://api.microbiomedata.org/nmdcschema/typecodes

Please see identifers for more information on the minting of identifiers in NMDC directly, and for more discussion on identifier resolution and harmonization in NMDC.