Widening base_uri to the namespace doubles the prefix in short forms and CURIEs (COHO_COHO_0000003)

[jamesamcl]

Describe the bug

An ontology whose terms live at <namespace>/PREFIX_nnnnnnn and whose properties live at <namespace>/name cannot be configured so that both of these hold:

1. its terms get the right short form and CURIE (COHO_0000003, COHO:0000003), and
2. all of its own entities, properties included, count as defined by it (isDefiningOntology).

Widening base_uri from <namespace>/PREFIX_ to <namespace>/ fixes 2 and breaks 1: every term gets a doubled prefix. COHO is the example here, EFO has the same IRI layout (see #836).

COHO terms are http://www.ebi.ac.uk/coho/COHO_0000003, its properties are http://www.ebi.ac.uk/coho/isSubCohortOf. Results of loading COHO with each config, preferredPrefix COHO throughout:

base_uri	term short form / CURIE	isSubCohortOf defining	imported terms' CURIEs
["http://www.ebi.ac.uk/coho/COHO_"]	COHO_0000003 / COHO:0000003	false	correct
["http://www.ebi.ac.uk/coho/"]	COHO_COHO_0000003 / COHO_COHO:0000003	true	correct
["http://www.ebi.ac.uk/coho/"] plus shortFormExtractionPattern ^http://www\.ebi\.ac\.uk/coho/COHO_(\d+)$	COHO_0000003 / COHO:0000003	true	broken, see below

With the pattern set, the v1 API obo_id of imported terms became COHO:000002 (ExO_0000002), COHO:00118 (HP_0000118), COHO_:0700096 (MONDO_0700096), COHO:C16632 (NCIT_C16632).

With the wide base_uri and no pattern, the v1 individuals endpoint also returned obo_id COHO:COHO:0000003, and properties get the short form COHO_isSubCohortOf.

Listing both base URIs (…/coho/COHO_ and …/coho/) does not help either, for the reason under "Cause" below.

To Reproduce

1. Load COHO with the config below (this is dataload/configs/coho.json as of aa121f8 on the branch of COHO browsing: reified annotation display, aggregation hierarchy, subset tree filter #1443, reduced to the relevant keys).
2. Look at any COHO term or individual, for example http://www.ebi.ac.uk/coho/COHO_0000003.
3. Its short form is COHO_COHO_0000003 and its CURIE COHO_COHO:0000003.

{
  "ontologies": [
    {
      "id": "coho",
      "preferredPrefix": "COHO",
      "base_uri": ["http://www.ebi.ac.uk/coho/"],
      "ontology_purl": "https://raw.githubusercontent.com/EBISPOT/cohort-ontology/refs/heads/main/coho.owl"
    }
  ]
}

Observed with the dataload image built from the #1443 branch at aa121f8. The two files involved, ShortFormAnnotator.java and OntologyBaseUris.java, are byte-identical on dev (30a9ec6), so dev behaves the same.

Expected behavior

With base_uri ["http://www.ebi.ac.uk/coho/"], or with both base URIs listed:

• http://www.ebi.ac.uk/coho/COHO_0000003 has short form COHO_0000003 and CURIE COHO:0000003
• http://www.ebi.ac.uk/coho/isSubCohortOf counts as defined by COHO
• imported terms keep their own CURIEs (MONDO:0700096, HP:0000118, …)

Cause

Two separate things in dataload/rdf2json/src/main/java/uk/ac/ebi/rdf2json/annotators/ShortFormAnnotator.java:

1. extractShortForm (lines 118-122) takes the first base URI the IRI starts with and returns preferredPrefix + "_" + rest:

   for (String baseUri : ontologyBaseUris) {
       if (uri.startsWith(baseUri) && preferredPrefix != null) {
           return preferredPrefix + "_" + uri.substring(baseUri.length());
       }
   }

   With the wide base URI the rest is COHO_0000003, so the prefix is doubled. The base URIs come from OntologyBaseUris.getOntologyBaseUris, which returns a HashSet, so when several base URIs match an IRI, the winner depends on hash order, not on specificity or on the order in the config. For COHO's two URIs, http://www.ebi.ac.uk/coho/ always iterates before http://www.ebi.ac.uk/coho/COHO_ (buckets 2 and 14 of 16), which is why listing both gives the doubled prefix as well.

2. When shortFormExtractionPattern is set (line 53), the CURIE of every entity whose short form contains _ is built as preferredPrefix + ":" + shortForm.substring(preferredPrefix.length() + 1), whether or not that entity's IRI matched the pattern. An imported ExO_0000002 becomes COHO: + "ExO_0000002".substring(5) = COHO:000002. So the pattern cannot be used as a workaround in an ontology that imports terms.

Suggested fix

• In extractShortForm, match the longest base URI first (sort by length descending, or pick the longest match), so that a config listing both …/coho/COHO_ and …/coho/ behaves predictably.
• Possibly also: if the rest after the base URI already starts with preferredPrefix + "_", don't prepend the prefix again. Then the wide base URI alone is enough.
• Apply the pattern's CURIE rule only to entities whose IRI matched shortFormExtractionPattern.
• ShortFormAnnotator has no unit tests at the moment. The three rows of the table above, plus an imported term, would make a small test.

Additional context

• Term is showing as imported but it isn't #836 is the same layout in EFO: http://www.ebi.ac.uk/efo/reason_for_obsolescence shows as imported because base_uri is http://www.ebi.ac.uk/efo/EFO_. Widening it to http://www.ebi.ac.uk/efo/ was proposed there. With the current code that would turn EFO_0000001 into EFO_EFO_0000001.
• COHO browsing: reified annotation display, aggregation hierarchy, subset tree filter #1443 sets the wide base_uri for COHO in aa121f8, so as it stands it produces the doubled short forms wherever COHO is loaded with that config.
• Earlier short form and CURIE issues with other causes: Some OBO ID/CURIE has unwanted prefix of wikipathways_ #978, Incorrect short form for MeSH terms #935.