Migrate the `medic-client/contacts_by_reference` call sites onto the `shortcodes` and `externalRefs` qualifiers

[witash]

Title: Migrate the medic-client/contacts_by_reference call sites onto the shortcodes and externalRefs qualifiers

---

title: Migrate the medic-client/contacts_by_reference call sites onto the shortcodes and externalRefs qualifiers
type: improvement
priority: medium
domain: data-sync
labels:

• cht-datasource
• migration

---

Description

Split out of #10976, which is scoped to adding the qualifiers themselves.

#10976 adds byShortcodes and byExternalRefs to cht-datasource and
exposes them on both contact list endpoints, but migrates no call site.
Every one of the nine existing medic-client/contacts_by_reference readers
changes behaviour when it moves onto the datasource, and each of those changes
needs a decision that the qualifier PR should not be making on its own.

This ticket is that work. It is not a mechanical sweep: the four causes below
affect different callers differently, one of them is a correctness risk for
shortcode generation, and two are visible to configurations in the field.

The four behaviour changes

1. Permissions. The datasource's remote context reads through
   /api/v1/contact, which is gated by can_view_contacts. The CouchDB proxy
   the callers use today is not. Any online caller that migrates newly
   requires that permission. The webapp picks the remote context for
   online-only users (cht-datasource.service.ts) and the admin app always
   uses it.
2. isContact filtering. contacts_by_reference emits any doc whose
   type is in its hard-coded list, including national_office and
   type: contact docs whose contact_type is not in settings. Both
   datasource paths filter rows through isContact, so a migrated caller
   stops seeing those docs. For most callers that is a harmless narrowing.
   For isIdUnique it is the opposite: a shortcode held by a filtered doc
   looks free, so the id generator can hand it out twice.
3. Loss of the key→id association. Six of the nine callers send a batch
   of keys and read row.key[1] off the result to work out which key each
   contact matched. The datasource returns a page of ids or docs with no key
   attached. A migrated batch caller either re-derives the match from the
   returned doc (patient_id / place_id / rc_code), which means using the
   doc path and paying include_docs, or issues one call per key.
4. Input handling. byShortcodes and byExternalRefs reject blank,
   padded and non-string entries rather than passing them through to a query
   that can only ever match nothing. Callers that today hand the view a
   numeric shortcode or an undefined reference get a thrown error where they
   used to get a silent empty result — or, if they add String() at the
   boundary, start matching numeric references that never matched before.

Two callers also have no datasource context to migrate onto (see below).

Technical Context

Components:

• webapp/src/ts/services/get-subject-summaries.service.ts — processReferences
• admin/src/js/services/get-subject-summaries.js — processReferences
• admin/src/js/services/message-queue.js — getSubjectsAndRegistrations
• shared-libs/lineage/src/hydration.js — contactUuidByShortcode
• shared-libs/transitions/src/lib/utils.js — getContact (behind
  getContactUuid / getContact)
• shared-libs/transitions/src/transitions/utils.js — isIdUnique
• shared-libs/transitions/src/transitions/update_clinics.js — getContactByRefid
• shared-libs/rules-engine/src/pouchdb-provider.js — contactsBySubjectId
• api/src/services/replication/authorization.js — findContactsByReplicationKeys

Existing References:

• Add byShortcodes and byExternalRefs qualifiers to Contact.v1 #10976 — the qualifier ticket this one is split out of.
• Migrate the medic-client/contacts_by_phone call sites onto the phones qualifier #11442 — the equivalent follow-up split out of Add byPhones qualifier to Contact.v1 #10973, for the
  contacts_by_phone callers. Same causes 1, 2 and 4; this ticket adds 3.
• Refactor code that is loading contacts via shared-libs/lineage to use cht-datasource  #10019 / PR #10026 — an
  earlier, closed attempt to move shared-libs/lineage onto the datasource.

What will exist to migrate onto (from #10976):

• Qualifier.byShortcodes([...]), Qualifier.byExternalRefs([...]),
  Contact.v1.getUuidsPage / getUuids / getPage / getAll
• the imperative surface: getUuidsPageByShortcodes, getPageByShortcodes,
  getByShortcodes and the ByExternalRefs equivalents on
  getDatasource(ctx).v1.contact
• GET /api/v1/contact/uuid and GET /api/v1/contact with ?shortcode= and
  ?external_ref=, both accepting a comma-joined list

Where a data context already exists: api/src/services/data-context.js,
shared-libs/transitions/src/data-context.js,
admin/src/js/services/data-context.js, the webapp's
CHTDatasourceService. shared-libs/lineage and shared-libs/rules-engine
have none: lineage is built from (Promise, DB) and the rules engine's
provider from a PouchDB instance.

Requirements

Call sites

Call site	Today	What migration changes	Decision needed
webapp/src/ts/services/get-subject-summaries.service.ts:118	{ keys: [['shortcode', ref], ...] }, ids only; findSubjectId matches row.key[1] === id.toString()	Cause 3: needs the doc path to map each summary's reference back to a contact, so the ids-only query becomes include_docs. Cause 1 for online users. Cause 4: a numeric reference is sent raw today and never matches the view's String() key; coercing at the boundary makes it start matching.	Doc path or per-reference calls. Accept the new permission for online users. Whether numeric references should resolve (they arguably always should have).
admin/src/js/services/get-subject-summaries.js:110	Same as the webapp copy	Same, and the admin app is always online so cause 1 always applies. admin/src/js cannot use async/await.	Same.
admin/src/js/services/message-queue.js:135	{ keys: referenceKeys }, ids only; findIdByKey on row.key[1]	Cause 3 and cause 1. Sits in the same $q.all as the reports_by_subject query that #<11150-migration-issue> migrates; a role reaching the message queue without can_view_contacts gets a rejected page rather than one without patient/place names.	Doc path or per-key calls. Same permission decision as #11442's getRecipients in the same file; make it once. Sequence with the reports_by_subject migration of this function.
shared-libs/lineage/src/hydration.js:248 (contactUuidByShortcode)	{ keys }, ids only; row.key[1] to build a shortcode→uuid Map, falling back to the shortcode itself	No data context: the lib is constructed with (Promise, DB) by api, sentinel, transitions and admin. Migrating means adding a context to the factory signature at every construction site, which is the #10019 problem. Cause 3 as well.	Whether to give lineage a data context here or leave this caller on db.query with a comment pointing at a lineage-specific ticket. Recommend the latter.
shared-libs/transitions/src/lib/utils.js:120 (getContact)	{ key: ['shortcode', id], include_docs }, warns on rows.length > 1, returns rows[0].doc or .id	Rows newly isContact-filtered (cause 2). Keeping the "more than one contact" warning needs limit: 2 rather than 1. Cause 4: shortCodeId is guarded for falsiness but not type; a numeric value throws instead of matching nothing.	Confirm the filter. Keep the warning or drop it. Coerce or guard at the boundary.
shared-libs/transitions/src/transitions/utils.js:62 (isIdUnique)	{ key: ['shortcode', id] }, !rows.length	Cause 2 is a correctness risk here. The id generator uses this to avoid re-issuing a shortcode. A shortcode held by a national_office doc or an unconfigured type: contact doc is in the view but filtered by the datasource, so it reports as unique and gets assigned again.	Either leave this caller on db.query with a comment, or accept the risk explicitly with a release note. Recommend leaving it.
shared-libs/transitions/src/transitions/update_clinics.js:26 (getContactByRefid)	{ key: ['external', doc.refid], include_docs, limit: 1 }, then contactTypesUtils.getContactType	Already discards non-contacts, so cause 2 costs nothing. refid is upper-cased on save (#5373), so the builder's upper-casing is a no-op. Cause 4: an undefined refid is sent as ['external', null] today and matches nothing; byExternalRefs([undefined]) throws, so guard before calling. Already has dataContext in scope.	Lowest risk of the nine. Migrate; add the guard.
shared-libs/rules-engine/src/pouchdb-provider.js:57 (contactsBySubjectId)	{ keys, include_docs }; returns matched doc ids plus the subject ids that matched nothing, using row.key[1] to tell them apart	No data context: the provider takes a PouchDB instance. Cause 3: which subject ids missed has to be re-derived from the returned docs' patient_id / place_id. Offline hot path; the provider already chunks keys at MAX_QUERY_KEYS.	Whether to inject a data context into the provider (the webapp's rules-engine.service.ts would supply it) or leave this caller with a comment. Depends on the same decision for the reports_by_subject calls in this file (#<11150-migration-issue>).
api/src/services/replication/authorization.js:456 (findContactsByReplicationKeys)	{ keys }, row.key[1] to map each replication key to doc ids, falling back to the key as a doc id, then allDocs	Security-sensitive. Cause 2: a subject that is a filtered doc drops out of the replication-key resolution, and its shortcode falls through to allDocs as a doc id and misses, so the reports about it can leave the user's replication scope. Cause 3 as well. Has dataContext available.	Confirm with whoever owns replication that narrowing to configured contacts is acceptable here, or leave on db.query. Do not migrate this one on the strength of the sweep alone.

Migrate every site whose decision comes out in favour. Any site deliberately
left on db.query keeps a comment pointing at whatever ticket covers it, as
#11151 did for its aggregate callers.

Tests

• Each migrated caller keeps its existing suite green. The suites that assert
  db.medic.query arguments — shared-libs/transitions/test/unit/utils.js,
  shared-libs/transitions/test/unit/patient_registration.js,
  admin/tests/unit/services/message-queue.spec.js,
  webapp/tests/karma/ts/services/get-subject-summaries.service.spec.ts,
  api/tests/mocha/services/replication/authorization.spec.js — assert the
  datasource call instead. Do not keep both.
• Cover each behaviour change that is accepted with a test that pins the new
  behaviour: unconfigured type: contact doc → not returned; numeric
  reference → whatever is decided; undefined refid → no lookup.
• For every batch caller that moves to the doc path, cover the association:
  N keys in, each summary/message/subject matched to the right contact, and a
  key that matches nothing handled the way it is today.
• Integration: tests/integration/sentinel/transitions/registration.spec.js
  and tests/integration/transitions/sentinel-api-transitions.spec.js stay
  green without modification.

Acceptance Criteria

• Every migrated call site reads through cht-datasource; any site left
  behind carries a comment saying why and which ticket covers it.
• grep -rnE "query\(['\"]medic-client/contacts_by_reference['\"]" webapp/src admin/src api/src shared-libs/*/src
  returns only the deliberately-skipped sites. The view name legitimately
  survives in shared-libs/cht-datasource/src/local/contact.ts and its tests.
• npm run unit-shared-lib, npm run unit-api, npm run unit-admin and
  npm run unit-webapp pass.
• Every accepted behaviour change is in the release notes.

Constraints

• admin/src/js is bundled through ng-annotate's 2015 acorn and cannot use
  async/await. Both admin callers stay promise-chained.
• Do not make byShortcodes coerce or trim. Coercion is the caller's job;
  that is the parity constraint inherited from Add byShortcodes and byExternalRefs qualifiers to Contact.v1 #10976.
• Do not change ddocs/medic-db/medic-client/views/contacts_by_reference/map.js.
  Emitting the doc type as the view value would let isContact run on
  row.value and save the uuid path an include_docs, and emitting the
  matched field would restore the key association without docs — both are
  ddoc changes and a reindex, so out of scope here.
• Do not give shared-libs/lineage or shared-libs/rules-engine a data
  context as a side effect of this sweep. If either is decided on, it is its
  own change with its own ticket.
• Keep the permission question a deliberate decision, not a side effect. If a
  caller migrates and starts requiring can_view_contacts, that goes in the
  release notes.

References

Similar Implementations:

• Migrate the medic-client/contacts_by_phone call sites onto the phones qualifier #11442 — the contacts_by_phone caller migration, same shape of ticket
• feat(#11151): add byForm qualifier to Report.v1.getUuidsPage #11327 — byForms, whose out-of-scope
  callers were split into Add an aggregate API surface to Report.v1 (countByForm) for the grouped reports_by_form callers #11328 the same way

Documentation:

• ddocs/medic-db/medic-client/views/contacts_by_reference/map.js
• shared-libs/cht-datasource/README.md