Skip to content

Annotation question extensibility: architecture and delivery map

Decision record for how SyRF's annotation-question model grows. Individual features ship in focused PRs, each aligned to this document. The underlying ideas are in response-modes-and-metadata-context.md and the original plan 2026-08-05-configurable-response-modes-and-metadata.md; this document records what was decided on 2026-08-11.

Decisions

  1. Vehicle: additive on live schema-v0. New capabilities are optional, additive fields on today's AnnotationQuestion (and response documents), the way ADR-011 added ConditionalParentAnswers. Nothing waits on the dormant QM v2 stack; when QM v2 revives, these fields become AQVersion content properties (edits create a new AQVersion) without re-design.
  2. Rendering surface: AF2 first. AF2 is in active delivery (Phase 4, PRs #2731–#2737 et seq.). New reviewer-facing UI — the outline panel, response-mode chooser, metadata display — is built in AF2. The legacy v1 form receives only what it already has: repeatable-group chrome driven by the answer label (with heuristic fallback). We do not grow the form AF2 is replacing.
  3. Branch overview: persistent outline/navigation panel (not a per-question summary list). It lists branch questions and their branches with status, and navigates on click. Because it spans the whole form it is an AF2 feature, integrated with AnnotationFormV2Store's open-unit/paging state — a jump must open the target unit/page, honouring AF2's bounded rendering rather than assuming the target is mounted.
  4. Umbrella doc lives here (this file), merged via the same PR that promoted the design context, so implementation PRs cite a tracked, merged decision.
  5. requiresReason is carried by a platform-reserved _reason metadata key, injected into the definition as a required, mode-scoped field whenever requiresReason is set — not by a first-class reason property on AnnotationResponse. One validation path instead of two, an export column from the existing q__<key> scheme, and no extra persisted field needing its own writer-compatibility floor. It follows that the explanation is exactly as strong as mode-scoped metadata validation; if that turns out too weak to enforce a required reason, this decision is what should be revisited, not patched around.
  6. Required metadata binds an ordinary value only. A question-level required field is not enforced when the reviewer selects a response mode; requirements that should apply to a mode are declared on mode-scoped definitions. Asking for the unit of a value the reviewer has just recorded as not reported is incoherent, and that is the case response modes exist to serve. Chosen as a blanket rule over per-field applicability because blanket → per-field (appliesTo) is additive later, while per-field → blanket would invalidate configuration already written.
  7. Every response records the definitionVersion it was written against. A monotonic integer on the response, stamped at write time. Schema-v0 has no immutable AQVersion, so without it a later definition edit leaves export and audit unable to tell which semantics a historical answer was recorded under. One integer per response is the cheap half of that problem, and it keeps the expensive half open: whether definitions are frozen once used or snapshotted per response is not decided (see Open questions).

The spine and its optional extensions

The question definition remains SyRF's nested tree. Every addition is an optional field; a definition using none of them is byte-for-byte today's behaviour.

AnnotationQuestionDefinition (schema-v0 today; AQVersion content under QM v2)
├─ existing: text, description, controlType, options, Optional,
│            Multiple/AnswerArray, Target (+ ADR-011 conditional parents)
├─ [1] answerLabel?    — singular noun for one repeatable answer instance
├─ [2] responseModes?  — project-defined alternatives to answering
│        platform supplies mechanics only:
│        suppressDescendants?, requiresReason?, mode-scoped metadataFields?
└─ [3] metadataFields? — typed response qualifiers (unit, evidence, confidence)
         key, label, type, options?, required?, display: inline | details

AnnotationResponse
├─ value XOR responseModeId   — mutually exclusive, never both
├─ metadata?                  — validated against the definition's fields
└─ definitionVersion?         — monotonic integer stamped at write (Decision 7)

Derived runtime state (computed, never stored)
└─ available | suppressedByAncestor (+ which ancestor)

Hard constraints

These are load-bearing; every implementation PR must respect them.

  • Writer-compatibility floor (ADR-011 precedent). Project writes replace the full Mongo document, so writers that do not know a field silently drop it. For every additive field: persistence round-trip (API + PM readers and writers) deploys before anything writes the field. Rollback below that release loses data written since.
  • Boundary discipline. Metadata fields never have children or conditional logic. Only response modes may affect descendants. This is the anti-"second form engine" rule; it is what keeps each PR small.
  • Derived state is never stored. suppressedByAncestor is computed from the hierarchy, conditions, and selected modes. It is not a reviewer answer and must never overwrite one.
  • Suppression is scoped to the answer instance, not the question. A repeatable parent stores each instance as its own Annotation, with descendants linked by annotation ParentId/Children. Suppression state keyed by questionId alone cannot distinguish one branch from its siblings, so selecting a suppressing mode in branch 2 would suppress descendants in branch 1 as well. Key it by the ancestor annotation instance.
  • Preserved means the backend must not tree-shake it. Today ExtractionInfo.AddAnnotations removes every existing annotation whose question is in the submitted stage before adding the submission. If AF2 omits suppressed descendants, that path deletes the answers this document promises to preserve. The domain change lands before the reviewer UI (see the delivery map), with a suppress-after-answer regression test.
  • Preserving an answer creates an export hazard — resolve it there too. The constraint above keeps a descendant answer when an ancestor later suppresses it. WideDataFormatRowWriter.AddAnswerQuadFromGroup exports every stored matching annotation, so that preserved answer would be emitted as though it were live and silently corrupt a downstream analysis — a worse outcome than deleting it, because it is indistinguishable from real data. Exports must resolve suppression: either omit inactive values or expose an unambiguous activity/status column. A suppress-after-answer export test belongs alongside the submission one.
  • Authoring is gated on actual AF2 eligibility, not just the flag. The schema flag is environment-wide, but AF2 fails closed to v1 for extraction stages, Experiment questions, unsupported question graphs and other eligibility failures, and eligibility is evaluated per host (stage-review, preview, reconcile) rather than once per environment, so "what the renderer supports" is not a single answer. Without a project/stage capability gate, an administrator can configure response modes and required metadata for a question whose reviewers permanently receive v1, which has no controls for supplying them — the reviewer then cannot complete the form and cannot see why. The designer surface must check the same eligibility the renderer does.
  • Metadata keys must not collide with generated export stems. Keys are project-defined and exported as q__<key>, so a key of value or response_mode would collide with the generated core columns. Reserve those stems, and validate at definition-save time rather than at export time — failing an export is far too late to tell an administrator their key is invalid. Project-defined keys must also not start with _: that prefix is reserved for platform-injected fields such as Decision 5's _reason, and without the rule a later platform key would silently collide with project data already written under the same name. Because the export scheme is q__<key>, the leading underscore is stripped when generating the column, so _reason exports as q__reason rather than q___reason — and that stripped form is itself reserved, or a project key of reason would collide with it.
  • Project-defined labels, stable machine keys. SyRF ships no global "Not reported" enum; projects define labels. IDs/keys are stable for persistence, export, and audit even if labels are edited or localised. (Covidence's own extraction guidance — never leave blanks; record "Not applicable/Not reported/Unclear" — confirms the modes are standard practice and the labels review-specific.)
  • Feature-flag discipline. answerLabel: no flag — additive, render falls back to the existing heuristic, no rollback floor beyond the writer rule. Response modes + metadata fields: flagged (schema-driven flag via env-mapping.yaml), enabled per environment only after compatible writers are deployed; reviewer-side rendering is AF2-only, so project-level enablement should track AF2 serving the annotation form.

The extensions

[1] Answer label (answerLabel)

A short singular noun phrase ("experimental protocol", "treatment cohort") that names one repeatable answer instance. It drives all repeatable-group chrome — group headings, Add/Delete buttons, delete-confirmation copy, status text — so wording can never disagree across surfaces.

  • Stored on the definition. The stored value is always used when present.
  • Authoring: field visible only for Multiple && !AnswerArray. Prefilled by the existing derivation heuristic (repeatableAnswerConcept) and it keeps tracking the question text until the author edits the label manually; thereafter their value wins.
  • Rendering: answerLabel?.trim() || repeatableAnswerConcept(question) — the heuristic remains as legacy fallback, so existing questions render unchanged and no migration is needed.
  • Consumed by v1 chrome today and by AF2 (including the outline panel).

[2] Branch outline / navigation panel (AF2)

A persistent panel listing each repeatable (branching) question and its branches: answerLabel + branch ordinal, primary answer excerpt, and the non-colour status (complete / incomplete / N errors). Clicking navigates.

  • AF2-only. Navigation goes through AnnotationFormV2Store: open the target unit/page first, then focus — AF2's virtualisation means the target may not be mounted, and AF2 already requires explicit continuation into unmounted ranges.
  • Also the natural home for "why is this question missing?" explanations once suppression exists (derived state, ancestor named).
  • The v1 form keeps its in-place grouping (headings, chip, containment work); it does not get the panel.

[3] Response modes (responseModes)

Project-defined alternatives to the ordinary answer ("Not reported", "Not applicable", "Unable to determine" — labels the project's own). Platform mechanics only: optionally suppress descendants, optionally require a reason, optionally attach mode-scoped metadata fields.

  • Reviewer UI (AF2): ordinary control stays central; a quiet "I cannot provide this answer" disclosure reveals the mode choice only where configured. Selecting a mode disables/replaces the ordinary control. Suppressed descendants leave the active form and never raise required-field errors; audit views explain the suppression.
  • Response: responseModeId mutually exclusive with value.
  • If a suppressing mode is chosen after descendants hold answers, those answers are preserved inactive (not cleared) pending explicit reconciliation — destructive clearing needs its own decision.

[4] Metadata fields (metadataFields)

Typed, admin-defined qualifiers stored alongside a response: unit, evidence location, reviewer confidence, note. display: inline for inseparable qualifiers (units, beside the control); display: details for documentary context (under a disclosure).

Test for what belongs here: if the information independently affects the review's substantive logic it is an ordinary question; if it qualifies or documents an existing response it may be metadata.

Designer (authoring) surface

The live question editor (annotation-question-designer) gains, with progressive disclosure:

  • the answer-label field (repeatable questions only, prefill-tracking);
  • later, a collapsed Response handling section: define modes (label, stable ID, suppression, reason), define metadata fields (key, label, type, options, required, display), duplicate-key validation, and a reviewer-facing preview.

QM v2, when revived, absorbs the same configuration into its properties panel as AQVersion content; changing a stable key or deleting a used mode is a breaking version change there.

Delivery map

Focused PRs, each citing this document. Order matters only where stated.

# Delivers Vehicle / notes
1 This PR: design context + architecture docs only
2 answerLabel persistence round-trip: domain + DTO + NSwag + render fallback schema-v0; deploy and verify before step 2b
2b answerLabel editor prefill + import template + seed fixtures — the first thing that writes the field Must not ship until every project writer from step 2 is deployed and old instances have drained. Project writes replace the whole document, so one un-upgraded writer erases a saved label; ADR-011 drains writers before enabling writes for exactly this reason. Rolling or independently-ordered API/PM/web deploys make "same PR" insufficient.
3 Branch chip in the number gutter (v1) already queued
4 Branch role="group" subtree containment (v1) already queued; template refactor
5 AF2 outline/navigation panel AF2; consumes answerLabel; store-integrated navigation
6 ADR + domain/persistence for response modes + metadata (writers first) schema-v0 additive; flagged; ADR records the rollback floor
6a Stop tree-shaking suppressed descendants in ExtractionInfo.AddAnnotations, with a suppress-after-answer regression test blocks 7 — without it the reviewer UI silently deletes the answers step 7 claims to preserve
7 AF2 reviewer UI for response modes (chooser, suppression, validation) flag-gated; requires 6a
8 AF2 metadata rendering (inline + details) flag-gated
9 Designer "Response handling" section after 6; gated on actual AF2 eligibility for the stage, not on the environment flag alone
10 Export/API columns for modes + metadata (q__value, q__response_mode, q__<key>) with definition manifest after 6; must resolve suppression so preserved-inactive answers are not emitted as live values, with a suppress-after-answer export test

Activation order and validation rules

The delivery map says ordering matters only where stated. Review established that several orderings are load-bearing, because the intermediate states lose or corrupt data rather than merely degrading. These bind the implementing PRs.

Nothing may write a mode or metadata response until the export path understands them (step 10). WideDataFormatRowWriter.AddAnswerQuadFromGroup calls GetAnswer() and notes only, so in the window between step 7 and step 10 a mode response exports as an ordinary blank and its metadata vanishes entirely — an export that is silently wrong, which is worse for a systematic review than one that fails. The feature flag may not be enabled in an environment until step 10 is deployed there.

The mode chooser may not ship before the metadata renderer. Selecting a mode with requiresReason immediately requires the injected _reason field (Decision 5). Step 7 without step 8 offers reviewers a choice they cannot complete, or submits an invalid response.

Reconciliation must be able to express what annotation can express. The reconciler's own editable form is still v1, which by Decision 2 gets no response-mode controls, and a reconciler must create a new authoritative annotation rather than pick a candidate. Enabling modes on a stage that requires reconciliation before the AF2 reconcile host is editable leaves the reconciler unable to submit at all. Gate on the reconcile host, not just the stage-review host.

Once a mode or metadata response exists, the stage may not fall back to v1. A metadata-only response — no mode selected, but qualifiers recorded — carries exactly the same exposure, because v1 cannot rehydrate metadata either. Disabling the flag, or any eligibility change that makes AF2 fail closed, routes the stage to a renderer that cannot rehydrate responseModeId or metadata — and the next v1 submission reaches the replacement-based ExtractionInfo.AddAnnotations and drops them. Step 6a protects suppressed descendants; this is the same hazard for the mode-bearing annotation itself. The flag is therefore one-way per stage once responses exist, and eligibility loss must fail the stage closed rather than silently downgrade it.

Authoring gates on every stage the question is assigned to. A question can be assigned to more than one stage — ExtractionInfoChildrenMergeTests covers exactly that — and the definition is project-level. Checking only the stage the administrator happens to be editing from lets modes be configured from an AF2-eligible stage and silently break reviewers on a v1 or extraction stage that shares the question. The check is not only at authoring time: assigning an already-mode-configured question to a new stage must revalidate, or the same breakage arrives through the assignment path instead of the editor.

Metadata keys are unique across scopes. AnnotationResponse has one flat metadata dictionary and exports one q__<key> column per key, so a question-level and a mode-scoped field sharing a key cannot be validated, persisted, or manifested unambiguously. Reject the collision at definition-save.

answerLabel records whether it was generated or authored. The prefill tracks the derived heuristic until someone edits it, after which a question-text change must not overwrite their wording. The stored string alone cannot distinguish the two, so provenance has to be persisted alongside it — and being itself an additive field, it falls under the writer-compatibility floor, so it ships in step 2's round-trip rather than being added later once labels exist and their provenance is already unrecoverable.

_reason is mode-scoped, not question-scoped. More than one mode on a question may set requiresReason, and each needs its own required instance. One reserved key with a per-mode requirement, not one shared field that a second mode silently inherits or overwrites. Because the response carries a single flat metadata dictionary, changing the selected mode clears _reason: otherwise switching from mode A to mode B leaves A's explanation in place and B's validation sees its required key already populated, accepting a reason written about a different answer.

A submitted responseModeId must exist in the definition the response is validated against. The contract as stated requires only that it be mutually exclusive with value, so a stale client or a direct API call could submit an unknown ID — which has no label for the export manifest, no reason or metadata requirements, and no suppression rules, producing a response nothing can interpret and which bypasses every configured check. Reject unknown IDs at write time rather than discovering them at export.

Future direction (deliberately unspecified): reference responses and customisable relationships

Recorded 2026-08-11 so nearer-term decisions do not foreclose it. No user requirements exist yet; this section is a thinking aid for when a specification arrives, not a commitment.

Today's mechanism is hard-coded. Cross-annotation relationships — how a Cohort is associated with a Disease Model, Treatment, and Outcome Assessment — are implemented as lookup system questions: an annotationLookup flag on the question entity, a fixed GUID enum (systemAnnotationQuestionGuids.treatmentControl / modelControl / cohortDiseaseModels …, annotation-question.entity.ts:293), and special-cased logic keyed on those GUIDs (annotation-form.service.ts, annotation-question.component.ts:370). AF2 carries the same annotationLookup flag through its models. The relationship specification is code, not configuration.

The idea. Generalise this into a configurable reference response: an answer whose value is a reference to answer instances of another question or category — a custom lookup. A project could then define its own relationship edges (which entity-like repeatable questions exist, and which questions reference them, with what cardinality), so the currently hard-coded Experiment → Cohort → {Disease Model, Treatment, Outcome Assessment} graph becomes one instance of a project-definable specification rather than the only possible shape.

Placement note (the one analytical claim worth recording now). Although this arose from response-mode thinking, a reference is not a response mode: modes are alternatives to answering ("cannot answer"), whereas a reference is an ordinary answer whose value happens to be a reference. The two should stay orthogonal — a reference question can still offer "Not applicable". The natural extension point is the answer-type system (a reference answer kind with definition-side configuration: target question/category, scope, cardinality), i.e. a fifth optional extension on the same spine, not a widening of responseModes. Widening modes into "any non-plain response" would repeat the second-form-engine mistake this document forbids for metadata.

What an eventual specification must answer (so requirements-gathering asks the right questions):

  • Reference scope: same study, same session, same stage?
  • Cardinality and ordering; are dangling references possible when the target repeatable instance is deleted, and what does deletion of a referenced branch mean (block, cascade, or mark-dangling)?
  • Versioning: how references survive AQVersion/SessionVersion transitions.
  • Interaction with suppression (referenced target suppressed by an ancestor).
  • Export and meta-analysis semantics — OutcomeData and PRISMA derivations are consumers of today's hard-coded graph, so any generalisation must keep their inputs derivable.
  • Reconciliation: comparing reference answers across annotators.
  • AF2's mutable-unit machinery (unit categories, one label question per category) is where relationship structure lives at render time; a configurable graph must map onto or replace it deliberately.

Preserved now, at zero cost: AnnotationResponse.value remains open to additive value shapes, and nothing in the delivery map assumes the system graph is fixed forever.

Open questions

Blocking step 6 — must be decided before domain/persistence

Definition-edit policy: freeze on first use, or snapshot per response?

Decision 7 stamps every response with the definitionVersion it was written against, which records that semantics may have changed. It does not decide what the platform then does about it, and that half is open:

  • Freeze on first use — once a key or mode has been answered, its key, type, options and suppression behaviour become immutable; only cosmetic properties (label, help text, display, order) stay editable. Cheap, and aligned with the existing "stable machine keys, project-defined labels" constraint, which already assumes label edits are safe because keys carry meaning. But it is a real restriction on administrators — "you cannot change this field's type now that anyone has answered it" — and that is a product call, not an engineering one.
  • Snapshot per response — store the definition alongside each response. No restriction on administrators, but it duplicates a definition onto every annotation, which is heavy at realistic form sizes and is precisely the job QM v2's immutable AQVersion exists to do properly.

Held pending a user specification. The definitionVersion stamp is what keeps both routes open — and a later QM v2 revival can reconstruct from it — so nothing in step 6 is blocked on choosing today, provided the stamp ships with the response shape.

Carried (not blocking the map)

  • Initial metadata type set (date? specialised unit fields?).
  • Schema reuse across category/stage vs per-question only.
  • Rename/removal semantics for keys and modes already used in responses (narrower than Q3, which is about already-recorded responses).
  • Localisation of labels vs stable keys in exports.
  • Which parts of suppression explanation belong in reconciliation views.

References