Skip to content

docs: document optional Pass 2 domain/range check (pass2.domain_range_policy) - #81

Closed
SenolIsci wants to merge 1 commit into
mainfrom
docs/pass2-domain-range-policy
Closed

SenolIsci wants to merge 1 commit into
mainfrom
docs/pass2-domain-range-policy

Conversation

@SenolIsci

@SenolIsci SenolIsci commented Oct 4, 2026 •

Copy link
Copy Markdown
Owner

Documents the new pass2.domain_range_policy key (off | warn | strict, default warn) in docs/architecture.md alongside Pass 2 validation. Docs-only; companion to the implementation PR. (CLAUDE.md D60 is gitignored, so it was updated locally only.)

🤖 Generated with Claude Code

Summary by Sourcery

Documentation:

  • Document the optional Pass 2 domain/range validation policy, including its off, warn, and strict modes, configuration behavior, and validation outcomes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

Adds architecture documentation for the new pass2.domain_range_policy setting, covering configuration, inheritance-aware validation semantics, warning versus strict enforcement, retry/recovery behavior, and the rationale for the default.

Flow diagram for Pass 2 domain/range policy enforcement

flowchart TD
    A[_extract_chunk] --> B[_normalize_scalars]
    B --> C[_apply_domain_range_policy]
    C --> D{domain_range_policy}
    D -->|off| E[Continue existing validation]
    D -->|warn| F[Keep remaining violations and tag domain_range_violation]
    D -->|strict| G[validate_extraction]
    G -->|Violations remain| H[Single LLM retry]
    H --> I[_partial_recover drops violating edges]
    I --> J[Clean up hallucinated anchor nodes]
    G -->|No violations| K[Continue extraction]
    F --> L[Assemble output]
    E --> L
    J --> L
    K --> L
Loading

File-Level Changes

Change Details Files
Document the configurable Pass 2 domain/range validation policy and its runtime behavior.
  • Describe off, warn (default), and strict modes, including inheritance-aware endpoint matching.
  • Document configuration lookup, profile-level opt-in, invalid-value fallback, and the intentional omission from mykg_config.yaml.
  • Explain validation, retry, partial-recovery, metadata tagging, and edge cleanup behavior.
  • Record rationale for the default and clarify that orphan_inferred edges are excluded.
docs/architecture.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="docs/architecture.md" line_range="344" />
<code_context>
+**Optional domain/range check (`pass2.domain_range_policy`).** Historically Pass 2 validated only edge type, `from`/`to` node IDs and confidence; a property's `domain`/`range` were prompt hints. The `pass2.domain_range_policy` key (`off` | `warn` | `strict`, default `warn`, exposed as `config.PASS2_DOMAIN_RANGE_POLICY`) adds an inheritance-aware check: endpoint types come from the response's nodes plus `prior_nodes`, and a subclass (walking each concept's `parent` chain) satisfies a domain or range. Edges whose property or endpoint type is unknown are left to the existing checks. The key is read but deliberately not surfaced in either `mykg_config.yaml` (same precedent as `llm.temperature_unsupported_prefixes`); opt in by adding it under a profile's `pass2:` block. An invalid value logs a warning and falls back to `warn`.
</code_context>
<issue_to_address>
**issue:** The documentation claims that `pass2.domain_range_policy` is read and exposed as `config.PASS2_DOMAIN_RANGE_POLICY`, but the repository has no such configuration constant or policy implementation, so adding the documented key currently has no effect.

**Triggers:** When this documentation is merged without the companion implementation being available in the same released codebase.

**Suggested fix:** Merge the implementation that reads and applies the setting before documenting it, or describe the key as unreleased and point to the required implementation version.
</issue_to_address>

### Comment 2
<location path="docs/architecture.md" line_range="349" />
<code_context>
+`_apply_domain_range_policy` runs in `_extract_chunk` after `_normalize_scalars`, on both the first attempt and the retry:
+
+- **`off`** — no-op.
+- **`warn`** (default) — an edge whose endpoint types match `(range, domain)` exactly is swapped (logged). Remaining violators are kept and tagged `domain_range_violation: true`, which reaches `edge_metadata.json` / `edges.jsonl` via the assembler's deepcopy of the first occurrence.
+- **`strict`** — same swap, then `validate_extraction` reports remaining violations as errors, triggering the existing single LLM retry with those errors as feedback. If the retry still violates, `_partial_recover` drops the violating edges *before* its hallucinated-anchor pass, so their anchor nodes are cleaned up too. Both pieces are required: `validate_extraction` only reports; `_partial_recover` is the only place that enforces.
+
</code_context>
<issue_to_address>
**issue (broader_impact):** A domain/range violation flag from a later occurrence of a duplicate edge is discarded when the assembler's first occurrence is clean, because edge deduplication deep-copies only the first occurrence and does not merge this top-level flag; the violation therefore does not reliably reach `edge_metadata.json` or `edges.jsonl`.

**Triggers:** When the same `(type, from, to)` edge is extracted in multiple files or chunks and the first occurrence does not violate domain/range while a later occurrence does.

**Suggested fix:** Merge `domain_range_violation` across all edge occurrences with an OR operation, or store it in a field that the deduplication merge logic explicitly preserves.
</issue_to_address>

Sourcery assessment

Approval pending. 2 findings to address first.

Blocking findings: docs/architecture.md:344, docs/architecture.md:349


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment thread docs/architecture.md

After each LLM call, the pipeline validates the response: edges whose type is not in the schema are rejected, as are edges that reference node IDs not present in the same extraction. Any node that exists solely to anchor a rejected edge is also dropped. Missing attributes are backfilled with a null value and zero confidence — they are never silently omitted.

**Optional domain/range check (`pass2.domain_range_policy`).** Historically Pass 2 validated only edge type, `from`/`to` node IDs and confidence; a property's `domain`/`range` were prompt hints. The `pass2.domain_range_policy` key (`off` | `warn` | `strict`, default `warn`, exposed as `config.PASS2_DOMAIN_RANGE_POLICY`) adds an inheritance-aware check: endpoint types come from the response's nodes plus `prior_nodes`, and a subclass (walking each concept's `parent` chain) satisfies a domain or range. Edges whose property or endpoint type is unknown are left to the existing checks. The key is read but deliberately not surfaced in either `mykg_config.yaml` (same precedent as `llm.temperature_unsupported_prefixes`); opt in by adding it under a profile's `pass2:` block. An invalid value logs a warning and falls back to `warn`.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: The documentation claims that pass2.domain_range_policy is read and exposed as config.PASS2_DOMAIN_RANGE_POLICY, but the repository has no such configuration constant or policy implementation, so adding the documented key currently has no effect.

Triggers: When this documentation is merged without the companion implementation being available in the same released codebase.

Suggested fix: Merge the implementation that reads and applies the setting before documenting it, or describe the key as unreleased and point to the required implementation version.

Comment thread docs/architecture.md
`_apply_domain_range_policy` runs in `_extract_chunk` after `_normalize_scalars`, on both the first attempt and the retry:

- **`off`** — no-op.
- **`warn`** (default) — an edge whose endpoint types match `(range, domain)` exactly is swapped (logged). Remaining violators are kept and tagged `domain_range_violation: true`, which reaches `edge_metadata.json` / `edges.jsonl` via the assembler's deepcopy of the first occurrence.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (broader_impact): A domain/range violation flag from a later occurrence of a duplicate edge is discarded when the assembler's first occurrence is clean, because edge deduplication deep-copies only the first occurrence and does not merge this top-level flag; the violation therefore does not reliably reach edge_metadata.json or edges.jsonl.

Triggers: When the same (type, from, to) edge is extracted in multiple files or chunks and the first occurrence does not violate domain/range while a later occurrence does.

Suggested fix: Merge domain_range_violation across all edge occurrences with an OR operation, or store it in a field that the deduplication merge logic explicitly preserves.

@codecov

codecov Bot commented Oct 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@SenolIsci

Copy link
Copy Markdown
Owner Author

Abandoned: domain/range policy feature dropped.

@SenolIsci SenolIsci closed this Oct 4, 2026
@SenolIsci
SenolIsci deleted the docs/pass2-domain-range-policy branch October 4, 2026 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant