Skip to content

Schema drift detection and pruning (detect-schema-drift, prune-schema) #88

Open

Description

@SenolIsci

Schema drift detection and pruning

Tracking issue for the stacked PRs that add schema drift detection and pruning. A declared concept or property with zero instances in the graph is "unused"; after documents are deleted and synced the schema never shrinks.

Commands

  • mykg detect-schema-drift --session S: read-only report, recorded to intermediate/schema_drift.json.
  • mykg prune-schema --session S [--preview | --plan FILE]: preview writes an editable plan file; apply backs up schema.json, prunes, records a schema_prune history entry, regenerates schema.ttl and flattened_schema.json, then refreshes outputs via validate_graph. Nodes and edges are never edited. No LLM calls, no new config keys, not a pipeline step.

Stacked PRs (merge in this order)

Review outcome

All Sourcery review comments were triaged and the fixes pushed to each PR. Declined: cross-process locking, moving the MCP scan to a thread, regenerating knowledge_graph.ttl in core. Refuted: the claim that grow-schema sessions lock every schema entry.

Known gaps

  • schema.pre_prune.json is overwritten on each prune (single backup).
  • Writes are ordered but not one transaction; re-running a prune is idempotent and approve-schema repairs schema.ttl.
  • Lock-file loading is duplicated between cli.py and walkthrough.py.
  • Suggested commands use POSIX shell quoting only.
  • The tier-4 "grace-period decay" is not implemented.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions