Docs adr python

DEVELOPMENT.md

@@ -0,0 +1,82 @@
+# Development Guide
+
+This repository contains the `agent-protocol-standard` schema definitions, the Python-based validator, and the GitHub Actions used by content repositories.
+
+## Technical Stack
+- **Language**: Python 3.10+
+- **Validation**: [Pydantic](https://docs.pydantic.dev/) for declarative schema definitions.
+- **Testing**: [Pytest](https://docs.pytest.org/) for model, fixture, and CLI validation.
+
+## Local Setup
+To run tests locally, install Python and the required dependencies:
+```bash
+pip install pytest "pydantic>=2.0.0" pyyaml
+```
+
+## Running Tests
+Run the entire test suite from the root of the repository:
+```bash
+pytest tests/
+```
+
+## Modifying the Schema
+
+The protocol frontmatter schema is defined in `scripts/models.py` using Pydantic. Document-level checks (like Materials/Steps and History & Reviews) are implemented in `scripts/validate_protocol.py`. 
+
+### Adding a New Field
+To add a new field to the protocol frontmatter, add it as a class attribute to the `ProtocolFrontmatter` class in `scripts/models.py`. Pydantic handles type coercion and basic validation automatically.
+
+```python
+class ProtocolFrontmatter(BaseModel):
+    # ... existing fields ...
+    funding_source: Optional[str] = None  # Example of a new optional field
+```
+
+### Adding a Complex Validation Rule
+If a field requires complex validation (e.g., cross-field dependencies, custom formatting, or complex error messages), use a Pydantic `@model_validator` or `@field_validator`.
+
+For example, to enforce that composite protocols declare dependencies:
+```python
+from pydantic import BaseModel, model_validator
+from typing import Literal
+
+class ProtocolFrontmatter(BaseModel):
+    type: Literal['atomic', 'composite']
+    protocols_used: list = []
+
+    @model_validator(mode='after')
+    def check_composite_dependencies(self) -> 'ProtocolFrontmatter':
+        if self.type == 'composite' and not self.protocols_used:
+            raise ValueError("Composite protocols must declare 'protocols_used'")
+        return self
+```
+
+## Testing Validation Changes
+
+Because we use Pydantic, testing frontmatter model rules does not require writing markdown files to disk or dealing with fragile string manipulation. You test those model rules in-memory by passing dictionaries to the models.
+End-to-end validator behavior is still tested with Markdown fixtures and CLI/filesystem execution paths in `tests/test_validate_protocol.py`.
+
+Add your tests to `tests/test_models.py`:
+
+```python
+import pytest
+from pydantic import ValidationError
+from scripts.models import ProtocolFrontmatter
+
+def test_composite_requires_dependencies():
+    # 1. Start with a valid baseline dictionary
+    bad_dict = {**valid_dict()}
+
+    # 2. Mutate it to trigger the failure state
+    bad_dict["type"] = "composite"
+    bad_dict["protocols_used"] = []
+
+    # 3. Assert that Pydantic rejects it with the expected error message
+    with pytest.raises(ValidationError, match="'type: composite' requires a non-empty 'protocols_used'"):
+        ProtocolFrontmatter(**bad_dict)
+```
+
+## Updating the Template and Fixtures
+If your schema change adds a new required field, ensure you also update `template/protocols/example-protocol/protocol.md` so that future protocols scaffolded from the template do not immediately fail validation.
+
+Additionally, update the valid fixtures in `tests/fixtures/valid/` and any affected content-repository protocols, otherwise CI will fail when validating existing protocols against the new schema.

docs/adr/0015-migrate-validator-to-python.md

@@ -0,0 +1,29 @@
+# 0015. Migrate Protocol Validator to Python and Pydantic
+
+- **Status:** Accepted
+- **Date:** 2026-09-13
+- **Deciders:** Levi Waldron (User), AI Agent
+
+## Context
+The `agent-protocol-standard` repository defines the schema for AI agent protocols. Initially, the validator and its test suite were written in R. This aligned perfectly with the lab's primary domain expertise in R and the Bioconductor ecosystem, ensuring that existing contributors could easily read and maintain the repository's infrastructure.
+
+However, R lacks a mainstream, declarative data-validation library equivalent to Python's Pydantic. Because of this, the R implementation suffered from two critical flaws:
+1. **Procedural Complexity:** The validation logic grew into a monolithic ~800-line script consisting of manual `if` statements, loops, and custom error formatting for every individual field and edge case.
+2. **Fragile Testing:** To test the CLI script, the test suite relied on error-prone string manipulation (custom regex to find and remove markdown headers or YAML blocks) to generate malformed protocols. A simple addition to the schema often broke the regex logic of dozens of unrelated tests, creating a massive maintenance burden.
+
+## Decision
+We will transition the `agent-protocol-standard` validator and its test suite from R to Python. We will use **Pydantic** to define the schema declaratively, and **Pytest** to run the test suite.
+
+## Alternatives Considered
+- **Keep the R implementation:** This would require continuing to maintain a custom validation script and fragile string manipulation in tests. Rejected because the maintenance burden and risk of regressions was too high for a standard that requires strict schema enforcement.
+
+## Consequences
+
+### Positive
+*   **Declarative Simplicity:** Complex validation logic (e.g., cross-field dependencies, type checking) is now handled natively by Pydantic. The core logic was reduced from ~800 lines of procedural R to ~160 lines of declarative Python.
+*   **Robust Testing:** Frontmatter model tests are now performed in-memory on Python dictionaries (e.g., `ProtocolFrontmatter(**bad_dict)`), reducing the need for fragile string manipulation for schema-rule validation while preserving end-to-end Markdown fixture and CLI validation.
+*   **Speed:** The test suite executes in a fraction of the time.
+
+### Negative / Trade-offs
+*   **Ecosystem Divergence:** The repository's tooling stack now diverges from the lab's standard R/Bioconductor ecosystem. 
+*   **Maintenance Barrier:** Contributors who are fluent in R but unfamiliar with Python will face a higher barrier to entry when attempting to modify the validator's logic or GitHub Actions CI pipelines. To mitigate this, clear developer documentation on modifying Pydantic models will be maintained.

docs/adr/README.md

@@ -18,5 +18,6 @@ This directory contains Architecture Decision Records (ADRs) for the `agent-prot
 - [0012. Require Materials, Steps, and a Resolvable Citation](0012-require-materials-steps-and-a-resolvable-citation.md)
 - [0013. `stable` Is an Assertion by the Authors](0013-stable-is-an-assertion-by-the-authors.md)
 - [0014. Ask Two Citation Questions, and Require the Answerable One](0014-two-citation-questions.md) — amends 0009 and 0012
+- [0015. Migrate Protocol Validator to Python and Pydantic](0015-migrate-validator-to-python.md)