Repository navigation
Conversation
Routing contract: every boards/<id>.json declares build.chip_family and ChipFamily.resolve() maps it to a family instance — no board-id substring matching anywhere. Registration happens at class definition (__init_subclass__); a new family is one module plus one import line in families/__init__.py. Esp32Family owns the ESP32 tool-expansion logic moved from platform.py (prefer-local tools, idf_tools.py expansion, esptoolpy runtime deps). Other families are thin delegates over platform_cfg/*_cfg for now. The package imports without PlatformIO installed so CI verifiers can read the registry. Co-Authored-By: Claude <noreply@anthropic.com>
Every manifest gets build.chip_family (esp32/nrf/samd/rpi/renesas/ siliconlab/stm32) inserted at its alphabetical position in the build section. New verifier rule R7 fails a board whose value is missing or matches no registered family, taking valid values from the families/ registry (single source of truth). Co-Authored-By: Claude <noreply@anthropic.com>
configure_default_packages, _add_dynamic_options and configure_debug_ session all resolve the family from the board manifest's build.chip_family and delegate to it. The global Architecture variable, both board-id substring blocks, and the ESP32 install-time methods (moved to families/espressif.py) are gone; platform.py keeps only zephyr per-board package routing and board listing. Behavior is unchanged: same package flips, same debug tools, same debug-session tweaks, same ESP tool expansion sequencing via the family's configure/post_configure hooks. Co-Authored-By: Claude <noreply@anthropic.com>
builder/main.py and the Arduino framework wrapper resolve the family from the board manifest and SConscript whatever the family returns — a local board_build/<family>/ script today, an upstream-platform script later — then apply the family's build hooks. The Zephyr builder replaces its two board-name substring PIOPLATFORM overrides/restores with family.zephyr_pioplatform(). The repo root is tail-appended to sys.path at SCons time (never shadowing stdlib platform). Co-Authored-By: Claude <noreply@anthropic.com>
The espressif32 upstream platform (pioarduino/platform-espressif32, locked at 53.03.13 = d2f1607c) is now declared as an optional package in platform.json and activated by Esp32Family.required_packages when Arduino is requested. The Arduino framework wrapper SConscripts the upstream package's builder/frameworks/arduino.py directly; the local esp_arduino.py copy (a reformatted 53.03.13 snapshot with Tasmota pip deps and an embedded-libs SDK dir tweak) is deleted. Known deltas vs the deleted copy: python deps return to the upstream set (PyYAML/intelhex instead of zopfli/tasmota-metrics) and the Windows include-shortening middleware resolves against the separate framework-arduinoespressif32-libs package layout, degrading to unshortened includes on this platform's embedded-libs layout. Co-Authored-By: Claude <noreply@anthropic.com>
The tool-package channel requires a package.json manifest, which pioarduino does not ship (it only has platform.json), so activating the espressif32 entry through the regular optional=False path fails with MissingPackageManifestError. PlatformPackageManager accepts platform.json manifests and installs into the platforms dir instead: Esp32Family now installs the locked upstream there (dependencies skipped — this platform owns its packages) during package configuration, and framework_script resolves the upstream builder from the platforms dir at SCons time. The espressif32 entry stays optional in platform.json as the locked-version declaration. Co-Authored-By: Claude <noreply@anthropic.com>
Registry completeness and singleton behavior, resolve() three states (missing field names the board file, unknown value lists valid ids), every boards/*.json routing by manifest field (not id substring — the stm32c5 board would misroute under substring routing), in-process platform delegation for configure_default_packages and _add_dynamic_options, and family-owned build script mapping. Co-Authored-By: Claude <noreply@anthropic.com>
The espressif32 entry in platform.json's packages table was never a core-installable package: the tool-package channel requires a package.json manifest pioarduino does not ship, and the family installs the locked upstream through PlatformPackageManager instead. Mixing a non-installable source reference into the packages table (where every other entry means 'core installs this') was misleading. The locked spec now lives on Esp32Family.upstream_version next to its consumer — source reference, installer, and script path resolution in one place. Bumping it is the maintainer-driven upstream upgrade point. Co-Authored-By: Claude <noreply@anthropic.com>
….03.312 The upstream lock moves from 53.03.13 (the Arduino snapshot origin) to 55.03.312, whose espidf.py is the direct origin of the deleted local 2817-line snapshot (real local delta was 24+/8-; the rest was snapshot age). Reinstall detection now compares the installed checkout's git HEAD against the locked commit (the manifest version is only the upstream tag string), and uninstall passes skip_dependencies so pioarduino's uninstall cannot remove this platform's own packages. builder/frameworks/espidf.py becomes a thin wrapper like the Arduino one; the local snapshot and its now-orphaned board_build/esp helpers (ulp.py, _embed_files.py) are removed. pioarduino's espidf.py loads component_manager from the HOST platform dir by env.PioPlatform().get_dir(), so builder/frameworks/component_manager.py is now a forwarding shim to the locked upstream copy — no code duplicated. ESP-IDF builds also activate tool-cmake/tool-ninja, which no local code ever did (there was no espidf consumer before). The package matrix stays on the existing registry versions (framework-espidf ~3.50503.0, toolchains 14.2.0+20251107): the 55.03.312 matrix (IDF 5.5.5 + shell zips + build-time tl-install bootstrap) conflicts with this platform's esp_install expansion channel. Validated on Windows PowerShell: espidf c3 smoke build (full IDF cmake/ ninja/ldgen chain) and all five esp32 Arduino boards; the earlier ldgen parse failures were an MSYS-environment artifact, not a version mismatch. Not yet validated on real hardware. Co-Authored-By: Claude <noreply@anthropic.com>
The local copy of the stub-expansion logic (idf_tools.py invocation, core/tools staging, file:// link-back) is replaced by a call into the locked upstream platform's own API: PlatformFactory.new() on the installed espressif32 package, then Espressif32Platform.install_tool() for each required tool whose installed package still carries a tools.json (the stub marker). Class-level reuse per the handbook — upstream upgrades now carry their own expansion fixes. This also fixes the stub-shadowing gap the local copy had: upstream install_tool removes the stub before installing the expanded payload back into the packages dir, so tool-esptoolpy now ends up as the real payload in packages (esptool.py present, tools.json gone) instead of a stub there with the payload stranded in core/tools. The manifest entry is then linked to the local directory so the core never re-downloads the stub. Verified on Windows PowerShell: fresh expansion of tool-esptoolpy from a clean packages dir, UPLOADER now resolves inside packages/ (no core/tools fallback hit), idempotent rebuild, all five esp32 Arduino boards pass. Not validated on hardware upload. Co-Authored-By: Claude <noreply@anthropic.com>
Esp32Family.build_script now points at the upstream pioarduino builder/main.py (55.03.312): env assembly, upload matrix, filesystem images, and diagnostics all come from the locked upstream package. The local board_build/esp/esp_build.py — an official-v6.9.0-era copy of the same file, plus its orphaned _bare.py — is deleted, together with its two companion orphans builder/penv_setup.py and builder/sizedata.py (consumed only by the retired copy; the live penv dependency declaration is read from the upstream package — see _upstream_python_deps). The ESP32 builder layer is now a single upstream anchor (312) across arduino, espidf and board-level scripts. Five host-side adaptations make the upstream script runnable here, all additive on our side, none touching the upstream package: 1. SeeedstudioPlatform.setup_python_env forwards to the family adapter (upstream main.py calls platform.setup_python_env on its host). 2. Esp32Family.setup_python_env returns the working (penv python, esptool) pair WITHOUT running pioarduino's penv machinery — its uv sync owns the shared penv and removed PIO core packages (certifi, uvicorn) when tried. esptool resolves to the penv esptool.exe: the upstream feeds the value straight into OBJCOPY, and a bare .py path cannot be spawned on Windows. 3. seed_penv_deps idempotently pre-installs the module-level imports the upstream builder expects (littlefs-python, fatfs-ng). 4. builder/spiffsgen.py is a forwarding shim: upstream loads spiffsgen from the HOST platform dir (same host-path expectation as component_manager / penv_setup). 5. toolchain stubs expand through the upstream installer API (previous commit). Validated on Windows PowerShell: all five esp32 Arduino boards, the espidf c3 smoke build (full cmake/ninja/ldgen chain), UPLOADER resolves to a spawnable esptool.exe, pytest 15/15, board verifier clean. Hardware upload not yet validated. Co-Authored-By: Claude <noreply@anthropic.com>
The upstream pioarduino builders locate their companion modules (spiffsgen.py, penv_setup.py, component_manager.py, boards/<id>.py) through env.PioPlatform().get_dir() — the HOST platform directory. Two static forwarding shims existed to satisfy that expectation (builder/spiffsgen.py, builder/frameworks/component_manager.py). This replaces them with a runtime proxy handed to the upstream scripts for the duration of their SConscript call: get_dir() resolves to the upstream package so every such lookup lands inside the locked upstream copy, while everything else (packages, install_tool, debug) forwards to the real host platform. Any future 'load from host dir' expectation is covered automatically — no new shim files per upstream addition. The three dispatchers (builder/main.py, the arduino and espidf wrappers) call the family's wrap_platform hook before SConscripting an upstream script and restore the original afterwards. Validated on Windows PowerShell: all five esp32 Arduino boards, the espidf c3 smoke build, pytest 15/15, board verifier clean. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
required_packages() only activated the espressif32 upstream platform when the board declared the arduino framework, but framework_script() demands it for espidf builds as well. Existing boards all declare both frameworks so this never surfaced; an espidf-first board (the planned S31) would have failed loudly on its first build. Activate on either framework. Found in review. Co-Authored-By: Claude <noreply@anthropic.com>
…CI cache key Three review findings on the upstream-platform dependency: 1. _ensure_upstream_platform is only invoked when a build requires an ESP32 framework script, so a failed install is now fatal at package configuration (message carries the locked spec and the pre-seed remedy) instead of degrading to a missing-builder error deep inside SCons. 2. Lock detection now writes and prefers a .upstream-lock marker file over the .git checkout: copied/deployed platforms dirs (offline distribution, CI cache extraction) have no .git and were misidentified as not installed, forcing a network reinstall on offline machines. rev-parse remains the fallback for pre-marker installs. 3. build-arduino-examples.yml keyed its whole ~/.platformio cache on platform source files (platform.py, builder/**...), so every PR push invalidated the platform/package cache and forced a fresh upstream git clone per run. Platforms+packages now cache under a key derived from the package matrix and the upstream lock only; core tools keep the old key. Note: PlatformIO only runs configure_default_packages when it must make install decisions, so on an already-satisfied machine the marker is written on the first real decision point (fresh machine / CI / cache restore), not on every build — verified by direct configure invocation. Co-Authored-By: Claude <noreply@anthropic.com>
CI (bare runner python: platformio + rich-click + intelhex only) failed with ModuleNotFoundError: No module named 'yaml' — upstream arduino.py imports component_manager, which imports yaml at module level. The local penv had PyYAML incidentally (installed by the zephyr builder's self-healing pip), masking the gap. Full closure audit of the upstream builder's third-party imports: main.py -> littlefs/fatfs (already seeded), arduino.py -> component_manager -> yaml (this fix); click and semantic_version are PlatformIO core dependencies present wherever core runs; spiffsgen and arduino_relinker are stdlib-only. The seed list is now complete. Co-Authored-By: Claude <noreply@anthropic.com>
The seeded penv package list no longer pins its own version constraints: seed_penv_deps now loads the upstream builder's penv_setup.py (declarations only, no top-level side effects) and reads its python_deps dict, so version floors travel with the locked upstream. Two explicit maps keep us in control: PENV_SEED (what we install, pip name -> import name) and PENV_SEED_EXEMPT (audited deps deliberately not seeded: managed separately or transitively present). A dependency that upstream adds to python_deps but appears in neither map prints a build-time warning — new upstream requirements surface for an explicit decision instead of a CI failure or a silent gap. Co-Authored-By: Claude <noreply@anthropic.com>
_ensure_esptoolpy_runtime_dependencies (rich_click<2 + intelhex, a point-fix predating the seed list) ran right next to seed_penv_deps in the same hook with identical mechanics — a first-generation leftover that the list-based track superseded. Its rich_click<2 pin had no upstream basis (upstream declares rich-click>=1.8.6 unbounded and its ecosystem runs unpinned) and was the only divergence between the two tracks. Both packages now live in PENV_SEED with constraints from the upstream list; the method, its flag, and the call line are gone. Other family classes are unaffected — they carry no penv handling. Co-Authored-By: Claude <noreply@anthropic.com>
The ESP32 adapters depend on implementation details of the locked pioarduino package (companion-file lookups via get_dir(), the platform.setup_python_env host-method call, install_tool, the python_deps declaration). A lock bump can silently invalidate any of them — worst case the proxy keeps 'working' with wrong semantics. tests/test_upstream_anchors.py fetches the anchor files at the LOCKED commit from GitHub raw and asserts each detail still holds; failures name the drifted anchor and point at the adapter list in the Esp32Family class header. Network-dependent tests skip cleanly offline. build-arduino-examples.yml now runs pytest (tests were local-only before), so a lock bump that breaks an anchor turns the PR red. Co-Authored-By: Claude <noreply@anthropic.com>
_upstream_python_deps exec'd the upstream penv_setup.py module on the assumption (stated in its own docstring) that it has no top-level side effects — one more unguarded implementation-detail dependency, and the exact failure mode the adapters exist to prevent: an upstream refactor moving logic to module level would run it on user machines at package configuration. The dependency declaration is now parsed with ast and literal_eval'd — zero upstream code execution. If upstream ever makes the dict non-literal or moves it, seeding degrades with a warning instead of executing anything; no anchor needed because there is nothing left to assume about the file. Review finding. Co-Authored-By: Claude <noreply@anthropic.com>
The unclassified-upstream-dep check lived as a build-time Warning in seed_penv_deps — the wrong layer twice over: nobody audits build logs for warnings, and anchors already fail hard on the same kind of drift from the same data source (the locked commit, fetched over network). Two standards for one concern. The check is now test_penv_deps_fully_classified in the anchor suite: exact-set match of upstream python_deps keys against PENV_SEED ∪ PENV_SEED_EXEMPT (both directions — unclassified upstream additions and stale local entries fail). The AST extractor moved to a module- level pure function (_parse_python_deps) so the family and the test share one implementation. Review finding. Co-Authored-By: Claude <noreply@anthropic.com>
Speculative interface from the original ADR-0001 sketch: no dispatch site in platform.py, builder/main.py, or the framework wrappers calls any of the three, and no family overrides them. ESP32 -- the one family fully on the registry -- needs none of them (flags and protocol come from the board manifest; env tweaks flow through get_build_hooks, which does have real consumers), proving them non-universal. If a future migrated family genuinely needs a hook, add it at that family with its own signature; promote to the base class only when a second family and a shared dispatch site appear. Co-Authored-By: Claude <noreply@anthropic.com>
CI (Linux) failed all five ESP32 boards at the bootloader step with
`sh: 1: esptool: Permission denied`. Three facts compound:
1. PIO core prepends the tool-esptoolpy package ROOT to $PATH for
uploader packages; the pioarduino esptool.zip payload has no bin/,
and its `esptool` is the Python package DIRECTORY — execve on it
is EACCES, which dash reports as "Permission denied".
2. PIO 6.2's where_is_program() returns the BARE program name when
lookup fails, so the previous `where_is_program("esptool") or ""`
guard never fell through and OBJCOPY became the bare name.
3. Windows masked both: a leftover penv esptool.exe (or the .py file
association) satisfied the lookup locally.
setup_python_env now validates the lookup result (regular executable
file only) and otherwise generates a launcher under the core cache
dir that runs tool-esptoolpy/esptool.py with the penv python; the
package's esptool.py itself is unpacked without the exec bit on
Linux, so it cannot be returned directly either. The wrapper embeds
both paths and rewrites itself when they move.
seed_penv_deps probed importability in the PIO host process but
installed into the penv — a system python preseeded on CI (the old
workflow line) masked a penv that lacked the packages for real. The
probe now runs inside the penv interpreter, and the workflow
preseed of rich-click/intelhex is dropped.
Tests cover: bare-name lookup must not leak out (CI regression),
a real $PATH entry point wins without a wrapper, the executable
predicate, and the penv-side import probe.
Co-Authored-By: Claude <noreply@anthropic.com>
Mechanical follow-up to the nordic migration: the ESP32 package matrix, debug-tool injection and flash-image load commands move from platform_cfg/esp_cfg.py into the family class (the per-family product-decision layer lives with its family; the platform_cfg string-hook shell keeps shrinking until the remaining five families migrate). Bodies move verbatim - the pseudo-self functions become real methods - and the debug-tool tests load Esp32Family instead of the standalone module. Co-Authored-By: Claude <noreply@anthropic.com>
The per-board renamed packages (framework-zephyr-nrf54l15 @ 4.2.1, framework-zephyr-nrf54lm20 @ 4.4.0) were both registry framework-zephyr tarballs installed under aliases; nothing consumes the difference, so the aliasing only multiplies package management surface. Unify on the registry identity framework-zephyr @ 3.40400.260428 (4.4.0) - the version 4 of the 5 zephyr boards already build against. seeed-xiao-nrf54l15 jumps 4.2.1 -> 4.4.0: build-verified for this PR (zephyr-blink), hardware verification pending. boards/*.json drop build.zephyr.package; variant/board_name/bootloader stay. Zephyr version bumps are now platform-level decisions only. Co-Authored-By: Claude <noreply@anthropic.com>
The per-board multiplexing had three layers: the build.zephyr.package field in each board manifest, the SCons-side lookups consuming it (nrf_build.py, zephyr.py, stm32_build.py), and the platform-side validation (platform.py''s _configure_zephyr_package_for_board, the zephyr-package half of verifier rule R1, and the READMEs documenting the field). With a single pinned package all three collapse: - builders reference "framework-zephyr" literally - the same form upstream builders use; - _patch_platformio_framework_package_name and the framework_package_name variable are gone: the patch existed solely to translate that literal into the per-board alias inside the framework package''s own scripts, and with the unified name its replacement map is the identity; - the configure-time validation and CI rule have nothing left to check; the READMEs document the single-package reality. Co-Authored-By: Claude <noreply@anthropic.com>
The upstream-platform machinery in Esp32Family (UpstreamPlatformProxy, the git-lock installer with its .upstream-lock marker, wrap_platform) is family-agnostic: it only consumes upstream_platform/upstream_version and the platforms dir. The nrf family is about to become the second caller, which is the point where the bottom-up rule says the capability moves to the base class. Bodies move verbatim (docstrings lose their pioarduino-specific wording); espressif.py keeps only its esp-specific adapters (penv seeding, esptool resolution, stub expansion). Co-Authored-By: Claude <noreply@anthropic.com>
The board-level builder for every nrf board is now the official platform-nordicnrf52 builder/main.py (locked at v11.0.0 through the ChipFamily upstream loader hoisted earlier), executed by families/nordic/build.py with env.PioPlatform() wrapped in an UpstreamPlatformProxy. The old nrf_build.py keeps serving the local upload matrix until upload.py lands in the next commits. The wrapper exposed three proxy-window leaks, fixed here: - framework script paths resolved against the upstream dir (PIO core falls back to get_dir() for relative scripts): platform.py now rewrites every frameworks[*].script against the host repo at configure time; - zephyr.py / arduino.py anchored host paths on get_dir()/__file__: host paths now come from family.host_dir() (new on ChipFamily; SCons deletes __file__ inside SConscript frames); - families/nordic becomes a package (class in __init__.py, the wrapper beside it) to keep family code and its build assets in one place. Verified with clean builds against PR-1 baselines: nrf54lm20a (zephyr) hex, afruit + mbed (arduino) hex and DFU-zip contents byte-identical. Co-Authored-By: Claude <noreply@anthropic.com>
The 346 upload-specific lines of the retired nrf_build.py move to the family directory (mcumgrupload.py comes along). The wrapper in build.py drives them around the upstream skeleton: - BEFORE the upstream script: upload.handles() picks the protocols where our behavior differs (nrfutil-mcumgr / pyocd / probe-rs, OpenOCD-class on nRF MCUs with RRAM/halt sequences, the bossac fork for sam-ba, jlink on DFU-bootloader builds); upload.configure() builds the full uploader command and UPLOAD_PROTOCOL is rewritten to "custom", whose upstream branch is a pure $UPLOADCMD passthrough; - AFTER it: upload.graft_artifacts() attaches the real flash target (imgtool-signed MCUboot image, raw bin, SoftDevice bootloader pair) to the upload/buildprog aliases - the upstream custom default is a plain hex - and repoints UPLOADCMD at it. Verified against PR-1 baselines: nrf54lm20b signed.bin and nrf54lm20a hex byte-identical; the cmsis-dap passthrough runs the full OpenOCD RRAM chain up to the physical probe (no hardware attached); upstream- native protocols (nrfutil/mbed) untouched. Co-Authored-By: Claude <noreply@anthropic.com>
…-owned The retired nrf_arduino.py was the upstream adafruit.py plus two deltas; the board now SConscripts the upstream script and the deltas return as post-script hooks (families/nordic/arduino_hooks.py): BSP library directories join LIBPATH, and the malloc/free-only wrap set is kept by filtering the realloc/calloc wraps the upstream script adds. Verified byte-identical (hex and DFU-zip contents) against the PR-1 baseline. arduino-core-mbed.py moves to families/nordic/ as a family-owned fork instead: it derives from an older upstream revision and carries two real fixes (variant-dir linker-script resolution, FPU flags disabled) whose upstream adoption changes behavior - deliberately deferred, the file header records the reasoning. _bare.py is gone; the upstream skeleton already dispatches its own copy for framework-less builds. Co-Authored-By: Claude <noreply@anthropic.com>
The 157-line product-decision layer (package matrix, default debug tools, debug session tweaks) moves from platform_cfg/nrf_cfg.py into NordicFamily - the per-family decisions now live with their family (the platform_cfg string-hook shell is on its way out; cross-family dedup of the debug table is deferred until the next family migrates). The J-Link package drop goes through the owned drop_package guard. The remaining board_build/nrf assets relocate: the OpenOCD configs (nrf54l.cfg, nrf54lm20a.cfg) to families/nordic/ with the four board manifests' openocd_target repointed, and nrf_build.py itself is gone - the upstream skeleton plus build.py/upload.py own everything it did. Co-Authored-By: Claude <noreply@anthropic.com>
Pin the upstream shapes the nordic family adapters rely on: the "custom" upload branch staying a pure $UPLOADCMD passthrough, the dispatch reading $UPLOAD_PROTOCOL from the env, the unconditional ElfToBin builder (graft chains), the framework-zephyr build-pre load (single-package key), the upstream _bare.py dispatch, the inherited J-Link quoting fix, and adafruit.py resolving its BSP package through get_package_dir. A lock bump that breaks one of these turns CI red before the behavior diff ships. Co-Authored-By: Claude <noreply@anthropic.com>
families/ is the Core-process import surface: one family, one module, platform-side only. The SCons-process assets (build wrapper, upload layer, arduino hooks and mbed fork, mcumgrupload.py, nRF54 OpenOCD configs) move to builder/families/nordic/ — the vertical axis next to builder/frameworks/ — mirroring esp, which has no family dir precisely because its SCons entry lives in the upstream package. Import mechanics: builder/ is not a package; sibling modules come into reach the way builder/frameworks/zephyr.py reaches zephyr_fixes — the family's builder dir on sys.path (SCons deletes __file__ in SConscript frames, so host paths derive from family.host_dir()). Also repoints the four nRF54 board manifests' openocd_target, the anchor-test docstrings, and the factory_reset scripts (whose board_build/nrf/ paths had been dangling since the nrf migration). Layout decision record: handover-nrf-layout-revision.md (D3' replaces D3;责任在决策侧). Co-Authored-By: Claude <noreply@anthropic.com>
D3' finishes the axis move: every family's SCons-side assets now live under builder/families/<family>/ (renesas/rpi/samd/siliconlab/stm32 move now, nordic landed in the previous commit), so board_build/ as a mixed old-world name disappears in one stroke instead of lingering as a half-migrated third location. AGENTS.md and the family classes point at the new path; renesas_cfg's ra4m1.cfg reference follows the move. build_lib/ is a proven orphan: its only consumer was the old espidf.py snapshot deleted with Seeed-Studio#121, and the upstream pioarduino package ships its own copy that the proxy resolves to (CI espidf smoke proves it). Remaining `board_build.*` mentions are PlatformIO project option names (board_build.mcu / .ldscript / .filesystem_size), not paths. Co-Authored-By: Claude <noreply@anthropic.com>
The upload layer's if/elif dispatch (handles + configure + graft_artifacts) becomes a table: PROTOCOLS maps each takeover protocol to its gate, uploader-command configuration and artifact graft, with the openocd entry serving every debug.tools key on nRF MCUs and the jlink entry prefix-matching. resolve() walks the table and returns a Takeover binding the concrete protocol name, so build.py collapses to a straight line: resolve -> configure -> rewrite UPLOAD_PROTOCOL to "custom" -> SConscript the upstream skeleton -> graft. Equivalence checked against the retired chain over a 90-case truth table (protocol x board tools x mcu x DFUBOOTHEX): zero mismatches, including the jlink-before-openocd-class ordering. Configure bodies move verbatim behind a uniform (env, platform, board, protocol) signature; graft bodies verbatim behind (env, board). Co-Authored-By: Claude <noreply@anthropic.com>
framework-zephyr 4.4's bluetooth controller hal/debug.h emits
__asm__ inline volatile(...), which GCC < 9 rejects with
"expected '(' before 'inline'" — every BT-enabled seeed-xiao-nrf54l15
sample failed CI on exactly that. Move the nordic and stm32 zephyr
pins from ~1.80201.0 (GCC 8.2.1) to ~1.120301.0 (GCC 12.3), the
version both upstream platform-nordicnrf52 and platform-ststm32 pin
for Zephyr, and advertise it in platform.json optionalVersions.
Arduino/mbed pins are untouched.
Co-Authored-By: Claude <noreply@anthropic.com>
Zephyr renamed the SSD1306 bindings (solomon,ssd1306fb* -> solomon,ssd1306*) before 4.4; the shared app.overlay still declared the old compatible, which only the retired framework-zephyr-nrf54l15 (4.2.1) package understood. On the unified 4.4.0 package the node stays unbound: oled fails to link (__device_dts_ord_127 undefined, no driver instance) and oled-lvgl fails on the dropped width/height properties. The nrf54lm20a board overlays already used the new name; align app.overlay with them. Co-Authored-By: Claude <noreply@anthropic.com>
families/ now owns the per-framework package pins (the toolchain version lives there), so changes under it must both re-run the example builds and rotate the PlatformIO package cache. Co-Authored-By: Claude <noreply@anthropic.com>
The board-level skeleton (environment assembly, toolchain selection,
target_firm matrix, upload dispatch) is now the locked
maxgerhardt/platform-raspberrypi builder/main.py @ 4d1ad09 (the fork
point; official platformio/platform-raspberrypi is an mbed shell with
no earlephilhower/RP2350/XIAO), executed by the thin wrapper in
builder/families/rpi/build.py.
Deltas vs the retired local snapshot (rpi_build/rpi_arduino/_bare):
- generate_uf2 bare picotool + openocd bare UPLOADER: previously
absolute paths in the snapshot; now healed by PATH-prepending the
earlephilhower tool packages in the wrapper post section
- Import("env")/DefaultEnvironment and the _bare path: regressed to
the upstream forms (nordic-verified)
- msdlabels 1-tuple bugfix: dropped (upstream bug, PR to mg pending)
- 230-line rpi_cfg inlined into families/rpi.py (debug prints and a
bare except dropped as cosmetics); toolchain-gccarmnoneeabi
cross-family protection comment carried verbatim
CI: arduino workflow now triggers on families/** and keys the package
cache on families/*.py (covers every family lock, not just espressif).
Verified: rp2040/rp2350/samd arduino-blink artifacts byte-identical
against the pre-migration baseline; 7 upstream anchors pinned in
tests/test_upstream_anchors_rpi.py.
Co-Authored-By: Claude <noreply@anthropic.com>
Port of d466200 (from the FamilyConfig line, which this tree does not carry) back to the function form: remove the type=uploader pruning loop entirely. samd's own uploaders carry no type field, so the loop collected and deleted esp/renesas/nrf uploaders from the shared packages dict, breaking later in-process configuration of those families; the deleted packages are optional and never install for samd builds anyway. Also drops two stray debug prints. Co-Authored-By: Claude <noreply@anthropic.com>
The gpio-all-pins-cdc example still mapped D8 to PA15 (the BAT_EN net); the board dtsi and the xiao-stm32c5-test-plan sample fixed this to PE2 (schematic D8 net) but the example was never updated. Port the fix. The board re-spin renamed the PB14 header net and silkscreen from D15 to D18. The example now displays D18 and accepts both d18 and d15 commands (matched before the generic dN path, since bare numeric 18 is BOOT0); board-file comments follow the rename. The connector gpio-map index stays 15 for API compatibility. Co-Authored-By: Claude <noreply@anthropic.com>
The board-level skeleton (environment assembly, toolchain defaults, target_firm matrix, upload dispatch) is now the locked official platform-atmelsam v9.0.0 builder/main.py, executed by the thin wrapper in builder/families/samd/build.py; the arduino framework chain is the upstream arduino.py -> arduino/arduino-samd.py -> arduino_common.py (the retired samd_arduino.py was its semantically identical flattened copy; _bare.py was byte-identical). Deltas vs the retired local snapshot: - bossac hardening (7b3b9e5) re-applied as a wrapper post-graft: absolute tool-bossac executable, quoted abspath SOURCE in UPLOADCMD, and -U/--offset as separate flags. Cannot be a get_build_hooks graft: hooks run before main.py's sam-ba branch whose env.Replace would overwrite them. The graft rewrites the upstream "-U true|false" pair back into the hardened form; other sam-ba branches (samd51 fork, sam3x8e --boot) already matched the snapshot and stay untouched. - zephyr build-pre branch and stk500v2 hex selection (samd21 dead code): regressed to upstream by construction. - platform_cfg/samd_cfg.py inlined into families/samd.py in the PR-122 form (uploader pruning already removed); its cross-family comment carried. Verified: samd arduino-blink elf/bin byte-identical against the pre-migration baseline; SCons env dump identical on the whole upload chain (UPLOADER/UPLOADERFLAGS/UPLOADCMD zero diff); remaining dump deltas are inert duplicates (ASPPFLAGS/LIBPATH/LIBSOURCE_DIRS) proven so by the identical artifacts, plus BUILD_SCRIPT bookkeeping and the timestamp. rp2040 rebuild on this branch still matches its baseline (cross-family intact). 5 upstream anchors pinned in tests/test_upstream_anchors_samd.py. Co-Authored-By: Claude <noreply@anthropic.com>
Inline platform_cfg/renesas_cfg.py into families/renesas.py (package matrix, debug-tools table, debug session; owned_packages now guards the uploader pruning) and declare the upstream lock. The board-level skeleton is the upstream builder/main.py executed via the thin builder/families/renesas/build.py wrapper; framework_script() points at the upstream arduino.py (the Seeed framework tarball rides under the upstream package name). Retire the renesas_build.py / renesas_arduino.py / _bare.py snapshots (_bare was byte-identical). Family deltas live in builder/families/renesas/upload.py: - pyocd: upstream has no pyocd branch, so the protocol is taken over (rewrite to the custom passthrough) and the hex artifact chain is grafted afterwards — the hex records embed the 0x4000 app offset, while pyocd flash -a 0x0 on the upstream default .bin would write the app over the bootloader. The artifact enters via $FLASH_IMAGE instead of $SOURCE (same runtime command line; the upstream alias source is always the .bin). Installers moved verbatim, still unpinned (debt R1). - sam-ba: rides the upstream branch (its BeforeUpload port/touch sequence is missing from the custom passthrough) and is re-hardened after the fact: explicit bossac path, quoted absolute SOURCE, --verify, conditional --usb-port and --offset (upstream hard-codes bare bossac and no offset; XIAO RA4M1 needs 0x4000). configure_debug_session's old cfg wiring passed one argument to a two-parameter module function (TypeError whenever a debug session was configured); the inline restores the intended body. Lock pins the v1.9.0 commit 14e6238 (the guide's 34112f8 is the tag object; _upstream_matches_lock compares HEAD SHAs). Anchor tests in tests/test_upstream_anchors_renesas.py pin the upstream shapes both deltas ride on, plus the no-pyocd-branch absence. A/B vs the 624edf8 baseline: firmware.elf/.bin (dfu build) and firmware.hex (pyocd build) byte-identical; SCons env dumps for dfu/pyocd/sam-ba differ only in UNIX_TIME plus the two intentional pyocd mechanism lines (FLASH_IMAGE, UPLOAD_PROTOCOL='custom'). Co-Authored-By: Claude <noreply@anthropic.com>
…ate) platform_cfg/stm32_cfg.py (76 lines) moves into families/stm32.py as real methods (the esp/rpi/samd/renesas pattern): package matrix, stlink debug-tools entry, debug session. The tool-jlink prune now goes through the owned drop_package guard. configure_debug_session's old cfg wiring passed one argument to a two-parameter module function (TypeError whenever a debug session was configured -- same latent bug the renesas inline just fixed); the inline restores the intended body. Local-keep is already the terminal form: no upstream (STM32C5 is not in platform-stm32), stm32_build.py and the UF2/stlink chain are untouched, no wrapper, no upstream anchors. A/B vs the pre-migration baseline: zephyr-blink firmware.elf/.bin/ .hex/.uf2 (UF2 = the real upload payload) byte-identical; pytest 46 passed; verify_board_manifests 24/0. Co-Authored-By: Claude <noreply@anthropic.com>
…eaders platform_cfg/siliconlab_cfg.py (98 lines) moves into families/siliconlab.py as real methods (package matrix, cmsis-dap debug-tools entry, debug session); the dead openocd_path dict at the cfg top dies with the inline, and configure_debug_session's old one-argument wiring (TypeError whenever a debug session was configured — same latent bug as renesas/stm32) is fixed by the inline. No upstream is declared: the platform layer has nothing to load (v11.0.0 has no mg24 board and no arduino framework; the real framework upstream is SiliconLabsSoftware/arduino), so the builder scripts stay local. Provenance hygiene on the local scripts: siliconlab_build.py now heads with its fork point (platform-siliconlabsefm32 v11.0.0, tag fb80476 -> commit 0483049) and the delta list (matter includes.txt long-path workaround incl. time.h-conflict skips, fsutil case-sensitivity, explicit toolchain paths, removed dead Zephyr branch, hex artifact matrix, OpenOCD upload branch); _bare.py is annotated as the upstream byte-identical copy (minus one trailing newline); siliconlab_arduino.py as the no-upstream SCons-ification of SiliconLabsSoftware/arduino 2.3.0. File bodies otherwise untouched — data bugs stay on the S1-S7 backlog. New family unit tests (tests/test_family_siliconlab.py, esp debug-tools test form): toolchain pin, mandatory framework/openocd, per-OS openocd redirect, no cross-family pruning, cmsis-dap server args / preload / init_cmds, missing-target assertion. No upstream anchors (local-keep). A/B vs the pre-migration baseline: mg24 + mg24-sense firmware.elf/ .hex byte-identical; pytest 62 passed; verify_board_manifests 24/0. Co-Authored-By: Claude <noreply@anthropic.com>
With siliconlab inlined, no code imports platform_cfg anymore; the directory was down to its README and an empty __init__.py. Delete both. The remaining references are stale pointers for the final cleanup PR: workflow trigger paths and the zephyr cache-key entry (match nothing once the directory is gone), and the AGENTS.md ownership row. families/base.py's hooks-section comment still described delegation to platform_cfg/*; update it to the current state (families own the hooks). pytest 62 passed; verify_board_manifests 24/0. Co-Authored-By: Claude <noreply@anthropic.com>
P1 (live bug): _apply_patch_hunk checked the old block before the already-applied (new) block, so a pure-context insertion hunk -- the old block still matches after the hunk lands -- re-inserted on every rebuild (a production package cache was found carrying the FindGnuLd insertion 4x; patch 0002 hunk 1 hits this on every stm32c5 build). Check new-block-first (mentor 609168e). Verified on a pristine extraction of the pinned 3.40400.260428 tarball: old order duplicates the marker (count 1 -> 2 across two runs, applied+1 each rebuild), new order applies once and the second run is a full no-op (already-applied=2, count stays 1). Also in this stop-the-bleed commit: - P4: drop _ensure_system_path_available -- it read PATH from os.environ and wrote the same value back (literal no-op). - P5: drop _patch_cdc_vidpid -- dead since the board copy mode became an unconditional "refresh" (17ada25): the framework copy of the 20B Kconfig.defconfig is refreshed from the platform source on every build, and that source already carries CDC_ACM_SERIAL_VID/PID (zephyr/nrf/boards/seeed/xiao_nrf54lm20b/Kconfig.defconfig:51), so the function's own guard always returned early. - double imports: shutil was imported at module top and again in two function bodies; keep the top-level one. - bare pip: the pyyaml bootstrap in zephyr_fixes.py ran bare "pip" (wrong interpreter if PATH differs); use sys.executable -m pip like zephyr.py already does. Validation: pytest tests/ -q -> 50 passed, 14 skipped (same suite as the branch base; no test imports these modules). P1 experiment above run against both old (bug reproduced) and new (no-op) executors. Co-Authored-By: Claude <noreply@anthropic.com>
cumin777
changed the base branch from
stack/siliconlab-family-migration
to
dev
October 30, 2026 01:53
cumin777
added a commit
to cumin777/platform-seeedboards
that referenced
this pull request
Oct 30, 2026
…ols/ retires builder/tools/ held exactly two scripts, both consumed only by builder/families/stm32/stm32_build.py (BinToUf2 conversion and the UF2 UPLOADCMD). A "generic tools" directory with no generic tools is a misfiled family asset, so the scripts now live flat in builder/families/stm32/ next to _bare.py and stm32_build.py -- same precedent as nordic's mcumgrupload.py: an UPLOADCMD helper split out as a subprocess for Ctrl+C interruptibility, held by the family whose builder invokes it. The empty directory (and its pycache) is gone, and AGENTS.md's ownership rules now say family utilities live with their family instead of describing a tools/ admission policy with no tenants. stm32_build.py points both invocations at the new location (still via platform.get_dir() -- SConscript frames have no __file__, and the local-keep stm32 family resolves get_dir() to this repo). uf2conv.py drops dead code in the move: the FAMILY_IDS lookup table, the -f/--family flag and its fallback were a generic-boards relic -- the single caller always passes --family-id with the raw value from the board manifest, so --family-id becomes required and the table goes. uf2upload.py is moved untouched: the 1200-bps touch semantics, busy/denied volume classification and WinError 433 handling were debugged the hard way and carry no dead code. Validation: baseline firmware.uf2 built on the parent commit, tree rebuilt after the move -- SHA256 identical (DED8FA6E...), 95744 bytes (pure move + dead-code removal, zero output change); pytest tests/ -q -> 97 passed, 1 skipped (unchanged); verify_board_manifests 24/0; verify_zephyr_routing OK; residual grep for builder/tools across tracked files: zero hits (workflow path filters included). Hardware: not performed (UF2 upload chain is queued for the PR Seeed-Studio#130 review round). Co-Authored-By: Claude <noreply@anthropic.com>
cumin777
force-pushed
the
feature/zephyr-builder-refactor
branch
from
October 30, 2026 05:38
dd34abc to
bcf227d
Compare
cumin777
added a commit
to cumin777/platform-seeedboards
that referenced
this pull request
Oct 30, 2026
…ols/ retires builder/tools/ held exactly two scripts, both consumed only by builder/families/stm32/stm32_build.py (BinToUf2 conversion and the UF2 UPLOADCMD). A "generic tools" directory with no generic tools is a misfiled family asset, so the scripts now live flat in builder/families/stm32/ next to _bare.py and stm32_build.py -- same precedent as nordic's mcumgrupload.py: an UPLOADCMD helper split out as a subprocess for Ctrl+C interruptibility, held by the family whose builder invokes it. The empty directory (and its pycache) is gone, and AGENTS.md's ownership rules now say family utilities live with their family instead of describing a tools/ admission policy with no tenants. stm32_build.py points both invocations at the new location (still via platform.get_dir() -- SConscript frames have no __file__, and the local-keep stm32 family resolves get_dir() to this repo). uf2conv.py drops dead code in the move: the FAMILY_IDS lookup table, the -f/--family flag and its fallback were a generic-boards relic -- the single caller always passes --family-id with the raw value from the board manifest, so --family-id becomes required and the table goes. uf2upload.py is moved untouched: the 1200-bps touch semantics, busy/denied volume classification and WinError 433 handling were debugged the hard way and carry no dead code. Validation: baseline firmware.uf2 built on the parent commit, tree rebuilt after the move -- SHA256 identical (DED8FA6E...), 95744 bytes (pure move + dead-code removal, zero output change); pytest tests/ -q -> 97 passed, 1 skipped (unchanged); verify_board_manifests 24/0; verify_zephyr_routing OK; residual grep for builder/tools across tracked files: zero hits (workflow path filters included). Hardware: not performed (UF2 upload chain is queued for the PR Seeed-Studio#130 review round). Co-Authored-By: Claude <noreply@anthropic.com>
cumin777
force-pushed
the
feature/zephyr-builder-refactor
branch
from
October 30, 2026 07:06
bcf227d to
07dd2ec
Compare
…ml registry
Fixes move into the owning board directory -- the directory IS the
registration:
zephyr/<family>/boards/seeed/<board>/fixes/<framework-relative-path>
(.patch suffix = unified diff, no suffix = full-file override with a
provenance header: upstream link, validated version, exit condition).
fixes.baseline records the pristine upstream sha256 per target, giving
content-based drift detection for both patches and overrides -- this
replaces the applies_to version strings, which gated on a version label
instead of the actual content and broke silently on any framework
repackage.
Review-round hardening (fixup into this commit):
- override baseline drift now FAILS instead of warn-and-copy: clobbering
a whole upstream file an override was never cut against is the most
destructive silent failure there is, and the patch path already failed
hard -- the two executors are now symmetric. (Test: drifted override
target raises and the file is left untouched.)
- deletion-only patch hunks that miss are counted distinctly
("already-applied-deletion") and, when the walker holds a baseline and
the target still hashes PRISTINE, raise "patch/baseline mismatch": the
old block being gone proves nothing there (G0's idempotence left this
gap -- at the patch layer a deletion miss is indistinguishable from a
legitimate re-run). Audit: all four engine patches carry add-present
hunks that fail first, so current exposure is zero; this guards a
future deletion-only patch. Real re-runs (target no longer pristine)
stay silent, unchanged.
Ported from the mentor line (609168e + 5683cc7 + f755cc6, adapted to
this repo's family layout) together with the six PR-Seeed-Studio#117 review fixes:
Windows relpath normalization (87cad4e), deletion hunks under
new-block-first idempotence (2b6328c), baseline sha validation in both
parsers (c6fcbc6), 18 walker/executor unit cases (e3c9b5f), OS/editor
artifact skip in the walk (9ca8649), hunk-edge deletion already-applied
trap (46c11a9).
Board-specific gates in zephyr.py now key on the canonical Zephyr board
name (build.zephyr.variant's first component) instead of the PIO board
id: _provision_xiao_dfu_module runs only for xiao_nrf54lm20b (it
previously required 20B sources on every board), the module copy block
runs only for xiao_stm32c5. The dead-variable no-op from the P4/P5
cleanup stays dead-deleted. fixes.yml, patches/ and overrides/ are gone
(git tracks the moves as renames).
verify_zephyr_routing.py is a new offline gate (board -> variant-derived
board name -> family board dir chain closes; baseline <-> fix file
two-way correspondence, sha shape validation), wired into
check-board-manifests.yml; the fixes.yml cross-check (R6) leaves
verify_board_manifests.py for it.
Equivalence proof (fresh extraction of the pinned 3.40400.260428
tarball; old mechanism replayed from commit 956acc4 next to the new
walker, sharing one executor):
- patch targets byte-identical (adc_stm32.c 085854cf, FindGnuLd.cmake
ae76d621 under both mechanisms);
- override targets differ only by the provenance header, zero removed
upstream lines;
- a second new-mechanism run is a full no-op for patches;
- all five fixes.baseline shas match the pristine tarball files.
Validation: pytest tests/ -q -> 84 passed, 0 skipped in this stop's slice (21 walker/executor cases: 18 as before + 3 new -- deletion-stats, deletion-vs-pristine raise, deletion re-run silence; the override-drift case now asserts a raise and an unclobbered file)
executor cases included); verify_board_manifests 24/0;
verify_zephyr_routing OK (5 zephyr boards); real-cache smoke build
pio run examples/seeed-xiao-stm32c5/zephyr-blink -> SUCCESS (63.6s,
fixes walker re-ran over a production-patched framework cache).
Hardware upload/debug: not performed.
Co-Authored-By: Claude <noreply@anthropic.com>
The five _patch_platformio_* string surgeries (~292 lines in builder/frameworks/zephyr.py) are now five standalone unified-diff patches under zephyr/fixes/scripts/platformio/, applied by the same fixes walker as the board-level fixes -- framework level first, board level on top. fixes.baseline records the pristine sha256 of scripts/platformio/platformio-build.py from the pinned framework-zephyr 3.40400.260428 tarball, so a repackaged tarball now fails loudly with baseline-drift context instead of silently not matching (the old surgeries just skipped on a needle miss). The patch file names (.01-.05) are labels only; a patch target of record is its own +++ header, so several patches may edit the same file and one baseline key covers them all. The walker and the offline gate both resolve targets this way (zephyr_patch.patch_target is the shared helper), and the gate now covers the framework-level directory too. The five changes were verified independent before cutting (each host function touched by exactly one patch), so the patches apply in any order; the walker applies them in sorted name order for determinism. Provenance (why / upstream / exit condition) is in each patch header, same convention as v2 overrides. Equivalence proof (fresh extraction of the pinned tarball; old mechanism replayed from commit e848884): - outputs byte-identical after LF normalization (both 5d1d7da34094); the raw-byte difference is the OLD mechanism Windows text-mode side effect -- it rewrote all 2331 line endings to CRLF (the production cache carries them); the new walker preserves the pristine LF. - two consecutive walker runs are a full no-op for all five patches; - upgrade drill: tampering the target raises "patch hunk did not match ... baseline drift: ... expected pristine 49741b88" as intended. Validation: pytest tests/ -q -> 85 passed, 1 skipped (4 new framework-level walker cases: label-named patches sharing one target, boardless application, drift reporting, missing +++ header); verify_zephyr_routing OK (5 zephyr boards); real-cache smoke build pio run examples/seeed-xiao-stm32c5/zephyr-blink -> SUCCESS (18.6s incremental, walker re-ran over the production CRLF-patched cache). Hardware upload/debug: not performed. Co-Authored-By: Claude <noreply@anthropic.com>
Three provisioning paths used to exist for one purpose: a hardcoded "seeed-xiao-stm32c5" block copying every zephyr/modules/ entry, _provision_xiao_dfu_module refreshing the 20B DFU module unconditionally for every Zephyr board (P6 leak), and _provision_edge_ai with external module pins written as code constants (P9) plus the dead PIO_NCS_MODULES variable (P8) and the xiao module-discovery engine patch Seeed-Studio#5. Now builder/frameworks/zephyr_modules.py walks zephyr/modules/ once: - a directory IS a module; pin.yml (optional) carries its gate (boards / families / prj_conf_tokens, AND across fields, OR within a list -- builder only reads fields, never module names) and, for external sources, remote/revision/override_env; - vendored modules refresh-copy into the framework cache (pin.yml stays out of the copied tree); external stubs ensure-clone the pinned revision on first use, apply the module own fixes/ (v2 convention, same walker as board fixes), and publish override_env so sample CMakeLists keep working. Each clone records its remote/revision in a .pin file and every cache hit reconciles against the pin.yml spec: a bumped revision (or a cache that predates .pin) re-clones instead of silently serving the stale tree -- "pinned" now holds for every build, not just the first; - everything registers through ZEPHYR_EXTRA_MODULES -- Zephyr official channel consumed by zephyr_module.cmake from the environment -- so sysbuild child images (mcuboot) see the modules too; engine patch Seeed-Studio#5 (CMakeCache -D injection, did NOT reach child images) is retired. The Edge Impulse SDK cmake corrections move from code (_patch_edge_impulse_sdk) to zephyr/modules/edge-impulse-sdk-zephyr/fixes/ as two provenance-headed patches with a pristine-revision fixes.baseline (sha256 values computed from the pinned revision 69a6b8f tree). zephyr.py sheds the Edge AI section, both module blocks and the clone helpers (git_clone_with_retry now lives with its two callers in zephyr_modules.py); west pre-install imports it from there. zephyr/ README.md is rewritten to the end-state layout (fixes v2 + modules + pin.yml reference; module section covers both vendored and pinned conventions). Stale fixes.yml pointers in AGENTS.md / README.md fixed. Validation (workspace platform via symlink override): - pytest tests/ -q provisioner slice -> 15 passed (12 provisioner cases as before + 3 pin-reconciliation: matching pin reuses cache with zero clones, revision bump re-clones exactly once and refreshes .pin, legacy .pin-less cache self-heals once; gate AND/OR semantics, pin.yml exclusion, ZEPHYR_EXTRA_MODULES dedup, override_env precedence, clone-failure tolerance, module fixes); verify_zephyr_routing OK; verify_board_manifests 24/0; - production cache: stm32c5 zephyr-blink SUCCESS, 20B zephyr-blink clean rebuild SUCCESS (mcuboot signing chain intact), edgeai-hello-ei SUCCESS (sdk-edge-ai gate hit via CONFIG_NRF_AXON=y; registered modules exactly xiao_dfu_reset + sdk-edge-ai); - CLEAN cache (pristine 3.40400.260428 engine, zero legacy injection code -- env channel is the only registration path): stm32c5 zephyr-blink SUCCESS (1326s full first build; cannectivity + uf2_dfu_reset registered via env only, xiao_dfu_reset correctly absent) and 20B zephyr-blink SUCCESS (44.8s; xiao_dfu_reset via env; sysbuild/mcuboot configured fine) -- Windows CMake semicolon-list + drive-letter parsing and sysbuild env penetration both hold (plan 4.3 checks 1+2), no thin sysbuild patch fallback needed. Hardware upload/debug/DFU touch: not performed (matrix queued for the review pass; 20B touch->loader, 20A-V2, stm32c5 1200->UF2). Co-Authored-By: Claude <noreply@anthropic.com>
…_dfu xiao_dfu_reset (nRF54LM20A V2/B) and uf2_dfu_reset (STM32C5) did the same job -- watch USBD_MSG_CDC_ACM_LINE_CODING for a 1200-bps DTE rate and enter the bootloader -- with two divergent implementations: ride-along callbacks on an existing USB stack vs. a self-contained USB stack with a hardcoded VID:PID duplicating board.json hwids; retention boot-mode vs. hand-written TinyUF2 magic; board-gated vs. unconditional Kconfig defaults. The merged zephyr/modules/cdc_touch_dfu/ is the thin generic core: - ride-along only: it walks the usbd_context section and registers one message callback per context the board/sample already created -- the module owns no USB stack of its own; - the reboot backend is board data, not code forks: a uf2_dbl_tap device node (STM32C5 DTS, reg = <0x2003fffc 0x4>) selects the TinyUF2 double-tap backend; without it, the retention boot-mode backend (nRF54LM20) applies. The retention Kconfig selects are gated on DT_HAS_ZEPHYR_RETENTION_ENABLED so they stay off where the board describes no retention node (a plain select aborted Kconfig on STM32C5). Zephyr 4.4 generates no DT_CHOSEN_* for custom chosen keys, hence the nodelabel reference rather than a chosen property; - Kconfig defaults are board-gated across all three boards (default y if BOARD_XIAO_NRF54LM20B_.../20A V2/XIAO_STM32C5). STM32C5 USB ownership moves to the board where it belongs: board.c (board dir CMakeLists.txt wires it in) now owns the device definition, descriptors and boot bring-up, with the VID:PID fed from new XIAO_STM32C5_USB_VID/PID Kconfig defaults (0x2886:0x00C5, matching the TinyUF2 bootloader and boards/seeed-xiao-stm32c5.json hwids) instead of a #define in module code. The block keys on a derived, promptless XIAO_STM32C5_USB_AUTOBRINGUP (default y if USB_DEVICE_STACK_NEXT && !USBD_GS_USB), not on USB_DEVICE_STACK_NEXT alone: samples that own their whole USB stack -- the gs_usb CAN bridge builds a second usbd context on the same zephyr_udc0 -- must keep the board context out entirely, or the board init would double-init the controller and register_all_classes would swallow the sample's classes; USBD_GS_USB is the same conflict signal the CONSOLE defaults already key on, so gsusb samples opt out with zero changes. The 42 sample prj.conf lines naming the old symbol carry the new CONFIG_CDC_TOUCH_DFU name (same polarity, incl. the =n power-measure set). The TinyUF2 double-tap hardware trigger remains the fallback everywhere. Validation (workspace platform): gsusb escape hatch -- the real zephyr-can-bridge-gsusb sample builds SUCCESS with USBD_GS_USB=y and contains ZERO xiao_stm32c5_usb_init/xiao_stm32c5_usbd symbols (the derived gate compiles the board context out); stm32c5 zephyr-blink clean build SUCCESS (both xiao_stm32c5_usb_init and cdc_touch_dfu_init in the ELF; UF2 image 187 blocks; CONFIG_RETENTION absent -- TinyUF2 backend clean); stm32c5 zephyr-empty (module AND USB stack off) SUCCESS, 52 blocks; 20B zephyr-blink clean build SUCCESS with MCUboot signing (bootmode backend). pytest tests/ -q -> 83 passed, 15 skipped; both offline gates OK. Hardware regression matrix NOT yet run (queued for the review pass): 20B touch->loader, 20A-V2 touch, stm32c5 pio upload (1200 touch -> XIAOC5BOOT drive -> uf2 copy), power-measure current on module-off builds. Co-Authored-By: Claude <noreply@anthropic.com>
…s on digest
The bare call chain (venv -> west workspace -> west deps) is now the
PREP_STEPS data table: order is explicit, adding a step is a one-line
diff, each step prints a [zephyr-prep] tag logs can be searched by, and
steps stay independently testable. No class hierarchy, no behavior
change (D8).
The two pip subprocesses every build spawned (requirements-base.txt +
the pinned set) even when nothing changed are now skipped: the venv
marker (pio-zephyr-venv.json) carries a deps digest (sha256 of
requirements-base.txt + the pinned list), and an unchanged digest
short-circuits both installs. The digest is recorded only after BOTH
installs succeed -- a failed install leaves the stale digest (or no
digest) in the marker, so the next build re-runs pip instead of
skipping it with a half-provisioned venv. pip is idempotent, so the
retry is harmless, and the reverse edge (installs succeed, marker write
fails) also self-heals on the next build.
The marker version field stays "1.0.0" -- it must equal the engine's
ZEPHYR_ENV_VERSION (framework package scripts/platformio/
platformio-build.py): that script reads the same marker after ours and
RECREATES the venv whenever the version differs, wiping deps_sha and
re-running its own pip. An earlier cut of this change bumped the field
to 1.1.0, which put the two layers into a recreate ping-pong on every
build (observed with a marker file watcher: our 1.1.0 write deleted
and rewritten as {"version": "1.0.0"} within the same build); deps_sha
rides along as an extra key the engine never inspects.
D5 note: the planned family.zephyr_provision() hook turned out to have
nothing left to host -- after the provisioner commit, every Edge AI /
module property is data (pin.yml gates), so the hook would be a
permanently-empty abstraction. Dropped in favour of the plain
provisioner call.
Validation: digest logic driven directly (matching digest -> zero pip
subprocesses; mismatch -> both pips + marker refresh). Failure path:
injecting a nonexistent pinned package -> pip fails, build aborts, the
marker carries NO digest; removing it -> pip re-runs, build succeeds,
digest recorded; next build -> "[zephyr-prep] venv dependencies
unchanged, skipping pip", no pip subprocess (stm32c5 6.5s / 20B 15.6s,
down from 40-60s). Marker file watcher across two consecutive builds:
stable {"version": "1.0.0", "deps_sha": ...} with no mid-build rewrite.
Smoke builds stm32c5 + 20B zephyr-blink SUCCESS; pytest tests/ -q ->
97 passed, 1 skipped (was 93+5 at dd34abc^ ; suite total unchanged).
Hardware: not performed.
Co-Authored-By: Claude <noreply@anthropic.com>
…phyr/, prefixes dropped
The five Zephyr builder files become one directory, matching the shape
upstream platforms already use (platform-nordicnrf52 dispatches
frameworks/arduino/adafruit.py, which our nordic family borrows):
builder/frameworks/zephyr.py -> builder/frameworks/zephyr/main.py
builder/frameworks/zephyr_fixes.py -> builder/frameworks/zephyr/fixes.py
builder/frameworks/zephyr_modules.py -> builder/frameworks/zephyr/modules.py
builder/frameworks/zephyr_patch.py -> builder/frameworks/zephyr/patch.py
builder/frameworks/zephyr_override.py -> builder/frameworks/zephyr/override.py
builder/frameworks/ is back to one file per framework (arduino.py,
espidf.py). Pure moves plus the mechanical renames: platform.json
framework entry, main.py's sys.path gains the "zephyr" segment (still
built from platform_dir = family.host_dir() -- SCons deletes __file__ in
SConscript frames), imports drop the zephyr_ prefix (fixes / modules /
patch / override -- same flat-name-on-sys.path mechanism nordic/build.py
documents; the names resolve only inside this build process), the three
test files follow, and stale path comments (zephyr/README.md,
nordic/build.py, families/nordic.py, one example ini) point at the new
layout. Two dead path filters in build-zephyr-examples.yml (repo-root
frameworks/zephyr.py never existed; builder/** already covers the
moves) are dropped.
One real code touch beyond renames: the patch-executor tests had a
local variable named "patch" that shadowed the newly bare module name;
renamed to patch_file.
Validation: pytest tests/ -q -> 97 passed, 1 skipped (count unchanged
from the pre-move base); verify_zephyr_routing OK (5 zephyr boards);
verify_board_manifests 24/0; smoke builds with the new entry point --
stm32c5 zephyr-blink SUCCESS 15.2s ("[zephyr-prep] venv dependencies
unchanged, skipping pip") and 20B zephyr-blink SUCCESS 18.9s (mcuboot
signing chain intact), exercising platform.json's new script path, the
zephyr/ sys.path segment and the modules -> fixes -> patch/override
load chain; residual grep for zephyr_(fixes|modules|patch|override) and
frameworks/zephyr.py across tracked files: zero hits.
Hardware: not performed.
Co-Authored-By: Claude <noreply@anthropic.com>
…ols/ retires builder/tools/ held exactly two scripts, both consumed only by builder/families/stm32/stm32_build.py (BinToUf2 conversion and the UF2 UPLOADCMD). A "generic tools" directory with no generic tools is a misfiled family asset, so the scripts now live flat in builder/families/stm32/ next to _bare.py and stm32_build.py -- same precedent as nordic's mcumgrupload.py: an UPLOADCMD helper split out as a subprocess for Ctrl+C interruptibility, held by the family whose builder invokes it. The empty directory (and its pycache) is gone, and AGENTS.md's ownership rules now say family utilities live with their family instead of describing a tools/ admission policy with no tenants. stm32_build.py points both invocations at the new location (still via platform.get_dir() -- SConscript frames have no __file__, and the local-keep stm32 family resolves get_dir() to this repo). uf2conv.py drops dead code in the move: the FAMILY_IDS lookup table, the -f/--family flag and its fallback were a generic-boards relic -- the single caller always passes --family-id with the raw value from the board manifest, so --family-id becomes required and the table goes. uf2upload.py is moved untouched: the 1200-bps touch semantics, busy/denied volume classification and WinError 433 handling were debugged the hard way and carry no dead code. Validation: baseline firmware.uf2 built on the parent commit, tree rebuilt after the move -- SHA256 identical (DED8FA6E...), 95744 bytes (pure move + dead-code removal, zero output change); pytest tests/ -q -> 97 passed, 1 skipped (unchanged); verify_board_manifests 24/0; verify_zephyr_routing OK; residual grep for builder/tools across tracked files: zero hits (workflow path filters included). Hardware: not performed (UF2 upload chain is queued for the PR Seeed-Studio#130 review round). Co-Authored-By: Claude <noreply@anthropic.com>
cumin777
force-pushed
the
feature/zephyr-builder-refactor
branch
from
October 30, 2026 08:21
07dd2ec to
4a297f9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Zephyr builder cleanup: fixes as data, one module provisioner, merged DFU trigger
Zephyr builder refactor per the 2026-10-29 plan (P1–P12 problem list → D1–D11 decisions). Stacked on #129 (
feature/siliconlab-family-migration, the last of the family-migration PRs): the base here is that branch so the diff shows only the six refactor commits; once #125–#129 merge, retarget this PR todev(change-base, no rebase expected — the family stack barely touchesbuilder/frameworks/zephyr.py).Draft until the hardware regression matrix at the bottom has run.
Story directory: problem → commit
_apply_patch_hunkold-first order: pure-insertion hunks re-applied every rebuild (production cache carried the FindGnuLd insertion 4x)956acc4_ensure_system_path_availableread PATH and wrote it back (literal no-op)956acc4_patch_cdc_vidpiddead since the copy mode became unconditional refresh (guard always hit)956acc4e848884board_name == "seeed-xiao-stm32c5"hardcode → canonical Zephyr board name (mentor 5683cc7)e848884_patch_platformio_*string surgeries (~292 lines), no version gating66467d0_provision_xiao_dfu_moduleduplicated the board-refresh loop and ran for every Zephyr board29521a4PIO_NCS_MODULESset, never consumed anywhere29521a429521a41d886a9dd34abce848884(rewritten again at29521a4to the end-state layout)Each commit message carries its full validation record (the intermediate commits have no independent CI — the evidence chain lives in the messages).
What changed, per layer
Executors (
zephyr_patch.py/zephyr_override.py): new-block-first idempotence (+ deletion-hunk semantics from the PR-#117 review fixes),patch_target()— a patch's+++header is the target of record; file names are labels, so several patches may edit one file against one baseline key.Walker (
zephyr_fixes.py): directory convention —zephyr/fixes/(framework level, engine patches) applies first,zephyr/<family>/boards/seeed/<board>/fixes/on top.fixes.baselinerecords pristine sha256 per target: overrides warn, patch hunks fail with baseline-drift context on framework upgrades. No registry file, no version strings.Engine patches (
zephyr/fixes/scripts/platformio/*.patch): the five string surgeries are now four standalone patches cut against the pristine 3.40400.260428 tarball (patch #5, CMakeCache module injection, retired in favor of the environment channel — see below), each with a provenance header (why / upstream / exit condition).Modules (
zephyr_modules.py+zephyr/modules/): one provisioner walkszephyr/modules/; a directory IS a module.pin.ymlcarries the gate (boards/families/prj_conf_tokens, AND across fields, OR within a list) and, for external sources,remote/revision/override_env. Vendored modules refresh-copy into the framework cache; external stubs ensure-clone the pinned revision and apply their ownfixes/(the two Edge Impulse SDK cmake corrections moved here from code). Registration is single-channelZEPHYR_EXTRA_MODULES— Zephyr's own mechanism, which reaches sysbuild child images (verified on a clean cache; the CMakeCache injection did not).DFU trigger (
zephyr/modules/cdc_touch_dfu/): the two 1200-bps reset modules merged into one ride-along module — it registers a line-coding callback on whatever USB contexts exist and owns no USB stack. The reboot backend is board data: auf2_dbl_tapdevice node (STM32C5) selects the TinyUF2 backend, otherwise retention boot-mode (nRF54LM20). STM32C5 USB bring-up and VID:PID moved to the board (board.c+XIAO_STM32C5_USB_VID/PIDKconfig defaults).Orchestrator (
zephyr.py): 853 → 417 lines; the environment prep chain is thePREP_STEPStable; the venv marker carries a dependency digest so unchanged builds skip both pip subprocesses. Plannedfamily.zephyr_provision()hook dropped — after the provisioner, there is nothing left for it to host.Verification done locally
verify_board_manifests24/0;verify_zephyr_routingOK (5 boards).Before requesting review (hardware matrix, plan §4.4)
pio uploadfull chain (1200 touch → XIAOC5BOOT drive → uf2 copy)Known local-environment caveat (not in the PR): on this dev machine PlatformIO's
symlink://platform installs as a copy underplatforms/SeeedStudioand does not auto-refresh; local rebuilds after editing the platform need a robocopy mirror first. CI is unaffected.