Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion .claude/skills/forge-ci/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ Then the two log sources:

```bash
# Job log (build phase, staging, packaging errors):
gh api repos/<fork>/actions/jobs/<job-id>/logs > job.log
gh api --allow-escape-sequences repos/<fork>/actions/jobs/<job-id>/logs > job.log
grep -nE "error:|CMake Error|No matching distribution|FAILED" job.log

# On-device test output (the actual pytest run) — it is an ARTIFACT, usually
Expand All @@ -185,6 +185,13 @@ gh run download <run-id> --repo <fork> -D artifacts/
cat artifacts/test-py3.12-<platform>-<pkg>-*/console.log
```

Without `--allow-escape-sequences`, `gh` may refuse (*"the response contains terminal escape
sequences"*), write nothing and exit 1 — and a grep over the empty file is a silent false
negative. console.log does not print the interpreter, so after a `mobile_test_pythons=ALL`
run confirm each leg from its job log: the interpreter, then the wheel pip actually
installed (flet's `-vv` output wraps lines, so join them first):
`grep -oE -- '--python-version [0-9.]+' job.log; sed -E 's/^[^ ]+ +//; s/ +$//' job.log | tr -d '\n' | grep -oE 'Processing */[^ ]*dist-test/<pkg>-[A-Za-z0-9_.+-]*\.whl'`.

A missing console.log artifact for a failed 3.12 job means the job died
*before* the device test — almost always at recipe-tester packaging
(resolution: see Chains) rather than on the device.
Expand Down
46 changes: 45 additions & 1 deletion .claude/skills/local-recipe-testing/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,24 @@ DATA=$(xcrun simctl get_app_container "$UDID" com.flet.recipe-tester data)
for i in $(seq 1 30); do grep EXIT "$DATA/Library/Caches/console.log" 2>/dev/null && break; sleep 5; done
```

## Shortcut: the consumer example pass on CI's wheels

Once a CI run is green, its `wheels-py3.X-<platform>-<pkg>-*` artifacts are the exact
wheels the device tests used. Flatten them into one find-links dir and the example app
builds with **no local forge env at all** — no support tarballs, no NDK, no `forge` run:

```bash
gh run download <run-id> --repo <fork> -p 'wheels-*' -D /tmp/ci_wheels
mkdir /tmp/ci_wheels/findlinks && cp /tmp/ci_wheels/wheels-*/*.whl /tmp/ci_wheels/findlinks/
cd recipes/<pkg>/examples/<example>
PIP_FIND_LINKS=/tmp/ci_wheels/findlinks uv run flet build apk # or ios-simulator
```

The example builds for 3.14 (gotcha #2), so the run must include the 3.14 leg. To read the
app's own output without racing a screenshot against the update, the Flet patch log
carries every control value: `xcrun simctl spawn "$UDID" log show --last 2m --style compact
--predicate 'process == "<app-name>"'` (a release APK logs no patches; use screenshots there).

### forge slice syntax (quick reference)

`android:arm64-v8a` | `android:x86_64` | `android:armeabi-v7a` | `iphonesimulator:arm64` | `iphonesimulator:x86_64` | `iphoneos:arm64` — the first token is the **SDK**, not the OS. `forge iOS:arm64` dies with a raw `KeyError: 'iOS'` (only the bare-platform forms `forge android` / `forge iOS` take the OS name, and those build every arch).
Expand All @@ -115,6 +133,12 @@ for i in $(seq 1 30); do grep EXIT "$DATA/Library/Caches/console.log" 2>/dev/nul
sed -i '' 's/hw.ramSize=.*/hw.ramSize=6144/' "$cfg" 2>/dev/null || echo 'hw.ramSize=6144' >> "$cfg"
```
(`aosp_atd` is rootable but headless — it can't run a Flet GUI app, so don't use it here.)
If the emulator dies with *"Cannot find AVD system path. Please define ANDROID_SDK_ROOT"*
(or, with that variable set, *"Broken AVD system path"*), the AVD survived but its system
image did not (`system-images/` empty) — exporting the variable only changes the message.
Reinstall the image with the SDK's own sdkmanager; a Homebrew `sdkmanager` first on `PATH`
installs into its own root, where the emulator never looks:
`"$SDK/cmdline-tools/latest/bin/sdkmanager" --sdk_root="$SDK" "system-images;android-34;google_apis;arm64-v8a"`.

5. **Give the emulator RAM + disk.** A heavy `.so` (polars ~130 MB) + Python + the Flutter engine OOM-kills the app on a default AVD (`lowmemorykiller: Kill 'com.flet.recipe_tester'`). 6 GB RAM avoids it. The ~100–235 MB APK install needs a big `/data` (6 GB partition); if you hit `INSTALL_FAILED_INSUFFICIENT_STORAGE`, free space (uninstall old apps) or use a fresh AVD.

Expand All @@ -126,7 +150,7 @@ for i in $(seq 1 30); do grep EXIT "$DATA/Library/Caches/console.log" 2>/dev/nul

9. **Android `console.log` lives in the app's CACHE dir — `/data/data/com.flet.recipe_tester/cache/console.log` — NOT under `files/flet/app/`** (that's the app code; `python_site_packages` is a SIBLING under `files/flet/`). Polling the wrong dir looks like "the app never wrote a result" and cost ~10 min during the sherpa-onnx validation. Root is still required to read it (gotcha #4): `adb root` then `adb shell cat …`, or `adb shell su 0 cat …` on a google_apis image.

10. **`flet build ios-simulator` resolves the `iphoneos` (device) wheel AS WELL as both simulator ones.** It configures pip for `iphoneos.arm64` + `iphonesimulator.arm64` + `iphonesimulator.x86_64` and needs a wheel for EACH — a partial local matrix fails with `No matching distribution found`. Build all three iOS slices first (for the recipe AND every `flet-lib*` host dep). CI never hits this because it dumps all of `dist/*.whl` into its find-links dir. (`flet build apk` needs only the one `--arch` slice — the asymmetry is iOS-only.) Long-standing gotcha; re-hit during the onnxruntime iOS spike. **Worse: serious_python's PER-ARCH native staging (`build/site-packages/<iosarch>/`) resolves those slice wheels from the INDEX directly and does NOT honor `PIP_FIND_LINKS`/dist-test locally** — so a hand-patched `-9999` wheel in your find-links dir is used for the initial pip install but the staged PYTHON code (e.g. `cv2/__init__.py`, whichever slice it picks — often `iphoneos.arm64`) still comes from the published wheel. Net: you cannot validate a *hand-patched loader* on a local `ios-simulator` build; use a real `forge` build of all slices, or verify in CI (where the freshly-built slice wheels ARE used — this is why coolprop iOS passed in CI but a hand-patched opencv wouldn't locally).
10. **`flet build ios-simulator` resolves the `iphoneos` (device) wheel AS WELL as both simulator ones.** It configures pip for `iphoneos.arm64` + `iphonesimulator.arm64` + `iphonesimulator.x86_64` and needs a wheel for EACH — a partial local matrix fails with `No matching distribution found`. Build all three iOS slices first (for the recipe AND every `flet-lib*` host dep). CI never hits this because it dumps all of `dist/*.whl` into its find-links dir. (`flet build apk` needs only the one `--arch` slice — the asymmetry is iOS-only.) Long-standing gotcha; re-hit during the onnxruntime iOS spike. **Worse: a hand-patched wheel can still lose in serious_python's PER-ARCH native staging (`build/site-packages/<iosarch>/`).** That pip run does read the inherited `PIP_FIND_LINKS` (flet 0.86.5 logs "Looking in links: …", and an unpublished package such as lz4 resolves from it), but during the opencv work a hand-patched `-9999` wheel in the find-links dir was used for the initial pip install while the staged PYTHON code (e.g. `cv2/__init__.py`, whichever slice it picks — often `iphoneos.arm64`) still comes from the published wheel. Net: you cannot validate a *hand-patched loader* on a local `ios-simulator` build; use a real `forge` build of all slices, or verify in CI (where the freshly-built slice wheels ARE used — this is why coolprop iOS passed in CI but a hand-patched opencv wouldn't locally).

10a. **Old local Xcode can't compile newer Flutter plugins** — e.g. Xcode 16.4 dies on `device_info_plus` 12.4.0 with `ARC Semantic Issue: No visible @interface for 'NSProcessInfo' declares the selector 'isiOSAppOnVision'` (a visionOS selector added in a newer SDK). This is a LOCAL toolchain gap, not your recipe (CI's Xcode 26.5 is fine). Pin the offending plugins older in the generated app pyproject before `flet build ios-simulator`:
```toml
Expand All @@ -136,6 +160,26 @@ for i in $(seq 1 30); do grep EXIT "$DATA/Library/Caches/console.log" 2>/dev/nul
```
(`stage_recipe.sh` regenerates the pyproject, so append this AFTER staging.)

10b. **New local Xcode rejects flet's iOS deployment target.** Xcode 27 makes
`IPHONEOS_DEPLOYMENT_TARGET = 13.0` (flet's template from 0.86.5 through 1.0.3, still on
`main` as of 2026-10-03) a hard error: *"set to 13.0, but the range of supported deployment target
versions is 15.0 to 27.0.x"*. Plain `flet build` output hides it behind "Failed to build
iOS app" and a doctor dump; `-vv` shows the `error:` lines. Not your recipe, and CI's
Xcode 26 is unaffected. Workaround — flet has already staged everything, so patch the
generated project and re-run only the flutter step with the same env:
```bash
cd build/flutter/ios
sed -i '' "s/platform :ios, '13.0'/platform :ios, '15.0'/" Podfile
# in Podfile's post_install target loop, before GCC_PREPROCESSOR_DEFINITIONS:
# config.build_settings['IPHONEOS_DEPLOYMENT_TARGET'] = '15.0'
sed -i '' 's/IPHONEOS_DEPLOYMENT_TARGET = 13.0;/IPHONEOS_DEPLOYMENT_TARGET = 15.0;/' Runner.xcodeproj/project.pbxproj
cd .. && SERIOUS_PYTHON_APP=$PWD/../python-app SERIOUS_PYTHON_SITE_PACKAGES=$PWD/../site-packages \
SERIOUS_PYTHON_VERSION=3.14 SP_NATIVE_SET=$(cat ../.serious_python_spm_key) \
flutter build ios --simulator --build-name 1.0.0 # VERSION = the Python flet built for
# → build/flutter/build/ios/iphonesimulator/<app>.app
```
A fresh `flet build` regenerates the template, so repeat the patch after every one.

11. **Two booted simulators make `simctl booted` ambiguous.** With more than one sim booted, `simctl install booted …` targets one device and your subsequent `get_app_container booted …` may query the OTHER — the app "isn't installed" / the container is empty despite a successful install. Use the explicit `$UDID` for every simctl call (as the loop above does); never rely on `booted` unless you've verified exactly one device is booted (`xcrun simctl list devices | grep -c Booted`).

12. **Verify the staged tests + the on-device test COUNT — staging can fail silently.** `stage_recipe.sh` wipes and re-stages `recipe_tests/`; if the invocation ever fails without you noticing (a scripted loop with a bad variable — zsh does NOT word-split unquoted `$VAR` like bash, so a `for r in $RECIPES`-style loop can pass the whole list as ONE argument), the PREVIOUS recipe's tests are still staged and run happily, reporting "N passed" for the wrong package. Two cheap checks after staging: `ls tests/recipe-tester/recipe_tests/` shows YOUR test files, and the "N passed" in console.log matches your recipe's test count. (Bit during the h5py→keras loop: the same 4 stale h5py tests "passed" three times.) **Stronger still — verify the built APK's CONTENTS, not just `recipe_tests/`:** a build that *fails* can leave a STALE `build/apk/recipe-tester.apk` that installs the wrong app entirely. `unzip -l build/apk/recipe-tester.apk` should show your recipe's test `.py` inside `app.zip` AND (for a native recipe) `lib/<abi>/lib*.so` for its libs. Caught an opaque run that silently installed a stale pysodium APK and reported "2 passed" for the wrong package. When in doubt nuke `build/apk` too, not just `build/site-packages`.
Expand Down
38 changes: 38 additions & 0 deletions .claude/skills/new-mobile-recipe/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -235,6 +235,18 @@ you can `export NDK_HOME=~/Library/Android/sdk/ndk/<version>` and use that — f
the differences between r27 / r27d / r28 are immaterial. CI builds against r27d; your local r27
should produce equivalent wheels.

### Trap: a truncated support tarball

`download_support` in `setup.sh` returns early once the extracted
`downloads/support/python-<plat>-mobile-forge-<ver>/support/` exists, otherwise skips the
download whenever the tarball exists, and never checks `tar`'s exit status. A shell killed
mid-`curl` leaves a partial tarball behind (2026-10-01: an iOS 3.14.6 tarball at 34 MB of
296 MB): `tar` reports `truncated gzip input`, and depending on how far it got, setup either
fails its support-path check or keeps a half-extracted tree on every later run. Compare a
cached tarball with the release (`curl -sIL <url> | grep -i content-length` against
`stat -f%z`), and delete both the tarball and its extracted `downloads/support/…` directory
before re-running. On a slow link, fetch with parallel `curl -r` ranges and concatenate.

### Note: Android sysconfigdata CI paths — self-healing, nothing to fix

Historically the `python-android-mobile-forge-3.12.tar.gz` broke macOS local dev: CI-runner
Expand Down Expand Up @@ -298,6 +310,9 @@ Open `tests/test_<name>.py` and replace the placeholder with a real smoke test:
- **Every test function has a docstring** — one line saying what behavior it proves.
- Tests must be **network-free and deterministic** (fixed seeds, committed tiny assets) —
they run on an emulator with no guarantees about connectivity.
- **Type every parameter where Python allows it** — tests, fixtures (`tmp_path: Path`),
helpers, and the example apps' functions and nested handlers alike. Only lambdas are
exempt (no annotation syntax); domain functions also get a return type.

For ML/inference recipes, raise the bar from import-only to real compute:

Expand Down Expand Up @@ -544,6 +559,29 @@ what's committed vs generated).

`>>>>>>>>>> EXIT 0 <<<<<<<<<<` in console.log on both platforms = ready to ship.

### Audit the consumer README and example before the PR

Every sentence in `recipes/<name>/README.md` and the example is a claim a bump can break,
and plausible ones are often wrong. An adversarial claim audit of lz4 (one skeptic per
claim group, each told to refute with source quotes and runs against the sdist, PyPI's
desktop wheel and the reference tools) turned up defects that a green CI could not:

- **Upstream constants can overstate the library.** python-lz4 exports
`COMPRESSIONLEVEL_MAX = 16`, but the vendored liblz4 clamps HC at 12, so levels 13–16
produce byte-identical output. Check a "max" against the vendored C, not the binding.
- **An example with several buttons and `page.run_thread` races itself.** The pool runs
workers concurrently, so a second tap mid-run overlapped the first on a shared file and
failed it every time. Disabling the controls in the handler is **not enough**: on an iOS
simulator two taps reached Python 11 ms apart, before the `disabled` patch reached
Flutter, and the race still fired. Guard the handler itself —
`if not busy.acquire(blocking=False): return`, release at the end of the worker — and
disable the controls for the visual cue (`recipes/lz4/examples/log-archive`).
- **"Stateful object, one per thread" understates it** when the binding mutates the native
context with the GIL released: sharing an `LZ4FrameCompressor` crashed the interpreter.
- **Prose that says "streams" needs code that streams** — one `write()` of the whole
buffer is not it, and neither is a test that never calls `write()` twice.
- **Run flake8 over the example** (`--config .flake8`); E741 slipped through review.

### Wheel hygiene checklist (before commit)

Unzip the wheel(s) from `dist/` and inspect every native binary:
Expand Down
11 changes: 7 additions & 4 deletions recipes/av/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,13 +133,16 @@ control for playing the result.
PyAV releases the GIL around demuxing, decoding and encoding, so this work genuinely runs in
parallel with the UI. Hand it to
[`page.run_thread(...)`](https://flet.dev/docs/controls/page/#flet.Page.run_thread) — a
half-second decode on the event-handler thread is a visibly frozen app. `run_thread` swallows
exceptions and does not carry an automatic update with it, so catch your own failures and
finish the handler with an explicit
half-second decode on the event-handler thread is a visibly frozen app. `run_thread` reports a
failure only in the log and does not carry an automatic update with it, so catch your own
failures and finish the worker with an explicit
[`page.update()`](https://flet.dev/docs/controls/page/#flet.Page.update).

A single container is not safe to use from two threads at once. Give each thread its own
`av.open(...)`, or serialise access with a `threading.Lock`.
`av.open(...)`, or serialise access with a `threading.Lock`. Two runs writing one output
file need a lock even with separate containers, and `run_thread` uses a pool, so two quick
taps on a button run concurrently. Take it in the click handler: disabling the button alone
does not stop a tap that is already in flight.

### What this build can encode

Expand Down
3 changes: 2 additions & 1 deletion recipes/av/examples/clip-roundtrip/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ What it demonstrates:
- **Work off the UI thread** — the encode runs in
[`page.run_thread(...)`](https://flet.dev/docs/controls/page/#flet.Page.run_thread), which
needs an explicit [`page.update()`](https://flet.dev/docs/controls/page/#flet.Page.update)
at the end.
at the end. The handler also takes a non-blocking lock: every run rewrites the same clip,
and disabling the button does not stop a second tap that is already in flight.

## Try it

Expand Down
11 changes: 5 additions & 6 deletions recipes/av/examples/clip-roundtrip/src/clip.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
"""Writes a short MP4 with a video and an audio stream, reads it back to describe
what actually landed in the file, and pulls stills out of it as JPEG bytes an
`ft.Image` can display.
"""
"""Writes a short MP4 with a video and an audio stream, reads it back to describe what
actually landed in the file, and pulls stills out of it as JPEG bytes an `ft.Image` can
display."""

import io
import math
Expand Down Expand Up @@ -57,7 +56,7 @@ def _video_frame(index: int):
return frame


def _audio_frames(stream):
def _audio_frames(stream: av.AudioStream):
"""A 440 Hz tone, resampled into whatever layout the AAC encoder asked for."""
resampler = av.AudioResampler(
format=stream.format, layout=stream.layout, rate=SAMPLE_RATE
Expand Down Expand Up @@ -129,7 +128,7 @@ def probe(path: str) -> list[tuple[str, str]]:
return rows


def _jpeg(frame, width: int) -> bytes:
def _jpeg(frame: av.VideoFrame, width: int) -> bytes:
"""Re-encode one decoded frame as a JPEG, scaled to `width`."""
height = width * frame.height // frame.width
scaled = frame.reformat(width=width, height=height, format="yuvj420p")
Expand Down
20 changes: 15 additions & 5 deletions recipes/av/examples/clip-roundtrip/src/main.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
import threading

import flet as ft
from clip import clip_path, library_versions, probe, thumbnails, write_clip


def still(label, jpeg):
def still(label: str, jpeg: bytes) -> ft.Column:
"""One filmstrip cell: a decoded frame above the timestamp it was taken at."""
return ft.Column(
horizontal_alignment=ft.CrossAxisAlignment.CENTER,
Expand All @@ -14,7 +16,7 @@ def still(label, jpeg):
)


def row(label, value):
def row(label: str, value: str) -> ft.Row:
"""One line of the probe readout: label on the left, what was read on the right."""
return ft.Row(
alignment=ft.MainAxisAlignment.SPACE_BETWEEN,
Expand All @@ -23,8 +25,15 @@ def row(label, value):


def main(page: ft.Page):
busy = threading.Lock()

def run():
"""Lock the button, raise the spinner, and hand the work to a thread."""
"""Unless a run is in flight, disable the button, raise the spinner, and hand
the work to a thread."""
# Every run rewrites the same clip. Disabling the button is not enough on
# its own: a second tap already in flight lands before the patch does.
if not busy.acquire(blocking=False):
return
button.disabled = True
spinner.visible = True
page.update()
Expand All @@ -33,8 +42,8 @@ def run():
def compute():
"""Write the clip, probe it, extract stills, and update the page.

run_thread swallows exceptions and does not carry an automatic update
with it, so this catches its own failures and ends with page.update().
run_thread reports a failure only in the log and does not carry an automatic
update with it, so this catches its own failures and ends with page.update().
"""
try:
path = clip_path()
Expand All @@ -47,6 +56,7 @@ def compute():
button.disabled = False
spinner.visible = False
page.update()
busy.release()

page.appbar = ft.AppBar(title=ft.Text("clip roundtrip"), center_title=True)
page.add(
Expand Down
Loading
Loading