Repository navigation
recipe: lz4 4.4.5 - #124
Merged
Merged
Conversation
Plain setuptools over the PyPI sdist, which vendors liblz4 (fast + HC codecs, frame layer, xxHash). No patches. PYLZ4_USE_SYSTEM_LZ4=0 keeps setup.py's pkg-config probe from swapping the vendored copy for an external liblz4 the wheel wouldn't ship. Tests cover the frame API with both checksums, every block mode, the chunked compressor/decompressor, lz4.frame.open on device storage, and decoding a frame written by the reference lz4 CLI. Adds the consumer README and a log-archive example app. Requested in flet-dev/flet#6908.
- Docs: HC caps at 12 in liblz4 (13-16 behave as 12); sharing a compressor/decompressor/file across threads can crash; lz4.frame.open reads are GIL-bound; three storage dirs; App size section; CI runs only two of six slices. - Example: lock the level buttons during a run (overlapping taps corrupted the shared file), write/read in 1 MiB chunks, HC 16 -> HC 12. - Tests: "fast" block mode with acceleration 8 (at 1 it equals "default"); the file test now writes in pieces.
Disabling the level buttons left a window: on the iOS simulator two taps reached Python 11 ms apart, before the disabled patch landed, and the overlapping runs still corrupted the shared file. The handler now takes a non-blocking lock. Docs carry the same advice, and the example's timing claims now match device measurements (iOS sim + arm64-v8a emulator).
- local-recipe-testing: run the consumer example on CI's wheels (no local forge env); Xcode 27 rejects flet's iOS deployment target 13.0 and how to work around it; emulator ANDROID_SDK_ROOT + missing system image. - forge-ci: gh job logs need --allow-escape-sequences; confirm per-leg Python after mobile_test_pythons=ALL. - new-mobile-recipe: truncated support tarballs are reused silently; audit the consumer README/example before the PR (upstream constants, run_thread races, thread-shared native state, streaming claims, flake8).
The clip-roundtrip example disabled its button in the click handler, but a second tap already in flight reaches Python before the disabled patch lands: on an iOS simulator two taps arrived 27 ms apart and started two overlapping runs rewriting the same clip, with the button and spinner reset while the second was still going. The handler now takes a non-blocking lock (one run per double tap, verified on the iOS simulator and an arm64-v8a emulator). README threading advice extended to match.
- README: lz4.frame.open readers do scale on Python 3.14 (128 KiB DEFAULT_BUFFER_SIZE), only not on 3.12/3.13; a truncated file read through it raises EOFError; lz4.block wording; link API names; drop a dangling intro sentence. - Examples (lz4 + av): restore main() docstrings, describe the in-flight guard and run_thread's log-only failures accurately, black/docformatter. - av README: clearer lock advice; run_thread reports failures in the log.
- local-recipe-testing: the emulator's ANDROID_SDK_ROOT messages both mean a missing system image, and the SDK's own sdkmanager must install it (Homebrew's uses another root); per-arch staging does read PIP_FIND_LINKS; 10b covers flet 0.86.5-1.0.3 and passes SP_NATIVE_SET; CI-wheels shortcut downloads to /tmp, no padding. - forge-ci: the job-log snippet carries --allow-escape-sequences; the per-leg check greps pip's joined (wrapped) install line. - new-mobile-recipe: truncated-tarball trap moved after the NDK fix and corrected (setup.sh also skips re-extraction once support/ exists).
Annotate all parameters in the lz4 and av examples and the lz4 tests (tmp_path: Path; domain functions also get return types). Lambdas are the only exemption. The rule joins the new-mobile-recipe test conventions. av's clip.py also gets the docformatter rewrap it already failed on main.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a recipe for lz4 4.4.5 — Python bindings to LZ4, the lossless compressor built for speed rather than ratio. Requested in flet#6908.
On a phone the cost of compression is CPU time and battery, which is exactly what LZ4 trades ratio away to save: its files are larger than zlib's, but it compresses several times faster and decompresses two to three times faster.
Recipe shape
Plain setuptools over the PyPI sdist, no patches. The sdist vendors liblz4 (
lz4libs/: the fast and HC codecs, the frame layer and xxHash) and each of the three extensions statically compiles the parts it uses, exactly as PyPI's desktop wheels do — soflet runand the device run the same liblz4.One
script_envsetting:PYLZ4_USE_SYSTEM_LZ4: "0".setup.pyotherwise links any liblz4 >= 1.7.5 thatpkg-configcan see. Forge'sPKG_CONFIG_LIBDIRhides the build host's today, but a liblz4 appearing in the cross env later would silently turn the self-contained wheel into one with a runtime dependency nothing ships.Validation
mobile_test_pythons=ALL, twice (fork runs 36916076140 and 37141952995, the latter on this recipe and test tree). On-device5 passed/EXIT 0on every leg, each confirmed to run its own interpreter.Machineper ABI, every AndroidLOADsegment aligned0x4000,DT_NEEDEDlimited to bionic andlibpython(noliblz4.so), iOSLC_BUILD_VERSIONplatform 2 on device / 7 on the simulators,lz4.__version__4.4.5 fromPKG-INFO(setuptools_scm does not pick up mobile-forge's own git).log-archiveexample builds and runs on the iOS Simulator and on an arm64-v8a Android emulator — the slice real phones use, which CI never executes. Files it writes decode byte-identical with the referencelz4CLI on both;test_decodes_reference_cli_framecovers the other direction on device.COMPRESSIONLEVEL_MAX = 16; sharing oneLZ4FrameCompressorbetween threads can crash the interpreter; the test's"fast"block mode was byte-identical to"default"until given an acceleration; and the example raced itself on a double tap. A second review of everything after that audit corrected one more claim: readers throughlz4.frame.opendo run in parallel on Python 3.14 (its read buffer grew from 8 KiB to 128 KiB), just not on 3.12/3.13.The double-tap race (also fixed in the av example)
Disabling a button in its click handler does not stop a second tap that is already in flight: on the iOS Simulator two taps reached Python 3–48 ms apart, before the
disabledpatch reached Flutter, and both started apage.run_threadworker. In the lz4 example the two runs share one file, and the first run's read-back failed every time. Both handlers now take a non-blockingthreading.Lockand drop a tap that finds it held; verified on device that a double tap produces exactly one run.recipes/av/examples/clip-roundtrip(#122) had the same shape. There the overlap showed up as the button and spinner resetting while a second encode was still writing the clip — no corrupt output was observed — so it gets the same lock, and the av README's threading section gets the same advice.CI on this PR rebuilds av as well, only because its example and README changed: the changed-recipe filter covers all of
recipes/**. The av recipe and its published wheels are untouched.Changes
recipes/lz4/—meta.yaml, 5 on-device tests,README.md, and thelog-archiveexample.recipes/av/— the example's tap guard and one paragraph in each README..claude/skills/— running the consumer example on CI's wheels with no local forge env; Xcode 27 rejecting flet's iOS deployment target (13.0) and the local workaround; the emulator's missing-system-image recovery;ghjob logs needing--allow-escape-sequences; truncated support tarballs being reused silently bysetup.sh; and an "audit the consumer README before the PR" checklist from the findings above.Consumer notes
lz4 gives a Flet app fast lossless compression for caches, logs and uploads; what it writes is the standard LZ4 frame format, so a server can decode it without Python. The things that are different on a phone — where files belong in Flet's storage, which calls really run in parallel under
run_thread, why a compressor must not be shared between threads, and the HC level ceiling — are in the recipe README, with a runnable example.