You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Islands: find_packages never sees the notebook's PEP 723 block, so non-bundled packages can't be installed — proposal: carry dependencies in the islands JSON payload #10806
A notebook rendered with MarimoIslandGenerator cannot install pure-Python PyPI packages that Pyodide doesn't bundle. Minimal repro (marimo 0.24.1, @marimo-team/islands@0.24.1):
@app.celldef_(mo):
importcowsay# pure Python, not in the Pyodide distributionmo.md(f"```\n{cowsay.get_output_string('cow', 'moo')}\n```")
Render with MarimoIslandGenerator.from_file(...) + render_head() / render_body(), open the page: the cell fails with ModuleNotFoundError: No module named 'cowsay'. The same notebook works under marimo export html-wasm, and works in islands if you hand-write an await micropip.install(["cowsay"]) cell (#8074).
Root cause (verified against 0.24.1)
The machinery to install packages already runs for islands — it just never gets a dependency list:
The islands worker is ReadonlyWasmController extends DefaultWasmController (frontend/src/core/islands/worker/controller.ts). DefaultWasmController.startSession (frontend/src/core/wasm/worker/bootstrap.ts) reads the notebook file back from the Pyodide FS, calls session.find_packages(code), and loadNotebookDeps then pyodide.loadPackagesFromImports(code) + micropip.install(missing).
find_packages (marimo/_pyodide/pyodide_session.py) only returns dependencies from a PEP 723 # /// script block. Import scanning was deliberately removed ("We don't return 'unknown' packages from the imports … without user consent"), so with no block it returns [].
The notebook file the islands worker reads is not the source notebook. It is synthesized in the browser by createMarimoFile (frontend/src/core/islands/parse.ts) from the per-cell code — import marimo / app = marimo.App() / one @app.cell per island, every body line indented four spaces. There is no header, and nothing in render_head(), render_body(), or the new JSON payload (feat: hydrate islands from JSON payloads #9987) carries the source's # /// script block over.
The PEP 723 regex (marimo/_utils/scripts.py, REGEX) anchors # /// at column 0, so the block can't be smuggled through inside a cell body either.
Net effect: for islands, find_packages is always []. loadPackagesFromImports still auto-loads Pyodide-bundled packages (numpy, pandas, scipy, matplotlib, …), which is why this only bites on PyPI-only pure-Python deps — silently, and only at runtime in the browser.
Proposal
Carry the dependency list through the payload and let the existing install path do the rest. Two small changes, no worker changes:
Python — MarimoIslandGenerator records the source notebook's PEP 723 dependencies (PyProjectReader.from_script(source).dependencies, filtered with filter_requirements_for_emscripten exactly as find_packages does today) when created via from_file / _from_ir, and render_payload() emits them as an optional top-level field:
isMarimoIslandPayload only checks the required keys, so an optional field is backwards compatible with schema 1; a schema bump is fine too if preferred.
Frontend — createMarimoFile prepends a synthesized block when the payload has dependencies:
That is the whole fix: the file lands in the Pyodide FS as before, find_packages now finds the block, loadNotebookDeps installs the missing packages before initSession, and the "consent" property of find_packages is preserved because the list comes from the author's explicit metadata, not from import scanning.
Rough scope: ~30 LOC plus tests in tests/_islands/test_island_generator.py and frontend/src/core/islands/__tests__/. Happy to send the PR if this direction is acceptable.
Context
Static exporters built on islands (marimo-book, and I believe mkdocs-marimo / quarto-marimo) hit this on any notebook with a PyPI-only dep. marimo-book currently works around it by emitting its own copy of the feat: hydrate islands from JSON payloads #9987 payload with an extra anchor-less cell that await micropip.install([...])s the list plus a sentinel variable every other cell references for ordering. It works on 0.24.1, but it is a hack around the exporter contract that the payload was introduced to simplify.
Problem
A notebook rendered with
MarimoIslandGeneratorcannot install pure-Python PyPI packages that Pyodide doesn't bundle. Minimal repro (marimo 0.24.1,@marimo-team/islands@0.24.1):Render with
MarimoIslandGenerator.from_file(...)+render_head()/render_body(), open the page: the cell fails withModuleNotFoundError: No module named 'cowsay'. The same notebook works undermarimo export html-wasm, and works in islands if you hand-write anawait micropip.install(["cowsay"])cell (#8074).Root cause (verified against 0.24.1)
The machinery to install packages already runs for islands — it just never gets a dependency list:
ReadonlyWasmController extends DefaultWasmController(frontend/src/core/islands/worker/controller.ts).DefaultWasmController.startSession(frontend/src/core/wasm/worker/bootstrap.ts) reads the notebook file back from the Pyodide FS, callssession.find_packages(code), andloadNotebookDepsthenpyodide.loadPackagesFromImports(code)+micropip.install(missing).find_packages(marimo/_pyodide/pyodide_session.py) only returns dependencies from a PEP 723# /// scriptblock. Import scanning was deliberately removed ("We don't return 'unknown' packages from the imports … without user consent"), so with no block it returns[].createMarimoFile(frontend/src/core/islands/parse.ts) from the per-cell code —import marimo/app = marimo.App()/ one@app.cellper island, every body line indented four spaces. There is no header, and nothing inrender_head(),render_body(), or the new JSON payload (feat: hydrate islands from JSON payloads #9987) carries the source's# /// scriptblock over.marimo/_utils/scripts.py,REGEX) anchors# ///at column 0, so the block can't be smuggled through inside a cell body either.Net effect: for islands,
find_packagesis always[].loadPackagesFromImportsstill auto-loads Pyodide-bundled packages (numpy, pandas, scipy, matplotlib, …), which is why this only bites on PyPI-only pure-Python deps — silently, and only at runtime in the browser.Proposal
Carry the dependency list through the payload and let the existing install path do the rest. Two small changes, no worker changes:
Python —
MarimoIslandGeneratorrecords the source notebook's PEP 723 dependencies (PyProjectReader.from_script(source).dependencies, filtered withfilter_requirements_for_emscriptenexactly asfind_packagesdoes today) when created viafrom_file/_from_ir, andrender_payload()emits them as an optional top-level field:{"schemaVersion": 1, "appId": "main", "dependencies": ["cowsay"], "cells": [...]}isMarimoIslandPayloadonly checks the required keys, so an optional field is backwards compatible with schema 1; a schema bump is fine too if preferred.Frontend —
createMarimoFileprepends a synthesized block when the payload has dependencies:That is the whole fix: the file lands in the Pyodide FS as before,
find_packagesnow finds the block,loadNotebookDepsinstalls the missing packages beforeinitSession, and the "consent" property offind_packagesis preserved because the list comes from the author's explicit metadata, not from import scanning.Rough scope: ~30 LOC plus tests in
tests/_islands/test_island_generator.pyandfrontend/src/core/islands/__tests__/. Happy to send the PR if this direction is acceptable.Context
await micropip.install([...])s the list plus a sentinel variable every other cell references for ordering. It works on 0.24.1, but it is a hack around the exporter contract that the payload was introduced to simplify.marimo export html-wasm) #9778 in June for the same gap, proposing that the generator emit<marimo-code>and the islands init re-use the html-wasm bootstrap. I've closed it in favour of this issue: the JSON payload (feat: hydrate islands from JSON payloads #9987) didn't exist then, and now that it does it's a far smaller seam — the worker already has the install path, so only the dependency list needs to travel.