Skip to content

Islands: find_packages never sees the notebook's PEP 723 block, so non-bundled packages can't be installed — proposal: carry dependencies in the islands JSON payload #10806

Description

@ljchang

Problem

A notebook rendered with MarimoIslandGenerator cannot install pure-Python PyPI packages that Pyodide doesn't bundle. Minimal repro (marimo 0.24.1, @marimo-team/islands@0.24.1):

@app.cell
def _(mo):
    import cowsay  # pure Python, not in the Pyodide distribution
    mo.md(f"```\n{cowsay.get_output_string('cow', 'moo')}\n```")

Render with MarimoIslandGenerator.from_file(...) + render_head() / render_body(), open the page: the cell fails with ModuleNotFoundError: No module named 'cowsay'. The same notebook works under marimo export html-wasm, and works in islands if you hand-write an await micropip.install(["cowsay"]) cell (#8074).

Root cause (verified against 0.24.1)

The machinery to install packages already runs for islands — it just never gets a dependency list:

  1. The islands worker is ReadonlyWasmController extends DefaultWasmController (frontend/src/core/islands/worker/controller.ts). DefaultWasmController.startSession (frontend/src/core/wasm/worker/bootstrap.ts) reads the notebook file back from the Pyodide FS, calls session.find_packages(code), and loadNotebookDeps then pyodide.loadPackagesFromImports(code) + micropip.install(missing).
  2. find_packages (marimo/_pyodide/pyodide_session.py) only returns dependencies from a PEP 723 # /// script block. Import scanning was deliberately removed ("We don't return 'unknown' packages from the imports … without user consent"), so with no block it returns [].
  3. The notebook file the islands worker reads is not the source notebook. It is synthesized in the browser by createMarimoFile (frontend/src/core/islands/parse.ts) from the per-cell code — import marimo / app = marimo.App() / one @app.cell per island, every body line indented four spaces. There is no header, and nothing in render_head(), render_body(), or the new JSON payload (feat: hydrate islands from JSON payloads #9987) carries the source's # /// script block over.
  4. The PEP 723 regex (marimo/_utils/scripts.py, REGEX) anchors # /// at column 0, so the block can't be smuggled through inside a cell body either.

Net effect: for islands, find_packages is always []. loadPackagesFromImports still auto-loads Pyodide-bundled packages (numpy, pandas, scipy, matplotlib, …), which is why this only bites on PyPI-only pure-Python deps — silently, and only at runtime in the browser.

Proposal

Carry the dependency list through the payload and let the existing install path do the rest. Two small changes, no worker changes:

Python — MarimoIslandGenerator records the source notebook's PEP 723 dependencies (PyProjectReader.from_script(source).dependencies, filtered with filter_requirements_for_emscripten exactly as find_packages does today) when created via from_file / _from_ir, and render_payload() emits them as an optional top-level field:

{"schemaVersion": 1, "appId": "main", "dependencies": ["cowsay"], "cells": [...]}

isMarimoIslandPayload only checks the required keys, so an optional field is backwards compatible with schema 1; a schema bump is fine too if preferred.

Frontend — createMarimoFile prepends a synthesized block when the payload has dependencies:

# /// script
# dependencies = ["cowsay"]
# ///
import marimo
app = marimo.App()
...

That is the whole fix: the file lands in the Pyodide FS as before, find_packages now finds the block, loadNotebookDeps installs the missing packages before initSession, and the "consent" property of find_packages is preserved because the list comes from the author's explicit metadata, not from import scanning.

Rough scope: ~30 LOC plus tests in tests/_islands/test_island_generator.py and frontend/src/core/islands/__tests__/. Happy to send the PR if this direction is acceptable.

Context

  • Static exporters built on islands (marimo-book, and I believe mkdocs-marimo / quarto-marimo) hit this on any notebook with a PyPI-only dep. marimo-book currently works around it by emitting its own copy of the feat: hydrate islands from JSON payloads #9987 payload with an extra anchor-less cell that await micropip.install([...])s the list plus a sentinel variable every other cell references for ordering. It works on 0.24.1, but it is a hack around the exporter contract that the payload was introduced to simplify.
  • I filed MarimoIslandGenerator should auto-install PEP 723 deps via micropip (parity with marimo export html-wasm) #9778 in June for the same gap, proposing that the generator emit <marimo-code> and the islands init re-use the html-wasm bootstrap. I've closed it in favour of this issue: the JSON payload (feat: hydrate islands from JSON payloads #9987) didn't exist then, and now that it does it's a far smaller seam — the worker already has the install path, so only the dependency list needs to travel.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions