Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1 +1,6 @@
*.sh text eol=lf

# Bundled third-party files are kept byte-for-byte as upstream ships them:
# no line-ending conversion on any OS, so app/ui/vendor/VENDOR.json's SHA-256
# values match every checkout (tests/test_vendor_manifest.py).
app/ui/vendor/** -text
28 changes: 28 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,34 @@ jobs:
if: runner.os == 'Linux'
run: node tests/desktop_download_ui.test.js

- name: Test save folder setting UI
if: runner.os == 'Linux'
run: node tests/save_folder_ui.test.js

- name: Test preview links and release notes UI
if: runner.os == 'Linux'
run: node tests/preview_links_ui.test.js

- name: Test theme toggle UI
if: runner.os == 'Linux'
run: node tests/theme_toggle_ui.test.js

- name: Test Settings drawer UI
if: runner.os == 'Linux'
run: node tests/settings_drawer_ui.test.js

- name: Test layout width setting UI
if: runner.os == 'Linux'
run: node tests/layout_width_ui.test.js

- name: Test preview text size UI
if: runner.os == 'Linux'
run: node tests/text_size_ui.test.js

- name: Test math formulas UI
if: runner.os == 'Linux'
run: node tests/math_render_ui.test.js

# installer.iss is otherwise only compiled by release.yml, after the tag is
# pushed, so a mistake in its Pascal [Code] cost a version number. Compile
# it here against a stand-in bundle (only the script is being checked).
Expand Down
33 changes: 24 additions & 9 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ the real `markitdown` public API (`MarkItDown.convert_local()` / `convert_uri()`

Core UX contract (do not break when changing things): **drop a file,
paste a URL, or click to browse → conversion starts immediately → the
resulting `.md` is auto-saved to `~/Downloads` with no export step.**
resulting `.md` is auto-saved to the save folder (`~/Downloads` unless
another folder is chosen in Settings) with no export step.**

## Commands

Expand All @@ -41,7 +42,7 @@ python -c "import urllib.request, time; time.sleep(2); res = urllib.request.urlo

# Test suite (CI runs exactly this) and the UI unit tests (Node, no npm)
python -m pytest tests/ -q
node tests/auto_engine_ui.test.js && node tests/preview_word_count.test.js && node tests/glm_ocr_ui.test.js && node tests/desktop_download_ui.test.js
node tests/auto_engine_ui.test.js && node tests/preview_word_count.test.js && node tests/glm_ocr_ui.test.js && node tests/desktop_download_ui.test.js && node tests/save_folder_ui.test.js && node tests/preview_links_ui.test.js && node tests/theme_toggle_ui.test.js && node tests/settings_drawer_ui.test.js && node tests/layout_width_ui.test.js && node tests/text_size_ui.test.js && node tests/math_render_ui.test.js

# Lint (also run in CI)
pip install ruff
Expand All @@ -59,19 +60,20 @@ item = QueueItem(source="/path/to/file.txt", kind=SourceKind.FILE, display_name=
print(convert_item(item, ConversionOptions()))
```

Any manual local testing that writes into the real `~/Downloads` (e.g. via
`auto_save_markdown()`) must clean up its own test files afterward —
nothing test-related should be left in the user's actual Downloads folder.
Any manual local testing that writes into the real `~/Downloads` or the chosen
save folder (e.g. via `auto_save_markdown()`) must clean up its own test files
afterward — nothing test-related should be left in the user's actual folders.

## Architecture

**Unified Domain-Driven Architecture (`app/`)**:

- `app/core/` — Core conversion logic & engine adapters:
- `converter.py` — Maps options to real `MarkItDown(...)` constructor kwargs. Dispatches `convert_local()` or `convert_uri()`. `auto_save_markdown()` / `get_downloads_dir()` / `unique_download_path()` implement collision-safe Downloads auto-save.
- `converter.py` — Maps options to real `MarkItDown(...)` constructor kwargs. Dispatches `convert_local()` or `convert_uri()`. `auto_save_markdown()` / `get_downloads_dir()` / `unique_download_path()` implement collision-safe auto-save into a given folder (Downloads by default). The file is created with `O_EXCL`, so a file or link that appears at the chosen name is never written through; the next free name is used, and a failed write removes its own partial file.
- `queue_model.py` — `EngineKind` enum (`markitdown`, `docling`, `markit`, `glm_ocr`, and `auto`, a routing mode that always resolves to one of the concrete engines), `QueueItem` dataclass.
- `engines/` — Engine adapters (`docling_engine.py`, `markit_engine.py`).
- `pdf_probe.py` — One cheap pdfium pass over a PDF: per-page text layer, image coverage, and the scan-likely classifier that tells scanned pages from cover photos. Imports `pypdfium2` lazily, never raises (returns `None`), serialises pdfium with a lock.
- `save_location.py` — The save folder. Stored as `save_dir` in `settings.json` (`""` = Downloads) and set only by the desktop folder dialog (`choose_save_folder` in `runner.py`); it is deliberately absent from `SettingsPayload`, so no HTTP request can set it. `save_markdown()` writes there and falls back to Downloads with a sentence saying why (missing folder: never re-created; folder or a parent replaced by a link or junction since it was chosen: not followed, detected because the stored path no longer resolves to itself; write error: `OSError`/`ValueError`); the choice is kept. A window that attached to an already-running InkDoc refuses to change it, since that process does the saving. `folder_state()` feeds `GET /settings` and the Download dialog's starting folder.
- `quality_check.py` — Missing-text check: compares a PDF conversion's Markdown with the PDF's own text layer and builds the user-facing warning and suggestion. Read-only; never fails a conversion.
- `auto_engine.py` — The Auto engine: pure routing (`decide`, `should_escalate`, `pick_result`, `glm_time_gate`, `AUTO_ROUTES`), OCR availability ("only what is on disk"; GLM-OCR first, then Docling), and the async `run_auto`, which takes each engine's lane *before* using a thread and may re-convert once with the OCR engine. GLM-OCR is used only for files estimated to finish within `GLM_OCR_TIME_LIMIT_S` (5 min); otherwise Docling.
- `jobs.py` — In-memory job registry behind `/convert/progress/{job_id}` and `/convert/cancel/{job_id}` (phase labels and cooperative cancel; engine-neutral).
Expand All @@ -82,19 +84,32 @@ nothing test-related should be left in the user's actual Downloads folder.
- `app/server/` — Embedded REST API backend:
- `server.py` — FastAPI REST API endpoints (`/convert/file`, `/convert/url`, `/convert/batch`, `/convert/progress/{job_id}`, `/convert/cancel/{job_id}`, `/health`, `/extensions`), dual-mounts `/static` and `/ui`. Lanes: Docling (1), GLM-OCR (1), default (2) and a probe lane (1) for pdfium work, all acquired in async code before `asyncio.to_thread`. `/engines/{name}/…` dispatches `glm_ocr` to `GlmOcrManager`; `/convert/estimate` gives GLM-OCR's page count and time estimate. Explicit engines convert → auto-save → check; Auto probes → converts → checks → (maybe re-converts) → auto-saves once. Response metadata is built in one place, `_conversion_payload`.
- `app/ui/` — **The single shared web & desktop UI**:
- `index.html` — Application DOM structure, dropzone, engine pills, live preview pane, settings popover.
- `index.html` — Application DOM structure, dropzone, engine pills, live preview pane, Settings drawer (General / Engines / Add-ons / Updates / About; it sits below the header so the title bar stays usable, and every engine prompt opens it on Engines).
- `style.css` — Inkbench design system: obsidian dark & light themes, glowing engine selector pills, responsive layout.
- `app.js` — Client-side logic: drag-and-drop, API conversion, live markdown preview (`marked.js`), auto-save toggle, engine switching.
- `app.js` — Client-side logic: drag-and-drop, API conversion, live markdown preview (`marked.js`), auto-save toggle, engine switching. Every preview parse goes through `markdownToHtml`: with `render_math` off, or no `$`, `\(` or `\[` in the document, it is the global `marked.parse`, unchanged; otherwise a separate `marked.Marked` instance with the math rules. `typesetMath` lazy-loads KaTeX and draws formulas in slices.
- `vendor/` — Bundled third-party JS/CSS/fonts (marked, DOMPurify, highlight.js, KaTeX). Never fetched at runtime; every file pinned by SHA-256 in `vendor/VENDOR.json` (`tests/test_vendor_manifest.py`); never line-ending converted (`.gitattributes`).
- **Add-ons** (Settings section): built-in add-ons are bundled code switched by a setting (math formulas → `render_math`). Downloadable add-ons may only ever be data (models, fonts, dictionaries) through the signed `AddonManager` catalogue, never JavaScript that runs in the page. The existing `addon*` element ids, `/addons/*` routes and `AddonManager` belong to the Docling recognition model, not to this section.
- `logo.svg`, `favicon.svg` — Brand assets.
- `app/desktop/runner.py` — Desktop application runner:
- Finds an available local loopback port (`13118` or ephemeral).
- Starts FastAPI in a background daemon thread (`uvicorn.Server`).
- Opens a native desktop window via `pywebview` (Microsoft Edge WebView2 on Windows) pointing to `http://127.0.0.1:<port>/InkDoc`.
- Gives the page nine functions as `pywebview.api.*` through `window.expose(*_page_functions(...))`: title-bar controls, `open_external_link`, `save_markdown` (the Download button's Save dialog), and `choose_save_folder` / `reset_save_folder` (the only way the save folder is set). Never pass an object as `js_api`: pywebview looks up `js_api` attributes by any name the page sends, underscore-named ones included, while exposed functions are found by exact name only. `tests/test_desktop_save.py` checks the nine names match `app.js` and that nothing else is reachable. Methods that write files or settings also refuse any page not served by InkDoc itself.
- In the desktop app, links inside converted documents and release notes never load in the window (`handleContentLinkClick` in `app.js`): web links open in the system browser, `#` links scroll, everything else is ignored.
- Handles graceful shutdown when the window is closed.
**Threading and Execution Model**:
- Conversions never block the main loop. In desktop mode, FastAPI runs in a background daemon thread (`uvicorn.Server`) while `pywebview` runs on the main GUI thread.
- In the browser/UI layer (`app/ui/app.js`), document conversions are dispatched via asynchronous `fetch()` requests to `/convert/file` or `/convert/url`. Each queue item updates reactively without freezing the user interface.
- Files are collision-safely auto-saved to `~/Downloads` by the backend (`auto_save_markdown()`) without requiring any manual export step.
- Files are collision-safely auto-saved to the save folder (`~/Downloads` by default) by the backend (`save_location.save_markdown()`) without requiring any manual export step.

## Known limitations (security)

Accepted for now, each needing write access to the folder involved or a local process:
- The save folder is checked for links a moment before the file is created, and only the file name itself is protected from links at creation (`O_EXCL`); a parent folder swapped in between is still followed.
- The default Downloads folder is checked with `is_symlink()` only, which does not detect Windows junctions.
- After a failed write, the partial file is removed by name.
- `unique_download_path`'s containment check compares path prefixes without a separator (file names are sanitised, so it cannot be escaped today).
- In attach mode the window trusts any loopback server on the port whose `/health` looks like InkDoc's.

## CI/CD (all in `.github/workflows/`)

Expand Down
20 changes: 14 additions & 6 deletions API.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ Convert an uploaded document to Markdown.
- `'markit'`: Broad-format specialist (EPUB, YAML, Jupyter Notebooks).
- `'glm_ocr'` (also `'glm-ocr'`): the downloadable [GLM-OCR](#glm-ocr) model, for PDFs and images only. `409 engine_not_installed` until it is downloaded in Settings.
- `'auto'`: InkDoc picks the engine per file (see [Auto engine](#auto-engine)) and reports its choice in `auto`.
- `save_to_downloads` *(bool, optional, default: `false`)*: Automatically save `.md` output directly to `~/Downloads` with collision-safe naming. The file is named after the uploaded file (`report.pdf` → `report.md`).
- `save_to_downloads` *(bool, optional, default: `false`)*: Automatically save `.md` output to the save folder with collision-safe naming. The save folder is `~/Downloads` unless another folder was chosen in the desktop app's Settings; a request cannot choose it. If the chosen folder is missing or can't be written to, the file is saved to `~/Downloads` and `save_fallback` says why. The file is named after the uploaded file (`report.pdf` → `report.md`).
- `response_format` *(string, optional, `'text'` | `'json'`, default: `'text'`)*.
- `enable_plugins` *(bool, optional, default: `false`)*: Enable MarkItDown plugins.
- `keep_data_uris` *(bool, optional, default: `false`)*: Retain base64 data URIs for images.
Expand Down Expand Up @@ -84,9 +84,10 @@ curl -F "file=@dataset.xlsx" "http://localhost:13118/convert/file?response_forma
"Page 1 is a full-page image; any text inside it wasn't extracted."],
"engine": "docling", "output_tokens": 5120
},
"warnings": [], // notes that are not failures (GLM-OCR: a page cut short, GPU fallback)
"warnings": [], // notes that are not failures (GLM-OCR: a page cut short, GPU fallback, save_fallback)
"markdown": "…",
"saved_to_downloads": "C:\\Users\\you\\Downloads\\report.md"
"saved_to_downloads": "C:\\Users\\you\\Downloads\\report.md", // where the file was saved (any save folder), or null
"save_fallback": null // e.g. "Saved to Downloads because the save folder D:\\Notes was not found."
}
```

Expand Down Expand Up @@ -133,7 +134,7 @@ Convert multiple files in a single HTTP request.
- `check_quality` *(bool, optional)*
- `job_id` *(string, optional)*: each file reports progress under `<job_id>-<index>` (0-based).

Each entry in `results` carries the same `engine_*`, `fallback*`, `auto` and `quality` fields as a single conversion.
Each entry in `results` carries the same `engine_*`, `fallback*`, `auto`, `quality` and `save_fallback` fields as a single conversion.

**cURL Example**:
```bash
Expand Down Expand Up @@ -278,10 +279,16 @@ Query or update application engine preferences.
{
"fallback_to_markitdown": true,
"quality_check_enabled": true,
"glm_ocr_download_source": "auto"
"glm_ocr_download_source": "auto",
"layout_width": "standard",
"preview_text_size": 100,
"render_math": true
}
```
`quality_check_enabled` (default `true`) turns the missing-text check on or off for requests that do not pass `check_quality`. `glm_ocr_download_source` is `auto` (official source first, InkDoc mirror as fallback) or `mirror_only`.
`quality_check_enabled` (default `true`) turns the missing-text check on or off for requests that do not pass `check_quality`. `glm_ocr_download_source` is `auto` (official source first, InkDoc mirror as fallback) or `mirror_only`. `layout_width` (default `standard`) is how much of a wide window the UI uses: `standard` (up to 1400px), `wide` (up to 1920px) or `full`; any other value is refused with `422`. The UI applies it from the next page load. `preview_text_size` (default `100`) is the document preview's text size in percent: `85`, `90`, `100`, `110`, `120`, `135` or `150`; anything else is refused with `422`. `render_math` (default `true`) draws LaTeX math in the UI preview with the bundled KaTeX; it never changes conversion output, saved files or API responses.
* **Save folder** (`GET` only): the response includes `save_folder`, for example
`{"path": "C:\\Users\\you\\Downloads", "chosen": "D:\\Notes", "custom": true, "problem": "This folder was not found. Files are saved to Downloads until it is back."}`.
`path` is where auto-save writes now; `chosen` is the folder picked in Settings (`""` for Downloads). The save folder can only be changed in the desktop app's folder dialog: `POST /settings` ignores `save_dir`, and no other request can set it.

---

Expand Down Expand Up @@ -354,6 +361,7 @@ Conversion responses include engine diagnostic headers:
- `X-Quality-Warning`: `true` or `false` when a PDF was checked for missing text.
- `X-Quality-Missing-Pct`: Estimated share of the PDF's text missing from the output, when measured.
- `X-Engine-Warnings`: Notes that are not failures (`warnings` in JSON), joined with ` | `.
- `X-Save-Fallback`: Present when auto-save could not use the chosen save folder and saved to `~/Downloads` instead (`save_fallback` in JSON). Like the other text headers it is limited to 200 characters and non-Latin-1 characters become `?`; use `response_format=json` for the exact text.

Reasons are reduced to one line of at most 200 characters.

Expand Down
Loading
Loading