Skip to content

Add authenticated PUT synchronization for OPDS reading progression - #3936

Open
panaC wants to merge 14 commits into
feat/GET-opds-progressionfrom
feat/PUT-opds-progression
Open

panaC wants to merge 14 commits into
feat/GET-opds-progressionfrom
feat/PUT-opds-progression

Conversation

@panaC

@panaC panaC commented Oct 1, 2026 •

Copy link
Copy Markdown
Member
  • Convert local locators using the same Readium position weights and equal-resource fallback as GET.
  • Apply a five-second trailing debounce to meaningful navigation in the locked reader.
  • Read the latest locator from Redux when the debounce expires.
  • Recheck lock ownership and GET/dialog readiness before sending PUT.
  • Upload progression, the current timestamp, and device identity.
  • Cancel local debounce work when remote resume is accepted and suppress the matching programmatic navigation.
  • Cancel debounce tasks when the reader closes.
  • Log failed PUTs without queuing, retrying, or authentication replay.
  • Extend the test server, documentation, and reducer/saga coverage.

GET’s existing resume policy remains unchanged. HTTP credential handling stays in the networking layer.

@panaC panaC self-assigned this Oct 1, 2026
@panaC
panaC added this pull request to stack #3937 October 1, 2026 15:12
panaC added 2 commits October 6, 2026 16:43
Preserve PUT upload and reconciliation behavior while incorporating GET diagnostics, fixture response delays, request logs, and Readium-weighted remote resume. Share the weighted model between downloads and uploads to keep percentages consistent. Resolve documentation conflicts with Unix command examples.

Validation: 474 repository tests, 20 fixture server tests, 83 final focused tests, targeted lint, and main/reader Webpack builds passed.
Log the resume heuristic before suppressing matching local and remote positions. Include compared progression values, position equality, timestamp freshness, and local movement alongside the locators and dialog decision.
@panaC panaC changed the title Feat/put opds progession Add bidirectional OPDS reading progression synchronization Oct 6, 2026
@panaC panaC changed the title Add bidirectional OPDS reading progression synchronization Add authenticated PUT synchronization for OPDS reading progression Oct 6, 2026
@panaC

panaC commented Oct 6, 2026

Copy link
Copy Markdown
Member Author

Extend the existing OPDS progression GET support with authenticated PUT uploads. Local reading-position changes are synchronized remotely after initial retrieval and any resume dialog have been resolved.

panaC added 3 commits October 6, 2026 17:20
Restore the GET branch's locator comparison, timestamp-based resume policy, and diagnostics. Remove matching-position prompt suppression and its regression test while retaining PUT retrieval gating, dialog reconciliation, and weighted upload mapping.
Keep one pending update, debounce timer, and retry/authentication state per publication. Send only from the locked reader, replace pending state on ownership handoff, and ignore stale upload outcomes.

Remove forced-shutdown coordination and close-time upload draining. Cancel pending updates when the owning reader closes, retain lightweight per-reader GET/dialog state, and cover ownership handoff and remote-resume suppression with regression tests.
Replace the synchronization coordinator with minimal Redux reader state and a five-second trailing saga debounce. Read the latest locator at expiry, recheck reader ownership and GET/dialog readiness, and suppress remote-applied navigation.

Remove upload queues, retry state, authentication forwarding and replay, and timestamp rebasing. Log PUT failures without replay and cancel debounce work on close. Preserve the GET resume policy and add reducer/saga regression coverage.
@panaC

panaC commented Oct 7, 2026 •

Copy link
Copy Markdown
Member Author
flowchart TB
    subgraph GET["GET — Retrieve remote progression"]
        G1["Open reader<br/>Block uploads"] --> G2["Retrieve and validate remote progression"]
        G2 --> G3["Compare remote and local positions<br/>Offer resume when appropriate"]
        G3 --> G4["Resolve resume decision<br/>Keep local position or navigate to remote position"]
        G4 --> G5["Allow uploads<br/>Skip uploading accepted remote navigation"]
    end

    subgraph PUT["PUT — Upload local progression"]
        P1["Local reading progression changes"] --> P2["Verify reader ownership<br/>Restart 5-second debounce"]
        P2 --> P3["Check GET/resume is complete<br/>Reader still owns lock<br/>Publication is eligible"]
        P3 --> P4["Read latest progression<br/>Add current timestamp and device identity"]
        P4 --> P5["Send authenticated PUT"]
        P5 --> P6["Validate response<br/>Log failures without scheduled retry"]
        P7["Reader closes"] --> P8["Cancel pending debounce<br/>No final upload"]
    end

    G5 -. Upload permission .-> P3

    style GET fill:#eef5ff,stroke:#3973ac
    style PUT fill:#eef8ef,stroke:#42854a
Loading

@panaC

panaC commented Oct 7, 2026 •

Copy link
Copy Markdown
Member Author

The PUT feature uploads the latest reading progression after movement, once the initial GET/resume decision is complete.

  1. Initialize when the reader opens

    • Convert the saved local locator into a publication-wide progression.
    • Set ready = false while GET and any resume dialog are pending.
  2. Resolve the GET/resume flow

    • No remote position offered, or user cancels: set ready = true.
    • User accepts: cancel the pending upload, set ready = true, and store a temporary marker containing the mapped remote progression.
    • Navigate to the accepted remote position.
  3. On each locator update

    • Verify the event belongs to the current reader and publication.
    • Convert the locator into a progression float between 0 and 1. Stop if conversion fails.
    • Compare it against the previous progression and the accepted remote progression to skip uploading, then store the new progression.
    • If it matches the accepted remote progression: clear the value to skip, cancel the pending debounce, and stop without uploading.
    • Otherwise, if a previous progression exists, the progression changed, and this reader owns the publication lock: restart a 5-second trailing debounce.
  4. When the debounce expires

    • Check that the reader still owns the lock.
    • Check that GET/resume reconciliation is complete and no resume dialog is pending.
    • Check that the publication is an EPUB with an OPDS progression URL.
    • Read the latest locator from Redux and convert it into progression.
    • Retrieve the device identity.
    • Recheck the lock and reconciliation state before sending.
  5. Send the authenticated PUT

    • Use a 6-second request timeout.
    • Set Accept and Content-Type to application/opds-progression+json.
    • Send only:
    {
      "modified": "<current ISO timestamp>",
      "device": {
        "id": "<device URI>",
        "name": "<device name>"
      },
      "progression": 0.625
    }
  6. Handle the result

    • 200 or 201 with a valid progression response document: success.
    • HTTP error, invalid response, or network failure: log the failure.
    • No upload queue or scheduled retry; later movement can trigger another PUT.
  7. When the reader closes

    • Cancel the pending debounce without sending a final upload.

Algorithm flowchart

flowchart TD
    L["Locator update received"] --> M{"Current reader<br/>and publication match?"}
    M -- No --> STOP["Stop processing this update"]
    M -- Yes --> N["Convert locator to progression<br/>between 0 and 1"]
    N --> O{"Conversion succeeded?"}
    O -- No --> STOP
    O -- Yes --> P["Compare with previous progression<br/>and accepted remote progression to skip<br/>Store new progression"]
    P --> Q{"Matches accepted remote<br/>progression to skip?"}
    Q -- Yes --> R["Clear progression to skip<br/>Cancel pending debounce"]
    R --> STOP
    Q -- No --> S{"Previous progression exists,<br/>progression changed,<br/>and reader owns the lock?"}
    S -- No --> STOP
    S -- Yes --> T["Cancel pending debounce<br/>Start a new 5-second trailing debounce"]

    T --> U{"Locator change<br/>before expiry?"}
    U -- Yes --> L
    U -- No --> V{"Reader still owns lock,<br/>ready = true,<br/>and no resume dialog pending?"}
    V -- No --> SKIP["Skip upload"]
    V -- Yes --> W{"EPUB with an OPDS<br/>progression URL?"}
    W -- No --> SKIP
    W -- Yes --> X["Read latest locator from Redux<br/>Convert to progression"]
    X --> Y{"Valid progression?"}
    Y -- No --> SKIP
    Y -- Yes --> Z["Retrieve device ID and name"]
    Z --> AA{"Still locked, ready,<br/>and no resume dialog pending?"}
    AA -- No --> SKIP
    AA -- Yes --> AB["Build JSON payload"]
    AB --> AC["Authenticated PUT"]
    AC --> AD{"200 or 201 with<br/>valid response document?"}
    AD -- Yes --> AE["Upload complete"]
    AD -- No --> AF["Log failure<br/>No scheduled retry"]
    AE -. Later movement .-> L
    AF -. Later movement may retry .-> L
    SKIP -. Later movement .-> L

    CLOSE["Reader closes"] --> CANCEL["Cancel pending debounce<br/>No final upload"]
Loading

@panaC

panaC commented Oct 7, 2026

Copy link
Copy Markdown
Member Author

The main risks are:

  • Progress can remain unsynchronized. Closing cancels the debounce without a final upload. Failed requests have no scheduled retry. If the debounce expires while GET or the dialog is pending, that upload is skipped; reconciliation alone does not reschedule it.
  • Device clocks affect synchronization. Resume decisions compare timestamps, and PUT uses the current local time. Clock differences can produce unexpected resume prompts or 409 conflicts.
  • Concurrent devices can overwrite progress. GET and PUT are separate requests. Another device can update the server between them; there is no version-based conditional update.
  • Progression mapping loses precision. A single float cannot preserve an exact text location. Equal-resource fallback is approximate, and progression 1 maps to 0.95 within the last resource.
  • Remote navigation suppression depends on matching floats. A navigation result differing beyond the tolerance could be treated as local movement and uploaded back.
  • State can change during an upload. Lock ownership is checked before sending, but losing the lock or closing the reader afterward cannot undo a PUT already sent.
  • Local changes during GET can affect the resume decision. The change heuristic can treat locator metadata rewrites as activity and suppress an otherwise newer remote position.

The most visible consequence is lost synchronization when the reader closes quickly, a request fails, or movement occurs before reconciliation finishes.

@panaC

panaC commented Oct 7, 2026 •

Copy link
Copy Markdown
Member Author

Tests:

Start the server:

node projects/opds-progression/server.mjs

Add http://127.0.0.1:4873/opds/v2/catalog.json in Thorium, then download Accessible EPUB 3.

Inspect request counters and uploaded JSON at:

curl http://127.0.0.1:4873/__test/state

Basic upload and debounce

  • Open the EPUB and resolve any resume dialog.
  • Navigate to a different reading position, then stop for more than five seconds.
  • Verify progressionPutCount increases and lastPutStatus is 200 or 201.
  • Verify lastPutBody contains modified, device.id, device.name, and progression between 0 and 1, without title or references.
  • Navigate several times within five seconds. Verify one upload occurs after the final movement, containing the latest progression.

GET and resume reconciliation

curl --request PUT http://127.0.0.1:4873/__test/state \
  --header 'Content-Type: application/json' \
  --data '{"progression":0.75,"modified":"2040-01-02T03:04:05.000Z"}'
  • Set a newer remote position:
  • Reopen the EPUB. Verify the resume dialog appears.
  • Leave the dialog open for more than five seconds. Verify no PUT occurs.
  • Accept the remote position. Verify navigation occurs without uploading that accepted position back.
  • Repeat and cancel the dialog. Verify the local position is preserved.

Retrieval failure

curl --request PUT http://127.0.0.1:4873/__test/state \
  --header 'Content-Type: application/json' \
  --data '{"delayMs":8000}'
  • Configure a GET delay exceeding the six-second timeout:
  • Reopen the EPUB. Verify reading remains available despite the GET timeout.
  • After retrieval finishes, navigate and wait more than five seconds. Verify PUT is attempted.
  • Restore the delay with {"delayMs":0}.

PUT errors and recovery

  • Set {"locked":true} through /__test/state. Navigate and wait. Verify lastPutStatus is 403 and reading continues normally.
  • Wait without moving. Verify no scheduled retry occurs.
  • Set {"locked":false,"modified":"2000-01-01T00:00:00.000Z"}. Navigate again and wait. Verify a successful PUT.
  • Set a future remote timestamp, then navigate again. Verify 409, continued reading, and no scheduled retry.
  • Restore an older timestamp. Verify subsequent movement can upload successfully.

Reader ownership and closing

  • Open the same publication in two reader windows. Verify only the window owning the publication lock uploads progression.
  • Navigate, then close the reader before five seconds. Verify the pending upload is canceled and no final PUT occurs.
  • Navigate, wait for a successful PUT, then close and reopen. Verify GET retrieves the stored progression; the resume dialog remains subject to timestamp comparison.

Progression mapping

  • Test movement near the beginning, middle, and end. Verify uploaded progression follows the publication-wide position.
  • Set remote progression to 0, 0.5, and 1, with a newer timestamp, and accept each resume prompt. Verify navigation reaches the expected area; 1 uses the safe end position.
  • Open a publication without an OPDS progression link. Verify reading works without progression requests.

@panaC
panaC marked this pull request as ready for review October 7, 2026 15:06
@panaC

panaC commented Oct 7, 2026

Copy link
Copy Markdown
Member Author

The PUT implementation now uses a five-second trailing debounce per publication. Locator events from any reader window showing that publication reset the same timer; the latest event supplies the progression uploaded by PUT. Different publications debounce independently.

PUT no longer depends on reader locks or main-process session snapshots. It also does not wait for GET or the resume dialog to complete. A received position can still upload if its reader closes during the debounce.

@panaC

panaC commented Oct 7, 2026

Copy link
Copy Markdown
Member Author
flowchart TD
    A[Main receives setLocator] --> B{Renderer sender and valid window?}
    B -- No --> X[Ignore event]
    B -- Yes --> C{Window belongs to sender's publication?}
    C -- No --> X
    C -- Yes --> D{EPUB with an OPDS progression endpoint?}
    D -- No --> X
    D -- Yes --> E[Convert locator to publication progression]
    E --> F{Valid progression?}
    F -- No --> X
    F -- Yes --> G[Cancel pending debounce for this publication]
    G --> H[Start a new five-second timer with this progression]
    H --> I{Another valid event before expiry?}
    I -- Yes --> G
    I -- No --> J[Look up the current progression endpoint]
    J --> K{Endpoint still exists?}
    K -- No --> L[Clear pending task]
    K -- Yes --> M[Read device identity and locale]
    M --> N[PUT progression with current timestamp]
    N --> O{Successful?}
    O -- No --> P[Log failure]
    O -- Yes --> L
    P --> L
Loading

The algorithm is:

  1. Validate the event’s sender, window, and publication.

  2. Require an EPUB publication with a progression endpoint.

  3. Convert the event’s locator into a publication-wide progression.

  4. Cancel the publication’s pending upload timer.

  5. Start a new five-second timer carrying that progression.

  6. At expiry, look up the endpoint again and send:

    {
      "device": {
        "id": "<device identifier>",
        "name": "<device name>"
      },
      "modified": "<current ISO timestamp>",
      "progression": 0.42
    }
  7. Log failures and clear the pending task. A later locator event can schedule another upload.

All readers of one publication share the timer. Different publications have independent timers. There is no upload lock, GET/resume gate, session-snapshot dependency, or automatic retry. Closing a reader does not cancel an already scheduled upload.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant