Skip to content

test(web): pace dashboard e2e under the API rate limit and pin the sign-in org - #2881

Merged
Windscar merged 5 commits into
mainfrom
claude/session-isolation-flake
Nov 13, 2026
Merged

Windscar merged 5 commits into
mainfrom
claude/session-isolation-flake

Conversation

@Windscar

Copy link
Copy Markdown
Contributor

Fixes the dashboard e2e flake on main (session-isolation.e2e.spec.ts), which has two separate causes. Test harness only: no API or product change.

  • Rate limit: every spec signs in as one Clerk user, and each page load makes about nine sdp-api calls, so the suite runs at CLERK_USER_MAX_REQUESTS (600/min). Whichever test crosses it gets a 429 and an error page.
  • Pacing reads the API's own counters from the run's Redis, before each test and before fixture provisioning, and waits only until 150 requests are free (PLAYWRIGHT_RATE_LIMIT_HEADROOM). Specs import test from playwright/support/test.ts.
  • Instrumentation: each check is written to test-results/rate-budget.jsonl, which CI uploads, so the headroom can be tuned from real runs.
  • Wrong org after sign-in: a fresh sign-in on the shared test user could open another run's fixture org, which also failed later specs (theme.e2e 401s). Both sign-ins now pin the org, and stale Session isolation orgs older than an hour are deleted.

Verification: dashboard suite run locally (35 passed; 2 pauses, about 3s; peak 463/600). Typecheck and biome clean. CI on this PR is the first run on CI hardware.

🤖 Generated with Claude Code

Windscar and others added 2 commits November 13, 2026 00:42
All CI runs share one Clerk test user, so a fresh sign-in can open in
another run's fixture organization that this run's database never
provisioned. session-isolation's two sign-ins now activate the expected
organization the way global setup does, through a shared helper. This also
stops the re-sign-in from leaving the shared dev-browser client in a foreign
organization, which failed theme.e2e with "Organization is not provisioned".

The spec also deletes "Session isolation" fixture organizations older than
an hour that killed runs left on the test user.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every dashboard spec signs in as one Clerk user, and each dashboard page
load makes about nine sdp-api requests, so the single-worker suite runs at
CLERK_USER_MAX_REQUESTS (600 a minute). Whichever test crosses it gets a
429 and an error page, which is how session-isolation flaked on main.

Before each test, and before a suite provisions fixtures, the run now reads
the API's own sliding-window counters from its Redis and waits until 150
requests are free. It waits only when the next test could run out, and only
as long as that takes. Each check and each test's ending count is written
to test-results/rate-budget.jsonl, which CI uploads, so the headroom can be
tuned from real runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
sdp-docs Ready Ready Preview Nov 13, 2026 7:25am UTC
sdp-web Ready Ready Preview Nov 13, 2026 7:25am UTC
2 Skipped Deployments
Project Deployment Actions Updated
sdp-web-beta Ignored Ignored Preview Nov 13, 2026 7:25am UTC
sdp-docs-beta Skipped Skipped Nov 13, 2026 7:25am UTC

Request Review

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

React Doctor found no new issues. 🎉

Reviewed by React Doctor for commit b3c6ce4.

@greptile-apps

greptile-apps Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Low impact] The PR appears safe to merge; the previous finding is fixed and no new blocking issue was found.

Summary

This PR adds rate-limit pacing to dashboard tests and pins the expected organization after sign-in.

  • Dashboard tests pause when the shared user is close to its API limits.
  • Fresh sign-ins use the intended fixture organization, and stale fixtures are removed.

Reviews (4) · Last reviewed commit: "test(web): read window.Clerk through Cle..." · Reviewed by Greptile

Comment thread apps/sdp-web/playwright/support/test.ts Resolved
Review fixes for the dashboard e2e pacing helper:
- the headroom env value and the Redis SCAN/MGET replies are parsed with zod
- a Redis connection that closes before replying rejects instead of hanging
- a local run that cannot read its counters, or whose budget never frees
  up, fails with that reason instead of starting the test anyway
- the fixture file exports only `test`; specs import `expect` and types
  from @playwright/test, and imports use the app alias
- the uploaded log leaves out the Clerk user and org ids

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The API launcher resolves REDIS_URL from Kora's api.env and
apps/sdp-api/.env.local before the process env, falling back to
redis://127.0.0.1:6379, so a runner without REDIS_URL failed before any
test and a runner with a different one paced against the wrong counters.
The pacing helper now resolves it the same way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@clerk/testing/playwright declares window.Clerk globally, so the helper
reads it directly instead of casting window through unknown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Windscar
Windscar merged commit 9285c5d into main Nov 13, 2026
55 checks passed
@Windscar
Windscar deleted the claude/session-isolation-flake branch November 13, 2026 15:09

This branch was successfully deployed

2 active deployments
Preview – sdp-web — b3c6ce4d Deployed Nov 13, 2026 by vercel[bot]
Preview – sdp-docs — b3c6ce4d Deployed Nov 13, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants