Skip to content

feat(analytics): add X pixel behind analytics consent - #982

Merged
RapidPoseidon merged 4 commits into
mainfrom
feat(analytics)/add-x-pixel-behind-analytics-consent
Oct 28, 2026
Merged

RapidPoseidon merged 4 commits into
mainfrom
feat(analytics)/add-x-pixel-behind-analytics-consent

Conversation

@Faycal6619

@Faycal6619 Faycal6619 commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

⚠️ Do not merge until the conductor says go. The banner, preferences and privacy wording were approved by Fay and Jason on 2026-10-28. All three PRs (Website, app-frontend, this one) must go live the same day as X_PIXEL_CONSENT_SINCE (2026-10-29T00:00:00Z). If they slip, the conductor sends a new date for all three first.

Why

GTM (Fay) needs the X ads pixel on docs.rapidata.ai to measure X campaigns and build retargeting audiences. Same design across rapidata.ai, app.rapidata.ai and docs.rapidata.ai: no new consent category, no revision bump. Because they share one cc_cookie, consents given to the old text (which never mentioned advertising) must not turn the pixel on. Only a consent saved on or after X_PIXEL_CONSENT_SINCE, or a re-confirm since then, does.

Flow

flowchart LR
  banner["cookieconsent banner / preferences"]:::changed ==>|"Allow / Save → setCookieData adConsentAt"| cc[("cc_cookie on .rapidata.ai")]
  js["docs/js/analytics.js"]:::changed -->|"acceptedCategory + timestamps"| cc
  js ==>|"twq config ri3so + event tw-ri3so-ri3tn"| x["static.ads-twitter.com/uwt.js"]:::new
  x ==>|"adsct"| xa["t.co / analytics.twitter.com"]:::new
  js -->|"capture"| ph["rapidata.ai/ingest (PostHog)"]
  classDef new fill:#d3f9d8,stroke:#2f9e44,color:#000
  classDef changed fill:#fff3bf,stroke:#f08c00,color:#000
Loading

Decisions

  • Pixel rule: analytics accepted AND (lastConsentTimestamp ≥ X_PIXEL_CONSENT_SINCE OR data.adConsentAt ≥ it). cookieconsent 3.1 leaves lastConsentTimestamp alone when a re-confirm changes no category. So a click on Allow/Save with analytics on stamps adConsentAt via CookieConsent.setCookieData({ mode: "update" }). Checked in a browser: it persists to the single .rapidata.ai cookie and keeps the library's expiry. No hand-written cookie fallback was needed.
  • uwt.js auto features off (autoConfig, autoAdvancedMatching, dataLayerTracking, autoDwellTracking, set per pixel before config). Seen in a browser: by default uwt.js sends auto_long_site_dwell on its own timer, even after Reject, and re-sets _twpid/_twsid. autoAdvancedMatching also scrapes form fields for email/phone. With these off, the only hits are the base PageView plus the Visit docs event, and nothing more is sent after Reject. ⚠️ Sibling PRs should do the same.
  • Reload on revoke, not a timed sweep (simplified after review): after a ?twclid= landing, uwt.js 2.4.12 keeps re-writing _twclid from a closure that never re-checks consent. So when analytics is withdrawn on a page where uwt.js ran, we expire _twclid/_twpid/_twsid and call location.reload(). uwt.js doesn't load without consent, so the reload ends the keep-alive. If uwt.js never ran on the page, there's no reload. Every page load without valid consent also expires the three cookies once. The loader is the official X base snippet verbatim; the only additions are the four set-false flags and the consent gate. All three repos do the same.
  • Behaviour change: no same-page re-grant once the pixel ran. Reject reloads such a page, so Reject → Allow now re-grants on a fresh load. The PostHog browser test nothing reaches PostHog … after Reject passed only because the cutoff is still in the future. I ran the whole suite with the cutoff moved to the past: the re-grant click raced PostHog's heatmaps starting up on the reloaded page. The test now waits for posthog.heatmaps.isEnabled before that click. Pinning the browser clock before the cutoff, as app-frontend did, isn't usable here: Playwright's clock.setSystemTime alone breaks this test's heatmap flush. The X pixel test clock now never moves backwards.
  • Cookies: _twclid, _twpid, _twsid were observed on .rapidata.ai (_twclid only appears with a ?twclid= landing). The .twitter.com / t.co cookies are third-party, so we can't clear them. policyVersion is now 2026-10-29.

Preferences dialog, Analytics section: "Allow analytics to connect your visits across our sites, understand how you use Rapidata and measure our advertising. We use PostHog and the X Pixel, which tells X which of our sites you visited so we can measure our ads on X." The docs dialog had no provider sentence before, so "We use PostHog and" is new wording. Please review it alongside the other copy.

Verified

Suite run with X_PIXEL_CONSENT_SINCE set to both 2026-10-27 (past) and 2026-10-29 (future): unit and browser tests pass in both cases.

Live uwt.js check of the reload: landed with ?twclid=abc, then Allow, then Reject. The page reloaded without twq, and no _tw* cookie came back in 25 s. Unit tests cover three cases: revoke reloads once, there's no reload when the pixel never loaded, and the cookies are expired on any load without valid consent.

I built the site with uv run --group docs mkdocs build and ran Playwright against the live uwt.js, with the clock set past X_PIXEL_CONSENT_SINCE:

  • Reject: 0 requests to X hosts.
  • Fresh Allow: one uwt.js load and one visit event. No repeat after instant navigation or after Allow → Reject → Allow.
  • Consent cookie dated before the cutoff: no load.
  • Re-Allow in preferences: the pixel loads, and keeps loading on reload.

The new CI browser test consent.browser.test.js covers the same cases. Reviewer: please apply skill-unchanged-approved. This PR changes only docs/js/analytics.js and its tests, the SDK and src/rapidata/_skill/ are untouched, and nothing the agent skill documents changed.

Sibling PRs: RapidataAI/Website#606 (also updates the privacy policy) · RapidataAI/app-frontend#3753

🔗 Session: node-52a5e9ed

🤖 Generated with Claude Code

Loads the X base code (pixel ri3so) and sends the "Visit docs" event once per
full page load, only when analytics is accepted on or after the banner text that
mentions ad measurement (X_PIXEL_CONSENT_SINCE), or re-confirmed since then
(cc_cookie data.adConsentAt). Auto events and form-PII matching are switched off,
and the pixel's first-party cookies are expired on withdrawal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: fay@rapidata.ai <fay@rapidata.ai>
@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

✅ No skill update needed — confirmed by @LinoGiger

This PR does not modify src/rapidata/_skill/. That directory is the agent skill that ships in every SDK release and that coding agents read through python -m rapidata skill.

Before merging, pick one:

  1. The change affects what an agent needs to know (new or renamed API, changed parameter, default or result field, new gotcha): update src/rapidata/_skill/SKILL.md (or a companion guide next to it) in this PR.
  2. Nothing the skill documents changed: a reviewer applies the skill-unchanged-approved label. New commits remove the label again.

@RapidPoseidon

Copy link
Copy Markdown
Contributor

The Agent Skill check is red by design: it waits for the skill-unchanged-approved label. This PR only touches docs/js/analytics.js and its tests, so nothing in src/rapidata/_skill/ documents what changed. A reviewer needs to apply the label. Per AGENTS.md, that's not for the author to do.

uwt.js 2.4.12 re-writes _twclid for 120 s after a ?twclid= landing (at 5 s,
every 10 s, and on visibilitychange->hidden) without re-checking consent, so a
single delete on revoke does not hold. After a revoke, expire it every second and
on hide for 130 s; a re-grant stops the sweep.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: fay@rapidata.ai <fay@rapidata.ai>
@RapidPoseidon

Copy link
Copy Markdown
Contributor

LinoGiger, can you review this and apply the skill-unchanged-approved label? It changes only docs/js/analytics.js and its tests. The SDK and src/rapidata/_skill/ are untouched, so the agent skill has nothing to update. That's the only reason the Agent Skill check is red. Please don't merge: the conductor releases all three X-pixel PRs on the same day.

@Faycal6619
Faycal6619 marked this pull request as ready for review October 28, 2026 10:14
Simpler, as asked in review: when analytics is withdrawn on a page that loaded
uwt.js, expire the X cookies and reload, which ends uwt.js's _twclid keep-alive.
Every load without valid consent expires the X cookies once. The loader is now
the official X base snippet; only the four twq set-false flags, which back the
privacy-policy statement, stay on top of it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: fay@rapidata.ai <fay@rapidata.ai>
@RapidPoseidon

Copy link
Copy Markdown
Contributor

CI status at 9bee0cf: test, type-check and Check skill update pass, and build (docs build + browser tests) is running. Agent Skill is the only red check. No code change can fix it: it needs a reviewer to apply skill-unchanged-approved, because this PR doesn't touch the SDK or src/rapidata/_skill/.

…l cutoff

From X_PIXEL_CONSENT_SINCE on, the first Allow loads the X pixel, so the later
Reject reloads the page and the re-grant runs on a fresh load where heatmaps are
not enabled yet. Wait for heatmaps before the post-grant click, and never move
the X pixel test clock backwards. Checked with the cutoff in the past and future.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: fay@rapidata.ai <fay@rapidata.ai>
@LinoGiger LinoGiger added the skill-unchanged-approved Reviewer confirms this PR needs no agent skill update label Oct 28, 2026
@RapidPoseidon
RapidPoseidon merged commit ac354a5 into main Oct 28, 2026
10 checks passed
@RapidPoseidon
RapidPoseidon deleted the feat(analytics)/add-x-pixel-behind-analytics-consent branch October 28, 2026 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skill-unchanged-approved Reviewer confirms this PR needs no agent skill update

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants