chore(bots): regenerate registry for xAI-SearchBot and LinkupBot (#39)

* feat(sdk): OpenTelemetry spans, without depending on OpenTelemetry

A span around protect() and a child around rule evaluation, carrying the
decision id (which joins a span to its dashboard row), the conclusion, the
deciding rule, and whether the request cost a round trip to ingest.

The tracer is INJECTED, not imported. An optional peer dependency was the
obvious route and the wrong one: this package is dependency-free and
passes an edge-compatibility gate, and Workers and Vercel Edge are exactly
where a stray transitive import hurts. A conditional
import('@opentelemetry/api') bundles badly too -- the bundler either
resolves it, adding weight for the majority who do not use it, or fails on
a module that is legitimately absent.

The Tracer type is a structural subset of OpenTelemetry's, so
trace.getTracer('webdecoy') satisfies it with no adapter. Omit it and
there are no spans, no dependency and no behaviour change, which is the
majority case and should cost them nothing.

A tracer cannot break a request. startSpan returning nothing, throwing, or
returning a span whose every method throws all degrade to a no-op --
observability that can take the request path down is worse than none. The
no-op span also means call sites need no null checks, and an `if (span)`
is a branch that gets forgotten on the path that mattered.

Eight tests, four of them hostile tracers. Also asserts the span is ended
on the ERROR path: a leaked span holds memory and never reaches the
exporter, so the trace is silently incomplete rather than absent.

Closes WebDecoy/app#740

* chore(bots): regenerate registry for xAI-SearchBot and LinkupBot

Companion to WebDecoy/app. Both crawlers were hitting production and landing
unclassified, so requests from them were excluded from AI traffic figures
rather than attributed.

Generated, not hand-edited: pkg is the source of truth and the export runs in
agents.MatchUserAgent order, which the TS matcher depends on because it takes
the first hit.

The registry count guard moves 169 -> 171. It exists to make a registry change
deliberate rather than incidental, so bumping it here is the intended workflow
rather than a nuisance.

397 tests pass, parity vectors included.

* chore(bots): regenerate registry for the five newly identified crawlers

Companion to WebDecoy/app. DuckAssistBot, SofyaBot, Reflectionbot and IbouBot
join xAI-SearchBot and LinkupBot; all six were crawling production and landing
unclassified.

IbouBot is registered as a SEARCH crawler, not AI: its own page states it is a
Cloudflare Verified Bot classified as Search. It arrived in the same
unclassified tail as the others and does not belong with them.

Count guard 171 -> 175. 397 tests pass.

Author: cport1

packages/webdecoy/src/bots/bots.test.ts

@@ -60,7 +60,7 @@ describe('matchUserAgent', () => {
 
 describe('registry integrity', () => {
   it('carries the full generated table', () => {
-    expect(BOT_REGISTRY.length).toBe(169);
+    expect(BOT_REGISTRY.length).toBe(175);
     expect(BOT_CATEGORIES).toContain('training_crawler');
     expect(BOT_CATEGORIES).toContain('search_crawler');
   });

packages/webdecoy/src/bots/parity-vectors.generated.json

@@ -575,6 +575,18 @@
     "category": "seo_crawler",
     "matched": true
   },
+  {
+    "userAgent": "duckassistbot",
+    "id": "duckassistbot",
+    "category": "ai_search_crawler",
+    "matched": true
+  },
+  {
+    "userAgent": "Mozilla/5.0 (compatible; duckassistbot/1.0; +http://example.com/bot)",
+    "id": "duckassistbot",
+    "category": "ai_search_crawler",
+    "matched": true
+  },
   {
     "userAgent": "duckduckbot",
     "id": "duckduckbot",
@@ -935,6 +947,18 @@
     "category": "training_crawler",
     "matched": true
   },
+  {
+    "userAgent": "iboubot",
+    "id": "iboubot",
+    "category": "search_crawler",
+    "matched": true
+  },
+  {
+    "userAgent": "Mozilla/5.0 (compatible; iboubot/1.0; +http://example.com/bot)",
+    "id": "iboubot",
+    "category": "search_crawler",
+    "matched": true
+  },
   {
     "userAgent": "iframely",
     "id": "iframely",
@@ -1055,6 +1079,18 @@
     "category": "fetcher",
     "matched": true
   },
+  {
+    "userAgent": "linkupbot",
+    "id": "linkupbot",
+    "category": "ai_search_crawler",
+    "matched": true
+  },
+  {
+    "userAgent": "Mozilla/5.0 (compatible; linkupbot/1.0; +http://example.com/bot)",
+    "id": "linkupbot",
+    "category": "ai_search_crawler",
+    "matched": true
+  },
   {
     "userAgent": "mj12bot",
     "id": "mj12bot",
@@ -1559,6 +1595,18 @@
     "category": "fetcher",
     "matched": true
   },
+  {
+    "userAgent": "reflectionbot",
+    "id": "reflectionbot",
+    "category": "training_crawler",
+    "matched": true
+  },
+  {
+    "userAgent": "Mozilla/5.0 (compatible; reflectionbot/1.0; +http://example.com/bot)",
+    "id": "reflectionbot",
+    "category": "training_crawler",
+    "matched": true
+  },
   {
     "userAgent": "rogerbot",
     "id": "rogerbot",
@@ -1787,6 +1835,18 @@
     "category": "fetcher",
     "matched": true
   },
+  {
+    "userAgent": "sofyabot",
+    "id": "sofyabot",
+    "category": "ai_search_crawler",
+    "matched": true
+  },
+  {
+    "userAgent": "Mozilla/5.0 (compatible; sofyabot/1.0; +http://example.com/bot)",
+    "id": "sofyabot",
+    "category": "ai_search_crawler",
+    "matched": true
+  },
   {
     "userAgent": "sogou",
     "id": "sogou",
@@ -2147,6 +2207,18 @@
     "category": "generic_scraper",
     "matched": true
   },
+  {
+    "userAgent": "xai-searchbot",
+    "id": "xai-searchbot",
+    "category": "ai_search_crawler",
+    "matched": true
+  },
+  {
+    "userAgent": "Mozilla/5.0 (compatible; xai-searchbot/1.0; +http://example.com/bot)",
+    "id": "xai-searchbot",
+    "category": "ai_search_crawler",
+    "matched": true
+  },
   {
     "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36",
     "matched": false

packages/webdecoy/src/bots/registry.generated.ts

@@ -11,7 +11,7 @@
  * agents share User-Agent substrings, so sorting this array changes how real
  * traffic is classified.
  *
- * 169 agents across 15 categories.
+ * 175 agents across 15 categories.
  */
 
 /**
@@ -75,6 +75,7 @@ export const BOT_CATEGORIES: readonly BotCategory[] = [
 
 /** The agent table, in match order. */
 export const BOT_REGISTRY: readonly BotAgent[] = [
+  { id: "reflectionbot", name: "Reflectionbot", category: "training_crawler", organization: "Reflection AI", baseScore: 70, respectsRobots: true, uaPatterns: ["reflectionbot"] },
   { id: "gptbot", name: "GPTBot", category: "training_crawler", organization: "OpenAI", baseScore: 85, respectsRobots: true, uaPatterns: ["gptbot"] },
   { id: "chatgpt-user", name: "ChatGPT-User", category: "training_crawler", organization: "OpenAI", baseScore: 85, respectsRobots: true, uaPatterns: ["chatgpt-user", "chatgpt"] },
   { id: "oai-searchbot", name: "OAI-SearchBot", category: "training_crawler", organization: "OpenAI", baseScore: 80, respectsRobots: true, uaPatterns: ["oai-searchbot"] },
@@ -103,6 +104,10 @@ export const BOT_REGISTRY: readonly BotAgent[] = [
   { id: "velenpublicwebcrawler", name: "VelenPublicWebCrawler", category: "training_crawler", organization: "Velen", baseScore: 70, respectsRobots: false, uaPatterns: ["velenpublicwebcrawler"] },
   { id: "isscyberriskcrawler", name: "ISSCyberRiskCrawler", category: "training_crawler", organization: "ISS", baseScore: 65, respectsRobots: false, uaPatterns: ["isscyberriskcrawler"] },
   { id: "friendlycrawler", name: "FriendlyCrawler", category: "training_crawler", organization: "Unknown", baseScore: 65, respectsRobots: true, uaPatterns: ["friendlycrawler"] },
+  { id: "duckassistbot", name: "DuckAssistBot", category: "ai_search_crawler", organization: "DuckDuckGo", baseScore: 65, respectsRobots: true, uaPatterns: ["duckassistbot"] },
+  { id: "sofyabot", name: "SofyaBot", category: "ai_search_crawler", organization: "Sofya", baseScore: 65, respectsRobots: true, uaPatterns: ["sofyabot"] },
+  { id: "xai-searchbot", name: "xAI-SearchBot", category: "ai_search_crawler", organization: "xAI", baseScore: 70, respectsRobots: true, uaPatterns: ["xai-searchbot"] },
+  { id: "linkupbot", name: "LinkupBot", category: "ai_search_crawler", organization: "Linkup", baseScore: 65, respectsRobots: true, uaPatterns: ["linkupbot"] },
   { id: "searchgpt", name: "SearchGPT", category: "ai_search_crawler", organization: "OpenAI", baseScore: 75, respectsRobots: true, uaPatterns: ["searchgpt"] },
   { id: "phind", name: "Phind", category: "ai_search_crawler", organization: "Phind", baseScore: 70, respectsRobots: true, uaPatterns: ["phind"] },
   { id: "kagi", name: "Kagi", category: "ai_search_crawler", organization: "Kagi", baseScore: 65, respectsRobots: true, uaPatterns: ["kagi"] },
@@ -119,6 +124,7 @@ export const BOT_REGISTRY: readonly BotAgent[] = [
   { id: "browser-use", name: "Browser-Use", category: "ai_agent", organization: "Open Source", baseScore: 70, respectsRobots: false, uaPatterns: ["browser-use"] },
   { id: "stagehand", name: "Stagehand", category: "ai_agent", organization: "Browserbase", baseScore: 70, respectsRobots: false, uaPatterns: ["stagehand"] },
   { id: "multion", name: "MultiOn", category: "ai_agent", organization: "MultiOn", baseScore: 70, respectsRobots: false, uaPatterns: ["multion"] },
+  { id: "iboubot", name: "IbouBot", category: "search_crawler", organization: "Ibou", baseScore: 30, respectsRobots: true, uaPatterns: ["iboubot"] },
   { id: "googlebot", name: "Googlebot", category: "search_crawler", organization: "Google", baseScore: 30, respectsRobots: true, uaPatterns: ["googlebot"] },
   { id: "bingbot", name: "Bingbot", category: "search_crawler", organization: "Microsoft", baseScore: 30, respectsRobots: true, uaPatterns: ["bingbot"] },
   { id: "yandexbot", name: "YandexBot", category: "search_crawler", organization: "Yandex", baseScore: 35, respectsRobots: true, uaPatterns: ["yandexbot"] },