fix(knowledge-base): enforce published-only context filter + authenticate the internal chat proxy

knowledge-base/AGENT.md

@@ -30,8 +30,9 @@ pnpm install && pnpm run bootstrap && pnpm dev
 
 - The dataset is **private**; read tokens live server-side only (Next API route, Hono proxy). Nothing secret in `NEXT_PUBLIC_*` / `SANITY_APP_*` vars — those are browser-bundled.
 - The App SDK app (`dashboard/`) has no server, so its chat goes through `dashboard-server/`. Browsing there needs no token (logged-in user session).
+- `dashboard-server/` `/api/chat` requires `Authorization: Bearer <DASHBOARD_API_TOKEN>` (401 otherwise, 500 fail-closed when unset) and CORS is pinned to `DASHBOARD_ORIGIN` (localhost:3333 fallback in dev only, never `*`). The dashboard sends the same value from `SANITY_APP_DASHBOARD_API_TOKEN` — bundled into the browser on purpose; it is a shared secret gating the proxy, not a Sanity token. Bootstrap generates both.
 - Agent Context MCP requires a **deployed Studio** — deployed schema alone returns `-32004`.
-- Scoping is the `sanity.agentContext` document's `groqFilter`, not the token. New document types must be added to the filters and instructions (seeded from `studio/scripts/generate-seed.ts`).
+- Scoping is the `sanity.agentContext` document's `groqFilter`, not the token. New document types must be added to the filters and instructions (seeded from `studio/scripts/generate-seed.ts`). The `customer-support` filter also requires `status == "published"` on types that carry a `status` field — a new external type with a status must be added to that clause, not just the `_type` list.
 - Hybrid search (`app/sanity/search.ts`): `text::semanticSimilarity()` only inside `score()`, needs Dataset Embeddings enabled, keyword `match` fallback on error.
 - Agent Insights (opt-in at bootstrap): both chat backends save conversations via `sanityInsightsIntegration` (new instance per request, Editor write token `SANITY_INSIGHTS_WRITE_TOKEN`, skipped when unset). The scheduled `classify-conversations` function (hourly, blueprint robot token, `ANTHROPIC_API_KEY` function env) fills in the Studio dashboard metrics; without the write token it deploys but idles.
 

knowledge-base/README.md

@@ -23,9 +23,11 @@ One Sanity dataset feeds two AI surfaces through two scoped [Agent Context](http
 ### Security model
 
 - Dataset is **private** — no API access without a server-side token.
-- The external surface uses a read token scoped (via its Agent Context GROQ filter) to external content types only.
-- The internal surface uses a separate read token scoped to all types.
-- **No token ever reaches the browser** on either surface.
+- The external surface uses a read token scoped (via its Agent Context GROQ filter) to external content types only — and, for `helpArticle` and `faq`, only documents whose `status` is `published`. Draft and archived content is invisible to the customer-facing agent, matching the help center's own search.
+- The internal surface uses a separate read token scoped to all types and every status (staff see drafts and archived content, with the agent warning when content is stale).
+- **No Sanity token ever reaches the browser** on either surface.
+- The internal chat proxy (`dashboard-server/`) is gated by a shared secret: the dashboard sends `DASHBOARD_API_TOKEN` as a bearer token on `/api/chat`, unauthenticated requests get `401`, and the proxy fails closed if the secret is unset. CORS is never a wildcard — `DASHBOARD_ORIGIN` must name the dashboard's origin in production. Bootstrap generates the secret and writes it to both `dashboard-server/.env.local` and (as `SANITY_APP_DASHBOARD_API_TOKEN`) `dashboard/.env.local`.
+  - This secret is bundled into the browser-only dashboard, so it is visible to anyone who can load the dashboard — staff signed in to Sanity, which is the intended audience. It stops anonymous callers, not dashboard users. It is a minimal gate for a starter: a real deployment should put the proxy behind its own SSO / auth (or verify the caller's Sanity session) instead.
 
 ### Agent Insights (opt-in)
 
@@ -68,6 +70,8 @@ knowledge-base/
 
 Each workspace manages its own `.env` — no cascading from root. Copy each `.env.example` to `.env`. See each file for required values.
 
+The dashboard and its chat proxy share one secret: `DASHBOARD_API_TOKEN` (`dashboard-server/`) must equal `SANITY_APP_DASHBOARD_API_TOKEN` (`dashboard/`). `pnpm bootstrap` generates it; to set it by hand use `openssl rand -hex 32`. When deploying the proxy, also set `DASHBOARD_ORIGIN` to the deployed dashboard's origin — it only defaults to `http://localhost:3333` in development.
+
 ## Available scripts
 
 | Command          | Description                                          |

knowledge-base/app/.env.example

@@ -9,7 +9,7 @@ NEXT_PUBLIC_SANITY_DATASET=production
 # sanity.io/manage -> Your Project -> API -> Tokens
 SANITY_READ_TOKEN_EXTERNAL=
 
-# External Agent Context (MCP) endpoint — scoped to customer-facing content.
+# External Agent Context (MCP) endpoint — scoped to published customer-facing content.
 # Shape: https://api.sanity.io/<version>/agent-context/<projectId>/<dataset>/customer-support
 # Confirm the exact URL in the Studio Agent Context panel.
 SANITY_AGENT_CONTEXT_URL=

knowledge-base/dashboard-server/.env.example

@@ -23,7 +23,17 @@ SANITY_AGENT_CONTEXT_URL_INTERNAL=
 # Anthropic API key — powers the chat model.
 ANTHROPIC_API_KEY=
 
-# Allowed browser origin (the App SDK dashboard) for CORS.
+# Shared secret the dashboard must present as `Authorization: Bearer …` on
+# /api/chat. Required — the proxy refuses every request (500) while it is unset
+# and returns 401 for a wrong or missing token. `pnpm bootstrap` generates it
+# (or use `openssl rand -hex 32`); it must match SANITY_APP_DASHBOARD_API_TOKEN
+# in dashboard/. This is a minimal gate for a starter: a real deployment should
+# put this proxy behind its own SSO / auth instead of a shared secret.
+DASHBOARD_API_TOKEN=
+
+# Allowed browser origin (the App SDK dashboard) for CORS. Never a wildcard.
+# Required when NODE_ENV=production (set it to the deployed dashboard origin);
+# falls back to http://localhost:3333 in development.
 DASHBOARD_ORIGIN=http://localhost:3333
 
 # Port the proxy listens on.

knowledge-base/dashboard-server/src/index.ts

@@ -6,6 +6,7 @@ import {serve} from '@hono/node-server'
 import {convertToModelMessages, stepCountIs, streamText, tool, type ToolSet, zodSchema} from 'ai'
 import {Hono} from 'hono'
 import {cors} from 'hono/cors'
+import {timingSafeEqual} from 'node:crypto'
 import {z} from 'zod'
 
 import {MODEL_ID, SYSTEM_PROMPT} from './constants'
@@ -61,17 +62,64 @@ function insightsTelemetry(threadId: string) {
   }
 }
 
+// ── Access control ───────────────────────────────────────────────────────────
+// This proxy fronts the internal knowledge base (policies, playbooks, HR,
+// security), so it must not be callable by anyone who can reach the port. The
+// dashboard presents a shared secret as a bearer token; requests without it
+// get 401, and the proxy fails closed (500) if the secret was never configured.
+// This is a minimal gate for a starter — a real deployment should sit behind
+// its own SSO / auth layer (or verify the caller's Sanity session) instead.
+
+const isProduction = process.env.NODE_ENV === 'production'
+const apiToken = process.env.DASHBOARD_API_TOKEN
+
+if (!apiToken) {
+  console.error(
+    'DASHBOARD_API_TOKEN is not set — /api/chat will refuse every request.\n' +
+      '  Run `pnpm bootstrap`, or set DASHBOARD_API_TOKEN in dashboard-server/.env.local and\n' +
+      '  the same value as SANITY_APP_DASHBOARD_API_TOKEN in dashboard/.env.local.',
+  )
+}
+
+// CORS is never a wildcard. Production must name the dashboard's origin; dev
+// falls back to the local App SDK dev server.
+const allowedOrigins = process.env.DASHBOARD_ORIGIN
+  ? [process.env.DASHBOARD_ORIGIN]
+  : isProduction
+    ? []
+    : ['http://localhost:3333']
+
+if (allowedOrigins.length === 0) {
+  console.error(
+    'DASHBOARD_ORIGIN is not set — browsers will be blocked by CORS. Set it to the deployed dashboard origin.',
+  )
+}
+
+const isAuthorized = (authorizationHeader: string | undefined): boolean => {
+  if (!apiToken || !authorizationHeader?.startsWith('Bearer ')) return false
+  const presented = Buffer.from(authorizationHeader.slice('Bearer '.length))
+  const expected = Buffer.from(apiToken)
+  return presented.length === expected.length && timingSafeEqual(presented, expected)
+}
+
 const app = new Hono()
 
 app.use(
   '/api/*',
   cors({
-    origin: process.env.DASHBOARD_ORIGIN ?? '*',
-    allowHeaders: ['Content-Type'],
+    origin: allowedOrigins,
+    allowHeaders: ['Content-Type', 'Authorization'],
     allowMethods: ['POST', 'OPTIONS'],
   }),
 )
 
+// Runs after cors() so preflight requests still get their headers.
+app.use('/api/*', async (c, next) => {
+  if (!apiToken) return c.json({error: 'DASHBOARD_API_TOKEN is not set'}, 500)
+  if (!isAuthorized(c.req.header('Authorization'))) return c.json({error: 'Unauthorized'}, 401)
+  await next()
+})
+
 app.post('/api/chat', async (c) => {
   // `id` is the chat id — the AI SDK's default transport sends it with every
   // request, so it doubles as a stable per-conversation thread id.

knowledge-base/dashboard/.env.example

@@ -9,6 +9,13 @@ SANITY_APP_DATASET=production
 # URL of the dashboard-server chat proxy.
 SANITY_APP_CHAT_PROXY_URL=http://localhost:8788/api/chat
 
+# Shared secret the chat proxy requires, sent as `Authorization: Bearer …`.
+# Must match DASHBOARD_API_TOKEN in dashboard-server. `pnpm bootstrap` generates
+# it. This one IS bundled into the browser on purpose: it stops anonymous callers
+# hitting the proxy, and anyone who can load this dashboard (staff signed in to
+# Sanity) is meant to have it. It is not a Sanity token and grants nothing else.
+SANITY_APP_DASHBOARD_API_TOKEN=
+
 # Organization ID — required for `sanity deploy` (App SDK apps deploy to the org
 # Dashboard). Find it at sanity.io/manage.
 SANITY_STUDIO_ORGANIZATION_ID=

knowledge-base/dashboard/src/components/ChatPanel.tsx

@@ -6,9 +6,16 @@ import {ResultCards} from './result-cards'
 
 // Chat talks to the dashboard-server proxy, which holds the internal read token
 // and calls the internal Agent Context (Team KB) — the token never reaches here.
+// The proxy requires a shared secret as a bearer token so it is not open to
+// anyone who can reach it; that secret is not a Sanity token and is bundled
+// into this browser app by design (see dashboard/.env.example).
 export function ChatPanel() {
   const transport = useMemo(
-    () => new DefaultChatTransport({api: process.env.SANITY_APP_CHAT_PROXY_URL}),
+    () =>
+      new DefaultChatTransport({
+        api: process.env.SANITY_APP_CHAT_PROXY_URL,
+        headers: {Authorization: `Bearer ${process.env.SANITY_APP_DASHBOARD_API_TOKEN}`},
+      }),
     [],
   )
   const {messages, sendMessage, status} = useChat({

knowledge-base/dashboard/src/env.d.ts

@@ -1,11 +1,15 @@
 // SANITY_APP_-prefixed vars are bundled into the browser by the App SDK build.
-// Never put secrets here — the chat token lives in the dashboard-server proxy.
+// Never put Sanity tokens here — the read token lives in the dashboard-server
+// proxy. SANITY_APP_DASHBOARD_API_TOKEN is the one deliberate exception: a
+// shared secret the proxy requires, visible to anyone who can load this app
+// (staff signed in to Sanity), and worthless against the Content Lake itself.
 declare global {
   namespace NodeJS {
     interface ProcessEnv {
       SANITY_APP_PROJECT_ID: string
       SANITY_APP_DATASET?: string
       SANITY_APP_CHAT_PROXY_URL: string
+      SANITY_APP_DASHBOARD_API_TOKEN: string
     }
   }
 }

knowledge-base/skills/add-scoped-agent-surface/SKILL.md

@@ -27,12 +27,15 @@ Add it to the seed generator (`studio/scripts/generate-seed.ts`) or create it di
   version: '1',
   name: 'Partner Portal',
   slug: {_type: 'slug', current: 'partner-portal'},
-  groqFilter: '_type in ["helpArticle", "faq", "playbook", "product", "topic"]',
+  groqFilter:
+    '_type in ["helpArticle", "faq", "playbook", "product", "topic"] && (_type in ["product", "topic"] || status == "published")',
   instructions: PARTNER_INSTRUCTIONS,
 }
 ```
 
 - `groqFilter` is the access boundary — only matching documents are visible through this surface. It can filter on any field, not just `_type` (e.g. `audience` arrays, `status == "published"`).
+- Enforce workflow status in the filter, not in `instructions`. The model treats instructions as advice; only the filter is guaranteed. Types without a `status` field (taxonomy like `product`, `topic`) need the `_type in [...] ||` escape hatch above, or a status clause silently hides them.
+- Context MCP queries the `published` perspective by default, so Sanity drafts (`drafts.*`) are already excluded — the `status` clause is about the workflow field on published documents.
 - `instructions` is a system-prompt addendum the MCP serves to the agent: describe the content model, which types answer which questions, and how documents reference each other. Copy the structure of `EXTERNAL_INSTRUCTIONS` / `INTERNAL_INSTRUCTIONS` in the seed generator.
 
 Publish the document (seed import publishes it; a draft is invisible to MCP).
@@ -60,7 +63,7 @@ The token is in the `.key` field of the JSON output. Put it in the env file of w
 Pick the host based on where the UI lives:
 
 - **UI in a server-backed app (Next.js):** copy `app/app/api/chat/route.ts` — `createMCPClient` (http transport, Bearer token) + `streamText` + a `displayCards` UI tool, capped with `stepCountIs(8)`.
-- **UI in a browser-only app (App SDK, SPA):** copy `dashboard-server/src/index.ts` — same pattern behind a small Hono proxy with CORS locked to the UI origin. The App SDK has no server and cannot hold secrets; the proxy is the secret boundary.
+- **UI in a browser-only app (App SDK, SPA):** copy `dashboard-server/src/index.ts` — same pattern behind a small Hono proxy with CORS locked to the UI origin and a shared-secret bearer token (`DASHBOARD_API_TOKEN`) required on the chat route. The App SDK has no server and cannot hold Sanity tokens; the proxy is the secret boundary. The bearer secret is a minimal gate against anonymous callers — a real deployment should front the proxy with its own auth.
 
 Point the endpoint at the new surface via env:
 

knowledge-base/studio/scripts/bootstrap.ts

@@ -13,7 +13,8 @@
  *  7. Make the dataset private (security model: no API access without a token)
  *  8. Enable Dataset Embeddings (powers hybrid semantic retrieval)
  *  9. Create the external + internal read tokens and write their Agent Context MCP URLs,
- *     plus the Agent Insights write token shared by both chat surfaces (if enabled)
+ *     plus the Agent Insights write token shared by both chat surfaces (if enabled),
+ *     plus the shared secret the dashboard uses to authenticate to its chat proxy
  * 10. Import seed data (ndjson)
  * 11. Restore dependencies (blueprint deploy can disrupt node_modules)
  * 12. Generate types (schema extract + typegen) so `pnpm dev` works out of the box
@@ -23,6 +24,7 @@
  */
 
 import {execFileSync} from 'node:child_process'
+import {randomBytes} from 'node:crypto'
 import {copyFileSync, existsSync, readFileSync, writeFileSync} from 'node:fs'
 import {resolve} from 'node:path'
 import {getCliClient} from 'sanity/cli'
@@ -406,7 +408,8 @@ try {
 // ── 9. External read token + MCP URL ─────────────────────────────────────────
 // A Viewer token, used server-side by the help center to call its Agent Context
 // MCP endpoint. Never exposed to the browser. The external/internal boundary is
-// the MCP slug (customer-support) + groqFilter, not the token's role.
+// the MCP slug (customer-support) + groqFilter (external types, published
+// status only), not the token's role.
 
 heading('External read token')
 try {
@@ -547,6 +550,34 @@ if (!insightsEnabled) {
   }
 }
 
+// ── 9d. Dashboard API token ──────────────────────────────────────────────────
+// A shared secret the dashboard presents to dashboard-server on /api/chat, so
+// the internal chat proxy is not open to anyone who can reach it. Generated
+// locally (no Sanity call) and written to both sides. Minimal gate — a real
+// deployment should put the proxy behind its own auth (see README).
+
+heading('Dashboard API token')
+try {
+  const existingToken = parseEnvFile(serverEnvLocal).DASHBOARD_API_TOKEN
+  const dashboardApiToken =
+    existingToken && isRealValue(existingToken) ? existingToken : randomBytes(32).toString('hex')
+  if (dashboardApiToken === existingToken) {
+    console.log('Dashboard API token already set — reusing')
+  } else {
+    patchEnvVar(serverEnvLocal, 'DASHBOARD_API_TOKEN', dashboardApiToken)
+    console.log('Generated dashboard API token')
+  }
+  patchEnvVar(dashboardEnvLocal, 'SANITY_APP_DASHBOARD_API_TOKEN', dashboardApiToken)
+  console.log('Wrote dashboard API token to dashboard and dashboard-server env')
+  success('Dashboard API token')
+} catch (err) {
+  failed(
+    'Dashboard API token',
+    err,
+    'openssl rand -hex 32  # then set DASHBOARD_API_TOKEN in dashboard-server/.env.local and SANITY_APP_DASHBOARD_API_TOKEN in dashboard/.env.local',
+  )
+}
+
 // ── 10. Import seed data ─────────────────────────────────────────────────────
 
 heading('Import seed data')

knowledge-base/studio/scripts/generate-seed.ts

@@ -629,7 +629,7 @@ const EXTERNAL_INSTRUCTIONS = `# Customer Support context
 - Hybrid retrieval: \`*[_type in ["helpArticle","faq"]] | score(text::semanticSimilarity("content", $query)) | order(_score desc)[0...5]\`.
 
 ## Content filter
-Scoped to external types only — internal playbooks and policies are never visible here.`
+Scoped to published external content only — help articles and FAQs still in draft or archived status, and all internal playbooks and policies, are never visible here.`
 
 const INTERNAL_INSTRUCTIONS = `# Team KB context
 
@@ -729,13 +729,20 @@ const emitInternal = (type: 'playbook' | 'policy', items: Internal[]) => {
 emitInternal('playbook', playbooks)
 emitInternal('policy', policies)
 
+// The filter is the enforced boundary; the instructions only advise. Content
+// types carry a workflow `status` (helpArticle, faq) and must be published to
+// be visible externally; taxonomy types (product, topic) have no status and
+// stay visible. Context MCP already queries the published perspective, so
+// Sanity drafts are excluded without a separate clause. Kept as a top-level
+// `&&` so it composes safely with whatever the agent's own query adds.
 docs.push({
   _id: 'agentContext.external',
   _type: 'sanity.agentContext',
   version: '1',
   name: 'Customer Support',
   slug: {_type: 'slug', current: 'customer-support'},
-  groqFilter: '_type in ["helpArticle", "faq", "product", "topic"]',
+  groqFilter:
+    '_type in ["helpArticle", "faq", "product", "topic"] && (_type in ["product", "topic"] || status == "published")',
   instructions: EXTERNAL_INSTRUCTIONS,
 })
 docs.push({