SignalHub 1.0 · Updated 2026-10-19 · Markdown source

AI Features: Overview and Build Order

Spec: AI-FEATURE-IDEAS.txt (repo root), sections F1, F2, A1, A2, B1, D1.

The "top five" from the idea list are split into four plans, one per subsystem, because they ship independently:

Recommended order: 1 -> 2 -> 3 -> 4. Plan 4 can run in parallel with anything.

What changed from the idea list

• A2 (AI-written messages for auto-incidents) is not a separate plan. Its safety goal, never publishing raw check errors, is already fixed in commit abca579 using publicFailureMessage() (no AI). An LLM rewrite would only see the same five error categories, so the marginal value is low. If wanted later, the draft builder from Plan 2 can be reused to polish the auto-incident body asynchronously. Not recommended now.
• F2 guardrails are built into Plan 1 (redaction, untrusted-data wrapper, per-org opt-in, provider kill switch, usage log) instead of being a separate project.

Decisions baked into the plans (change them before starting if you disagree)

1. No vendor SDKs for LLM calls. Plain HTTP through the existing guardedFetch, two wire formats: Anthropic Messages and OpenAI-compatible chat completions (covers OpenAI, Ollama, vLLM, LM Studio, LiteLLM, Azure/Bedrock behind a gateway). No new dependency in Plans 1-3.
2. Configuration is platform-level plus per-organization opt-in. Platform admin sets one provider for the installation (key encrypted like SMTP/SMS secrets); each organization Admin turns AI on for their organization. Per-organization keys are not built.
3. AI drafts, a human publishes. No feature in these plans publishes AI text by itself.
4. Plan 4 uses the official MCP TypeScript SDK v2 (@modelcontextprotocol/server, Apache-2.0, depends on zod ^4 which the repo already has). It is the only new dependency. Reason: the MCP spec's current revision (2026-07-28) changed the transport (no sessions, per-request metadata headers, header/body validation), so hand-rolling JSON-RPC is risky. The SDK line @modelcontextprotocol/sdk@1.x stops at revision 2025-11-25 and is not used.
5. Statistics before LLMs (anomaly detection, alert grouping) are out of scope here; they are Phase 3 in the idea list.

Pre-existing issues found while planning (not fixed by these plans unless noted)

• lib/migrations.ts:35 LATEST_MIGRATION_ID is "013_more_sms_providers.sql" but migrations 014 and 015 exist. It is only displayed on two platform pages, so it is cosmetic, but it is wrong. Plan 1 Task 1 bumps it when adding migration 016.
• The REST write routes under /api/v1/manage/* do not write tenant audit entries for API-key actions. Plan 4 audits MCP writes; adding the same to REST is a separate decision.

Constraints that apply to every plan

• Read the relevant guide in node_modules/next/dist/docs/ before touching a Next.js route or server-action convention (project AGENTS.md: this Next.js version has breaking changes). For Plan 4 read node_modules/next/dist/docs/01-app/01-getting-started/15-route-handlers.md.
• Schema change = new numbered migration in db/migrations/ + matching lib/postgres/schema.ts update. Migrations are checksummed; never edit one that has been committed.
• Run npm run verify (lint with zero warnings, typecheck, tests, build) before the last commit of each plan. Commit finished tasks; do not leave them pending.
• Tests live in tests/, import with relative paths (../lib/...) like tests/csv.test.ts, and run with npx vitest run tests/<file>.test.ts.
• Database-backed code is not unit-testable here (integration tests need INTEGRATION_DATABASE_URL pointing at a disposable DB whose name contains "test"). Plans therefore keep logic in pure functions or in factories with injected dependencies, and list a manual check for the thin database wiring.
• Every AI call goes through lib/ai (Plan 1). No feature calls a provider directly.
• Never put secrets in prompts: monitor authSecret, requestHeaders, requestBody, SMTP/SMS keys, subscriber contact data. Build prompt context from an allowlist of fields.

How to execute

Use superpowers:subagent-driven-development (one fresh agent per task plus review) or superpowers:executing-plans (inline). Plans 1-3 touch shared files (app/admin/(protected)/incidents/actions.ts, lib/postgres/schema.ts), so run them sequentially, not in parallel.

Verification status (what was and was not checked)

Checked, by actually running it:

• Every pure module and test file in Plans 1-4 was extracted verbatim from these documents into the repo, run (red first where the plan says so, then green), typechecked and linted with zero warnings.
• Every UI, server-action and wiring edit in Plans 1-3, and the MCP route and database wiring in Plan 4, was applied in a throwaway tree: npm test 346 passed (255 existing plus 91 new), tsc --noEmit clean, eslint clean, next build --webpack succeeded and listed /api/mcp. The throwaway tree was discarded; none of this code is in the repository yet, only these documents and AI-FEATURE-IDEAS.txt.
• The MCP SDK was probed against both request shapes (2026-07-28 envelope and the 2025-11-25 initialize handshake): both answered, GET returned 405, unknown tools returned a protocol error, invalid arguments returned isError.

Defects the dry run found and the plans now contain the fix for: the regex s flag is rejected by this repo's TypeScript target (Plan 2, finalizeDraft); tool().run threw synchronously instead of rejecting (Plan 4); a key with no MCP scope got a bare "Method not found" (Plan 4, now a clear 403); a truncation test that could not fail (Plan 2); an untyped header object (Plan 4).

Not checked (needs live services; each plan lists a manual check):

• Applying migrations 016 and 017 to a real database and the Kysely queries in lib/ai/index.ts and lib/mcp/deps.ts.
• Real provider calls (Anthropic, OpenAI, Ollama). Open question: whether OpenAI's newest models reject max_tokens (Plan 1, Review Focus 6).
• The UI flows in a browser, and a real MCP client (claude mcp add).
• @modelcontextprotocol/server v2 is new (2.3.1). Pin it and re-run Plan 4 Task 3 tests when upgrading.