Attention gating

Why

In a group chat with four humans and a bot, the bot should not respond every 3 seconds. But it should respond when the conversation is about a bug it can help with, even if no one @mentioned it. That's the "discriminator" lane.

The ladder

 silent ──▶ discriminate-quiet ──▶ discriminate ──▶ mentions-only ──▶ always
     (record)          (LLM)           (LLM)          (strict match)    (no filter)

Going right = more responsive. Going left = quieter.

Rule before LLM

The LLM discriminator is slow and costs money. Hard rules short-circuit:

/picortex <cmd> slash command → always respond
Reply-to a bot message → respond
@picortex mention → respond
Admin override from Jacob's DM → respond
Non-text payload → mode-dependent

Discriminator prompt

Lives at $CHAT_HOME/.picortex/prompts/discriminator.md, git-tracked inside the chat's home. Seeded from a default that can be tuned live.

Default (sketched):

You are deciding whether a chat bot should respond to the most recent message. Only respond if the message:

asks a question that a capable coding / research assistant could help with, OR

is reacting to a previous message from you, OR

contains a task/request that the assistant should pick up.

Do NOT respond if the message is:

small talk between humans

private conversation you weren't addressed in

off-topic or tangential

Return JSON: {"should_respond": bool, "reason": "short"}.

Prompt is git-versioned so Jacob can tune per-chat without redeploying.

Model choice

Default: claude-3-5-haiku (cheap, fast). Swap via chat_config.discriminator_model.

Cost: ~200 input tokens + 30 output tokens per classification ≈ 0.0005. 1,000 classifications =0.50. Cheap.

Failure modes

LLM returns non-JSON → 1 retry with reformat prompt; if still bad, fail-open for 1:1, fail-closed for groups.
LLM times out (> 5 s) → same fail policy.
Cost spike → per-day per-chat budget cap in chat_config. Hitting the cap disables discriminate until midnight.

Observability

Every discriminator decision is logged as an event_type: "discriminator.decision" with: chat_id, message_id, should_respond, reason, model, latency_ms, cost_usd.

Future

Personalized discriminator (fine-tune or few-shot on Jacob's own "respond / don't respond" label history). v0.3.
Hierarchical gating: per-user overrides inside a group. v0.2.

References

Spec 005
Cortex R4 — the pattern we adopted
Default template: /usr/local/share/picortex/discriminator.default.md (created by installer in S4)