AI Summaries
A short daily or weekly narrative of what happened on a site — what moved, what broke, and what to do about it — on the AI digest widget of the site's home dashboard.
The numbers are not written by the model
This is the part worth understanding before you rely on it.
Every figure in a summary is computed first, deterministically, by SQL over the same rollups that feed the rest of the dashboard. That fact sheet is what the model receives, and the widget renders the sheet, not the model's arithmetic. The model's only job is prose.
The distinction matters because prose can be wrong and a number cannot: if the model writes "sales fell sharply" over a day that was flat, you have a bad sentence over correct figures. It cannot invent a revenue number, because it never computes one.
What is sent to the model
Only aggregates:
- Pageviews, sessions, unique visitors, bounce rate, average time on page
- Error, click, rage-click and dead-click counts
- The top five page paths (never full URLs, so no query strings)
- The top three device types and countries
- Funnel step labels and their session counts
- Money: abandoned carts and their value, carts blocked by errors, orders, currency
- The top three error messages
No email address, phone number, name, IP address, visitor id or session id is ever included. The error messages — the only free text that goes — are passed through the same redaction the rest of the platform uses (addresses, long digit runs, tokens) and truncated to 140 characters.
Because summaries are site-level aggregates, GDPR erasure does not touch them: there is nothing in one that belongs to a person.
Staleness, and why nothing regenerates itself
Each summary stores a hash of the fact sheet it was written from, plus the version of the prompt. When either changes, the widget shows a badge and a button — it does not rewrite itself.
There is no TTL and no automatic regeneration, on purpose. A narrative that silently changed under someone who had just read it, or that spent model time every hour on a day nobody looked at, would be worse than a stale one that says it is stale.
A model that answers prose instead of the JSON it was asked for produces a failed row with a "Try again" button — never a garbled summary.
Configuration
Off unless both variables are set. When off, the widget explains that in one sentence rather than erroring; this is a supported configuration, not a broken one.
| Variable | Meaning |
|---|---|
AI_BASE_URL | Origin of an OpenAI-compatible server. Without /v1 — the client appends /v1/chat/completions itself. |
AI_MODEL | Model name, e.g. llama3.1:8b |
AI_API_KEY | Only if the endpoint requires one. Local Ollama does not. |
AI_TIMEOUT_MS | Default 180000. An 8B model on CPU legitimately takes minutes. |
Any OpenAI-compatible endpoint works — Ollama, llama.cpp, vLLM, or a hosted provider speaking the same dialect.
:::tip The /v1 trap
Almost every OpenAI-compatible example on the internet ends in /v1. Setting AI_BASE_URL=http://localhost:11434/v1 produces requests to /v1/v1/chat/completions, which 404 — and a 404 here looks exactly like an unreachable model. The value is stripped defensively and the startup config check warns about it, but the setting should read the way it behaves.
:::
Where to run the model
An external endpoint is recommended for a production deployment: a separate GPU box running vLLM, or a hosted OpenAI-compatible provider.
A model resident on the application server competes with everything else that runs at night — the 02:15 rollups, the 02:45 re-scoring, the 03:30 backup and the 04:00 retention sweep — and with the headless Chromium that renders heatmap backgrounds. A model that gets swapped out blows past AI_TIMEOUT_MS, and every summary becomes a failed row, which reads like a model-quality problem rather than a memory one.
If you do want the model on the same host, the production compose file ships an ollama service behind a profile, so it does not start by default:
docker compose --profile ai up -d ollama
docker compose exec ollama ollama pull llama3.1:8b
Then set AI_BASE_URL=http://ollama:11434 and AI_MODEL=llama3.1:8b. Only on a machine with at least 16 GB, and pull the model before the first nightly run — an unpulled model answers 404 and every site's summary lands as failed.
When summaries are written
- On demand, from the widget's Generate button.
- Nightly at 04:30, after the rollups have closed the previous day, for every site that had traffic. Weekly summaries are added on Mondays.
The nightly pass is idempotent: a site whose facts have not changed since its last summary is skipped without calling the model.
Verifying a setup
pnpm --filter backend build && pnpm verify:ai
37 assertions covering the week arithmetic, the hashing, the redaction, the failure modes and the nightly batch's idempotency — run against scripted models, so it needs no endpoint of its own.