Two Engines, One Code Path

GEO Trace is a small tool. One HTTP function, an engine registry, a citation parser, and about 360 lines between them. The interesting part is not its scale. It is that the parts most likely to let us lie are the parts that were designed hardest.

This post walks the real implementation, including the shortcuts, because a lab that publishes other people’s citation data should be legible about its own.

What it actually does

One POST runs one query against one engine and returns the URLs that engine cited. There is no fan-out across a fleet. The request body carries an optional engine field, defaulting to claude, and the function runs that adapter and nothing else.

POST /geo-trace  { query, domain?, engine? }
    ↓
[rate limit]  →  5 per IP per hour, in-memory
    ↓
[validate]    →  query ≤ 200 chars, domain ≤ 100
    ↓
[registry]    →  getEngine(id), isConfigured(engine)
    ↓
[adapter]     →  run(query) → { model, citations, summary }
    ↓
[rank]        →  findDomainRank(citations, domain)
    ↓
JSON

Two adapters exist: Claude Sonnet 4.6 with Anthropic’s web_search tool capped at three searches per query, and Perplexity Sonar. Both are capped at 1024 output tokens. That is the whole fleet.

The honesty rule is structural, not editorial

The failure this codebase is most designed against is the one every GEO product commits: claiming engines it cannot run.

So “live” is not a string anyone types. It is derived:

export const isConfigured = (engine) => Boolean(process.env[engine.envKey]);

export const engineStatus = () =>
  ENGINES.map((e) => ({ id: e.id, label: e.label, model: e.model, live: isConfigured(e) }));

GET /geo-trace returns exactly that array, and the tool page lights its engine strip from the answer. A POST naming an engine whose key is absent gets a 503 that says so by name. The consequence is worth stating plainly: we cannot advertise an engine on the site that the deploy cannot actually call, because the advertisement is generated from the thing that calls it. Turning on the second engine was one environment variable, not a code change and not a copy change.

This is a small idea that removes an entire category of drift. Marketing copy ages. A value read out of process.env at request time does not.

One code path for the tool and the research

The second-riskiest failure for a measurement lab is subtler: the published research numbers being produced by different code than the shipping tool. Then the essays describe a system nobody can reach.

lib/trace.mjs is the fix, and it is deliberately boring:

export const TRACE_MODEL = 'claude-sonnet-4-6';
export const TRACE_MAX_TOKENS = 1024;
export const TRACE_TOOLS = [{ type: 'web_search_20250305', name: 'web_search', max_uses: 3 }];

export function buildTraceRequest(query) { /* the exact messages.create payload */ }
export async function runTrace(client, query) { /* → { model, citations, summary } */ }

Both the live function and the offline panel script import it. Same model, same prompt string, same tool config, same token ceiling. When the research pages say Claude cited 610 URLs across 100 queries, that number came out of the identical call a visitor makes when they run a trace.

Defensive parsing, because a false negative is worse than an error

Perplexity has returned citations under different response keys across API versions: search_results as objects, citations as either plain URL strings or objects. A tool that reads only one key would, after a rename upstream, quietly return zero citations.

Zero citations is not a visible failure. The interface would render it as “your domain was not cited” — a confident, wrong answer, delivered to someone who came here specifically to trust the measurement. That is strictly worse than a 500.

So the parser accepts every shape it has seen, from either key, and dedupes by URL while assigning rank in first-seen order:

const seen = new Map();
let rank = 0;
for (const c of candidates) {
  if (!c.url || seen.has(c.url)) continue;
  rank += 1;
  seen.set(c.url, { rank, url: c.url, title: c.title, snippet: c.snippet });
}

The general rule this encodes: when a parse failure and a real negative result are indistinguishable to the user, spend the code to keep them distinguishable.

Errors say nothing useful to strangers

The function is CORS-open to *, so anonymous cross-origin callers can hit it. Upstream error text can carry account and quota detail, so it never crosses that boundary. The real message goes to console.error; the caller gets a fixed string and a 502.

console.error(`[geo-trace] ${engine.id} upstream failed:`, detail);
return Response.json(
  { error: 'upstream', message: 'The engine request failed upstream. Please try again shortly.' },
  { status: 502, headers: CORS_HEADERS },
);

What this tool deliberately does not do

Every item here is a real gap, listed so nobody has to discover it by reading the source.

No retries. One attempt per trace. A retry policy would improve the success rate and would also mean a visitor waits longer without being told why. At five traces per IP per hour, a failed trace is cheap to repeat by hand.

No persistent rate-limit store. The limiter is a Map in function memory and it resets on cold start. A determined caller can reset their own budget by waiting out an idle period. This is a cost guard, not a security control, and it is sized for current traffic rather than for an adversary.

No fan-out. Comparing engines is done offline by the panel, not live per request. Running both engines on one request would double the cost and the latency of every trace to serve a comparison most visitors do not ask for.

No streaming. The function returns all-or-nothing. Per-engine streaming would need more state and harder error recovery for a response that already arrives in one shot.

No timing instrumentation. There are no P50 or P95 numbers on this page, because nothing in this codebase records them. An earlier version of this essay quoted latency percentiles anyway. They were not measured, and they are gone.

Why the response is JSON and the export is CSV

The function returns JSON; the tool page turns a result into a CSV on the client. The reasoning has survived several rounds of “why not a dashboard”:

A chart imposes a narrative — sort order, colour encoding, what gets emphasised. A CSV says here are the citations, here are the ranks, build your own reading. It also lands wherever the practitioner already works, a Sheet or a notebook or a BI pipeline, instead of fighting that. And the patterns worth finding in this data are the ones we have not found; a fixed dashboard would freeze the analysis at our current priors.

If you are building something similar

The two mechanisms worth copying are not the clever ones. They are isConfigured, which makes overclaiming a capability structurally impossible, and a shared call module, which makes your published numbers and your live product the same artefact. Everything else here is normalisation tax and can be rewritten in an afternoon.