An LLM agent hallucinated completely incorrect answers about our knowledge base. It wasn’t because the model was weak, and it wasn’t because the data was missing. It was because we fed the model raw JSON rows, and 70% of its context window was instantly consumed by timestamps and UUIDs.

We treated the LLM like a database engine. It acted like one that was out of memory.

The JSON Overhead

When building the Equill agent interface, the easiest way to give the model context was to dump the retrieved SQLite rows directly into the prompt. The JSON objects contained the content, but they also contained a massive payload of structural metadata: internal UUIDs, creation timestamps, update timestamps, user IDs, and foreign keys.

To a traditional application, metadata is essential. To an LLM, metadata is noise.

As the context window filled up with {"created_at": "2026-10-26T14:32:00Z"} and {"id": "a1b2c3d4-e5f6-7890"} across dozens of rows, the model’s attention mechanism was diluted. The actual prose content—the signal the model needed to reason about—was buried under a mountain of brackets, quotes, and irrelevant identifiers.

Bare Prose

We rewrote the retrieval pipeline to aggressively strip the JSON formatting before it reached the LLM.

Instead of dumping database objects, we transformed the rows into bare Markdown text, extracting only the human-readable content and perhaps a simple title. We stripped every UUID, every timestamp, and every curly brace.

The hallucination stopped immediately. The model could now process three times as many relevant facts within the same token limit, and its attention was focused entirely on the content.

Stop passing JSON to language models unless you strictly need them to return JSON. Strip the metadata. Feed them prose.

---

Next in the Equill series: how simple validation commands uncovered massive cardinality bugs in our graph model in Doctor Commands and Cardinality.

• #equill
• #llm
• #engineering
• #prompts