In early October we built a small offline machine we call the Ark: a local language model plus a library of reference material, meant to answer practical questions when the internet is down. It runs entirely on hardware we own.
What went wrong
We tested it with first-aid questions. One answer came back with confident steps and a citation to a source. The source did not exist. The model had written something shaped like a reference, because that is what an answer is supposed to look like.
For most topics a made-up citation is embarrassing. For first aid it is dangerous, because the citation is what makes a wrong answer feel checked.
What we did first: stop presenting it as an authority
Before fixing anything, we stopped treating the Ark's question page as a first-aid authority for the family. The plain library browser on the same machine stayed up, because it shows real source text with no model in between.
Root cause
The agent runtime has a setting that decides when the model must use tools to look things up instead of answering from memory. In "auto" mode it didn't recognize the model alias we had given this box, so it let the model answer from memory. We forced the setting on, and added grounding rules to the system prompt. On our test suite the earlier run scored 18 of 20 with no fabrications and long stalls. After the config change it passed 10 of 10 with no fabrications, in 18 to 35 seconds per task.
Why that still isn't enough for first aid
A model that usually grounds its answers is still a model. So for emergencies we took the writing away from it. We assembled 23 emergency cards directly from the offline library, with the source text copied verbatim from references such as MedlinePlus, WikEM, Ready.gov and the CDC. A checker confirms every card's text actually exists in its source. The cards are still marked as drafts until a person has reviewed them.
The rule we kept
- For safety content, retrieve the real text. Don't have the model rewrite it.
- Check citations mechanically against the source. A citation that looks right proves nothing.
- A passing test suite measures the questions in the suite. Keep adding the ones that scared you.
The Ark is still useful, and it is still a work in progress. The part where a model speaks about your health is off until the cards have been reviewed by a human.