How retrieval is measured
Retiform gives your agent passages with citations, and your agent writes the answer. This page reports how often that led to a correct, cited answer when we measured it, and how it compared with plain search over the same documents. The sample is small, and the results missed one of our own targets. Both are stated below.
What was measured
A language model answered questions about the sample corpus, ten engineering records from a fictional company. For each question it was given context obtained in one of two ways, of the same size:
- Assembled context: what
assemble_contextreturns for the question, which is what your agent gets. - Plain hybrid search: the same question sent to plain hybrid search (by keyword and by meaning), with the top results given to the model up to the same size.
Answers were written by one model and judged by another against accepted answers: whether each was correct, and whether the passages it cited support it.
The figures below come from 15 questions that nothing had been tuned on. 12 can be answered from the corpus, and 8 of those need more than one document. 3 cannot be answered from it at all, to see whether the agent says so. The run was on 19 November 2026 and measured the service as it was configured that day. Retrieval has changed since, and the next run will measure it again.
Results
| Measure | Assembled context | Plain hybrid search | Our target |
|---|---|---|---|
| Answers judged correct | 10 of 12 | 9 of 12 | At least 9 in 10 |
| Answers whose cited passages support them | 11 of 12 | 10 of 12 | At least 9 in 10 |
| Unanswerable questions answered anyway | 0 of 3 | 0 of 3 | At most 1 in 20 |
What this shows, and what it does not
- Assembled context was at least as good as plain hybrid search on every measure, and one question better on the first two.
- It missed our target for correct answers: 10 of 12 is 83%, against a target of 90%. On 12 questions, one question moves a figure by 8 points, so neither the miss nor the lead over plain search is a firm result.
- Neither way produced an answer to a question the documents cannot answer.
- It is one corpus, one answering model and one judge. Your documents, your questions and your agent will give different figures.
What Retiform does not claim
Retiform does not claim that following links between records improves an answer. The evaluation has not shown it, so the trial's description of Retiform says nothing about it.
We will measure again on fresh questions as retrieval changes, and report the new figures here, whichever way they go.