The 96% Recall Number Brands Keep Misreading

A small GPT-2 model lifted factual-question accuracy from 9.7% to 96.2% in a controlled biography experiment — and vendors selling "brand memory" packages should stop quoting that number as if it settles anything. That is the core warning in a new analysis by Pedro Dias, published on The Inference, which stress-tests what marketers claim when they say an AI model "knows" a brand.

Dias draws on "Physics of Language Models: Part 3.1," where researchers pre-trained models (training them from scratch) on synthetic biographies, then fine-tuned them (further training on question-answer tasks) on some people and tested others. The distinction matters: a model can predict every fact in a biography and still fail when asked where the person works.

"There is quite a lot hiding in 'knows,'" Dias wrote.

What does repetition actually buy?

The comfortable counterclaim — that duplicate copies teach a model nothing — is also wrong. In "Quantifying Memorization Across Neural Language Models," duplication in the training corpus increased verbatim extraction: given the start of a passage, the model more often reproduced the rest.

But the caveat follows fast. A newer paper, "Don't Forget! Decomposing the Training Dynamics of Memorization in Language Models," used saved Pythia checkpoints (archived model versions) to simulate training updates. Learning from other data worked against predicting frequently repeated passages more than rare ones, especially in the one-billion-parameter model. The researchers suggest repetition may keep passages reproducible despite that interference — yet they did not vary repetition and competing data separately to prove it, and they tested word-for-word continuation, not factual answers.

Dias notes what the mention-counters skip: a repeated publication is not automatically a repeated training exposure. The provider still has to collect the copy, keep it after filtering and use it in training.

Why did accuracy jump 92 points?

In the biography experiment, researchers varied wording, shuffled sentence order and repeated each person's full name. Accuracy on test-group questions rose from 9.7% to 96.2% — synthetic facts, small model, not commercial recommendations.

The sharper finding came second. The team left one group's biographies unchanged, added varied biographies for another group, and trained question-answering only on the added group. Accuracy on the unchanged group rose from 4.4% to 86.8%. Training on material about other people improved answers about this group. As Dias puts it, the pupil got better at using a lesson because the teaching improved elsewhere.

The takeaway in plain words: if similar transfer holds in production, a brand's recall could improve after a model update without anyone rewriting the brand's pages. That is a hypothesis about commercial systems, not a measured result.

Retrieval still beat fine-tuning

A separate study, "Fine-Tuning or Retrieval?" — found that Mistral-7B scored 50.4% on new-fact multiple choice after training on original documents and 58.8% with paraphrased material. Supplying relevant passages at answer time scored 87.5%. That is retrieval-augmented generation, or RAG. Dias flags the design flaw: the questions came from the same text chunks the retrieval system could access, and multiple choice measured recognition, not open recommendation.

"Source Preference in the Wild" adds another complication: with titles and snippets fixed, changing only the displayed source changed the models' selections, and preference training made models favor a source even when competing items met the same measured requirements.

Dias, who works at geoSurge, where model memory is central to the approach, closes on the power asymmetry: publishers control the reading material; providers control the curriculum. Anyone selling webpages as guaranteed model memory is promising an improvement the researchers never tested.