Factuality is important for making Giant Language Fashions (LLMs) dependable. When a mannequin solutions a factual query incorrectly, is it as a result of the very fact was by no means encoded, or as a result of the very fact is encoded however not accessible? Customary accuracy metrics collapse these circumstances collectively, though they counsel very completely different limitations and really completely different interventions. Encoding failures name for scaling mannequin measurement or increasing information protection, whereas recall failures may also level to post-training and inference-time strategies that assist LLMs higher make the most of what they already encode.
In “Empty Cabinets or Misplaced Keys? Recall Is the Bottleneck for Parametric Factuality”, we introduce data profiling, a behavioral framework that measures each encoding and recall, and use it to look at the underlying bottlenecks of factuality in frontier LLMs (resembling Gemini3 and GPT-5). We then present that many factual errors in frontier LLMs are higher understood as misplaced keys (recall failures), not empty cabinets (encoding failures).
By analogy, we use encoding to indicate parametric illustration of info, recall to indicate retrieving encoded info with out exterior cues, and recognition to indicate figuring out the proper truth when it’s introduced amongst alternate options. To help this evaluation, we introduce WikiProfile, a benchmark of two,150 Wikipedia-derived info, every paired with ten questions that probe encoding, recall, and recognition.
