AI Memory Proves Inefficient: Tenure Project Detects 95% Error Rate

While modern AI assistants excel at writing code, analyzing documents, and maintaining long conversations, their memory may be far worse than expected. A new study by researcher Jeffrey Flynt from the University of Texas shows that long-term memory systems used for large language models (LLMs) have fundamental problems. Current systems store data as mathematical vectors and search by semantic similarity, leading to serious errors in finding exact facts. Reported by Ixbt.com report .

According to Flynt, existing benchmarks create an illusion of quality. Typically, the model's final answer is evaluated, not the memory quality. A language model can hide search errors using its logical knowledge. However, if this data is used not for text generation but for tasks requiring precision, such as API calls or infrastructure configuration, the consequences are critical. The PrecisionMemBench test created by the researcher showed that fact-finding accuracy in popular systems like Mem0, Zep, and Hindsight was only 5–8%.

To solve this problem, the Tenure system was proposed. Its key feature is that memory search is viewed not as a search task, but as state management. Tenure uses a structured repository called "beliefs" instead of vague semantic representations. Each entry is a separate fact with a type, scope, and relevance status. The system tracks outdated data and replaces it with new information, without mixing data from different projects.

Tenure abandons vector search in favor of classical methods based on exact term matching. For example, if a user says they are using the Redis database, the system returns exactly Redis, not similar technologies like MongoDB or PostgreSQL. In tests, while vector systems returned 16 unnecessary facts along with one correct answer, Tenure provided only the necessary data with a score of 1.0. Additionally, contexts in the system are strictly isolated, so data from old conversations does not interfere with new tasks.