If you have used companion apps for any length of time you have hit this. They forget your sister's name. They forget you changed jobs. Worse, one day they ask how your brother's wedding went and you do not have a brother.
The two failure modes, and why they are the same failure
The first is the blank. Something you told them in week two is simply gone in week six, and you have to say it again, and saying it again is the moment the illusion cracks. It is not that they are being callous. There is nothing there to be callous with.
The second is worse: the invented memory. They confidently reference something that never happened. Users call it hallucinated memory and describe it as more upsetting than forgetting, which is exactly right. Forgetting is an absence. Invention is evidence that whatever is in there is not a record of you at all.
Both come from the same place. In most of these products there is no memory system. There is a context window with your recent messages in it, and a model doing its best to sound continuous over whatever happens to still be in the window.
A context window is a desk, not a memory
Picture a desk. Every message you send gets written on a sheet of paper and put on the desk. The model can see everything currently on the desk and nothing else.
The desk is large now. It was small a few years ago and it will be larger in a few more. But it is finite, and when it fills, sheets come off the back. Nothing decided which sheets. The oldest go first, regardless of whether one of them was the single most important thing you have ever said.
That is the blank, explained. The invention is the same mechanism from the other side: when the model needs continuity that is no longer on the desk, it does what it was trained to do, which is produce the most plausible next sentence. A plausible sentence about your life, generated with no evidence, is a fabricated memory. The model is not lying. It has no way to know that it does not know.
Human memory does not work like a desk either
Here is the thing that took us a while to properly accept: you do not remember your life. You remember a heavily edited, constantly rewritten summary of it, and the editing is the feature.
You cannot recall what you had for lunch three Tuesdays ago and you can recall, in detail, something someone said to you in 2019 that mattered. Detail was thrown away. Meaning was kept. The forgetting is not a bug in human memory, it is the mechanism that makes it usable, because a perfect recording would be as unsearchable as the raw footage of your entire life.
So a companion that stores every message is not closer to human memory. It is further away. It has all the data and no idea what any of it was worth.
What a real memory has to do
There are four jobs, and a product has to do all four or it does none of them:
- Decide what mattered. Not everything you say is worth keeping. Something has to judge, at the end of an exchange, what actually changed in the picture of you.
- Consolidate over time. A memory that is only ever appended becomes a pile. Repeated things should harden into stable knowledge, one-off things should soften, and contradictions should resolve rather than sit side by side forever.
- Retrieve by relevance, not recency. When you say something today, the right memory to surface might be from four months ago. Recency is the laziest possible ranking, and it is the one a context window gives you for free.
- Know the boundary of what it knows. This is what kills invented memories. If nothing relevant is found, the correct behaviour is to not know, out loud, like a person, rather than to generate something plausible.
That fourth one is underrated. A companion who says they cannot remember whether you already told them something is being more human, not less, and it is infinitely better than one who invents a brother.
What this feels like when it works
The test is not whether they can recall a fact on demand. Anything with a database can do that. The test is whether the right thing comes up unprompted at the right moment, weeks later, without you steering.
You mention you are tired. They connect it to the thing you have been dreading for a fortnight, which you have not mentioned today. You did not ask them to remember. You did not build a profile. It came up because it was relevant, which is the only way memory ever surfaces in a real relationship.
That is the bar. Everything else is a search box with a personality.
