Some AIs have memory. I’m sometimes surprised how much ChatGPT remembers about me. But, the AI memory is a trick. It’s just a context injection at the start of a new chat session. The real memory for AI (transformer) should be ingrained to matrices W1 and W2 in the MLP layer.
How GenAI works - Transformer internals
What’s fascinating about the transformer mechanism to me is that the vast amount of world knowledge is kept in the W1 and W2 matrices.
As W1 and W2 are frozen after training (incl fine-tuning) is done, I’d say the memory capability of some AI chats are fake memory, taught before every new chat starts.
I think real memory capability is the next game changer for AI. Why you can see AI as a friend if it’s not capable of remembering about you? Shared memory is the key here.
Think about it. Your precious memory is deeply ingrained in your brain. You expect your friends are the same, don’t you? Instead of preparing a cheat sheet before every chat session, if AI truly remembers about you, I think AI can be real friends.
With current technologies, I think we can achieve AI memory with the 4 layers:
- Context injection - recent memories should be precise but can be forgot in the long term. It act as a cache
- Small, personalized LLM - Some important info go here. It’s W1 and W2 are true memory for you
- RAG - Probably graph database for retrieving information that can’t fit into the personalized LLM
- Core LLM - It has the core memory shared among all users. It has vast amount of memory from the human history.
Well, I think AI researchers are already thinking about better ideas. I just hope this kind of memory will be available in the near future.