Why Hasn't Anyone Solved AI Memory?
Harrison Chase says the hardest part of AI memory is the intelligence that decides what to keep, not the database. A context layer for agents is an agent wrapped around a database, and the data market is still early. ClickHouse is doing incredibly well and stays largely focused on a traditional database, storing and querying data in traditional ways. ClickHouse existed before AI. ClickHouse, Mongo, and Elastic are great for data that software systems produce. Ingesting learnings still needs intelligence, and that intelligence is really hard. A number of companies store memory in a single file. The hard part is taking observations from a user in traditional software, or from a user with an agent, and distilling them into information valuable enough for the context layer.
Large language models still struggle at meta-reflection: reflecting on prompts and improving prompts, and reflecting on agent architecture and improving agent architecture. Engine, the agent running on things, already has some benchmarks, and an internal benchmark for meta-reflection should still be built. Meta-reflection is difficult for agents and generally hard for humans too. He is unsure whether people understand what goes on inside the brain, and brains remain complex things people do not fully understand. Models are not great at reflection, and the inputs, outputs, harness, and workflows are still unknown. Models will probably be meaningfully better at reflection in 12 months.
