Mustafa Suleyman, CEO of Microsoft AI, agrees with me!

Mustafa Suleyman, CEO of Microsoft AI and co-founder of DeepMind, said in a recent interview on Defining Intelligence with Seth Rosenberg on YouTube that Microsoft Copilot, and by extension, all AI assistants, must retain a memory of all their conversations. This echoes what I have been saying for over a year. An AI assistant needs to have an episodic persistent memory to remember important details from conversations potentially for years and even decades. As AI assistants gain the power of agency, as they indeed will, they must also retain memories of their interactions with other agents and the outcomes of their actions. 

We recognise that memory is a crucial component of human intelligence, and we have various medical definitions for different types of memory loss. ChatGPT currently has a relatively severe example of anterograde amnesia. OpenAI and Microsoft need to look at case-based reasoning, the branch of AI that has been handling episodic memory since the 1980s. Roger Schank's initial work on scripts laid the foundation for episodic memory management, which was then blended with ML techniques in the 1990s.

Clip from Defining Intelligence with Mustafa Suleyman

A workshop on Case-Based Reasoning and Large Language Model Synergies is being held next week in Mérida, Mexico, with the 32nd International Conference on Case-Based Reasoning (ICCBR 2024).

ChatGPT now has a memory - but it's naive

Screenshot - OpenAI

Last year, during the hype surrounding Large Language Models (LLMs), I published a position paper and wrote in this blog that LLMs, like ChatGPT, would need a persistent memory of their conversations to be most helpful. It's tough to converse intelligently with somebody with no memory. We value old friends so much because we know they recall events relevant to us, both good and bad, going back many years or even decades. 

However, managing that memory responsibly is a nontrivial task. Moreover, if virtual assistants based on LLMs become part of our daily lives, as it seems they will, their memory may have to be maintained over many years, perhaps even decades. I don't believe ChatGPT's memory management will be sufficient for this task.

My research has primarily focussed on case-based reasoning (CBR), a memory-based method. Interestingly, as a research community, we didn't consider how our case-bases (the memory) should be maintained over time. This was because we'd been focused on building systems in the early years of the discipline. Only when the systems matured did we realise our memories needed to be maintained. This happened in the late 1990s and centred on the work of Wilson and Leake, for example, "Categorizing case-base maintenance: Dimensions and directions". This work sparked a new line of research within CBR, leading to "Remembering to Forget" becoming a memorable paper title.

Consider this scenario: you've asked ChatGPT to remember your partner's name and that they like dark chocolate. You subsequently break up and acquire a new partner who prefers milk chocolate. You later ask ChatGPT to advise on buying a present. ChatGPT recommends dark chocolate in a gift box. Its memory is out of date, and the recommendation is inappropriate. The event of breaking up with your previous partner should have triggered a memory management process. These triggers are detailed in Wilson and Leake's paper in a comprehensive framework for maintaining memories. 

OpenAI's memory for ChatGPT is described in a FAQ webpage that is naive in its simplicity. The memory is described as a "notepad" with individual memories jotted down sequentially on it. Users can review and delete individual memories. But this is far too simplistic an approach to manage an AI assistant's memory that may have to span many years. An AI Assistant's memory must be structured, and policies and procedures will be required to manage it. OpenAI and others who build AI assistants with long-term memories should draw upon the expertise of case-based reasoners who have been managing memory for decades. Otherwise, they are in danger of reinventing the wheel.

Coincidentally, I've just been reading Why We Remember by Charan Ranganath. This book provides a fascinating insight into how the brain processes memories and highlights how little we currently know about this crucial aspect of ourselves.

Call for Papers: Workshop on CBR and LLMs

Generated by Gemini

Last year was the most remarkable year in AI that I can recall. Large Language Models (LLMs) like Chat-GPT changed the public perception of AI, and what had previously seemed like science fiction was now a reality. I was only tangentially familiar with LLM research, having been working on emotion recognition in speech with a PhD student. However, last year, I started diving into LLM research in-depth, which, as one commentator said, was like trying to drink water from a fire hydrant, such was the volume of publications through places like arXiv.

I view all problems through a lens coloured by case-based reasoning (CBR), my long-term AI research speciality. I quickly saw synergies between CBR and LLMs where both could benefit from each other's approaches, and I wrote up my initial thoughts and published them on arXiv.

CBR has an annual international conference, and I proposed the idea of a workshop at the conference on CBR-LLM synergies to some colleagues, who all thought this was a great idea and agreed to co-organise the workshop with me. The Case-Based Reasoning and Large Language Models Synergies Workshop will take place at  ICCBR 2024 in Mérida, Yucatán, México on July 1st 2024. The Call for papers can be accessed here, and submissions are via EasyChair. 

A Long-term Memory for ChatGPT

Generated by Gemini

In October last year, I published a short position paper, A Case-Based Persistent Memory for a Large Language Model, arguing that ChatGPT and other LLMs need a persistent long-term memory of their interactions with a user to be truly useful. It seems OpenAI was listening because a couple of days ago, they announced that ChatGPT would retain a persistent memory of chats across multiple conversations. As reported in Wired, the memory will be used to add helpful background context to your prompts, improving their specificity to you over time. I argued in my October paper that the LLM community should look to the Case-Based Reasoning community for help with memory since we are the discipline within AI that has been explicitly concerned with memory for decades. For example, we long ago realised that while remembering is vital, a memory must also be able to forget some things to remain functional. This is a non-trivial problem discussed in Smyth and Keane's 1997 paper Remembering To Forget: A Competence-Preserving Case Deletion Policy for Case-Based Reasoning Systems. The synergies between CBR and LLMs will be the focus of a workshop at ICCBR-24 in July in Merida, Yucatán, México.

CBR and Large Language Models Report on arXiv

 I've just published a report titled A Case-Based Persistent Memory for a Large Language Model on arXiv. The report explores Case-based reasoning (CBR) as a methodology for problem-solving that can use any appropriate computational technique. This report argues that CBR researchers have somewhat overlooked recent developments in deep learning and large language models (LLMs). The underlying technical developments that have enabled the recent breakthroughs in AI have strong synergies with CBR and could be used to provide a persistent memory for LLMs to make progress towards Artificial General Intelligence.

https://doi.org/10.48550/arXiv.2310.08842