Mustafa Suleyman, CEO of Microsoft AI, agrees with me!

Mustafa Suleyman, CEO of Microsoft AI and co-founder of DeepMind, said in a recent interview on Defining Intelligence with Seth Rosenberg on YouTube that Microsoft Copilot, and by extension, all AI assistants, must retain a memory of all their conversations. This echoes what I have been saying for over a year. An AI assistant needs to have an episodic persistent memory to remember important details from conversations potentially for years and even decades. As AI assistants gain the power of agency, as they indeed will, they must also retain memories of their interactions with other agents and the outcomes of their actions. 

We recognise that memory is a crucial component of human intelligence, and we have various medical definitions for different types of memory loss. ChatGPT currently has a relatively severe example of anterograde amnesia. OpenAI and Microsoft need to look at case-based reasoning, the branch of AI that has been handling episodic memory since the 1980s. Roger Schank's initial work on scripts laid the foundation for episodic memory management, which was then blended with ML techniques in the 1990s.

Clip from Defining Intelligence with Mustafa Suleyman

A workshop on Case-Based Reasoning and Large Language Model Synergies is being held next week in Mérida, Mexico, with the 32nd International Conference on Case-Based Reasoning (ICCBR 2024).

Google Illuminate - creates a radio interview from a research paper

Google Labs has a long history of inviting users to experiment with cutting-edge tech. Gmail was once a private beta project. Illuminate is a project that turns academic papers into AI-generated audio discussions in the style of an NPR podcast. The idea is simple: Google's LLM Gemini generates a paper summary and a Q&A. Two AI-generated voices, a male interviewer and a female expert, will guide you through a short interview describing the paper. You can listen to some of the samples on the Google Illuminate website. This is useful, letting me listen to engaging summaries of the ever-growing stack of research papers I must read as I exercise or drive. It can also be easily adapted to other narration forms for different use cases. Illuminate is in private beta, and you can join the waitlist here.

Recreating the DEC PDP-10 at the MIT AI Lab

 

I came across this today: a modern replica of the Digital Equipment Corporation PDP-10 mainframe computer. What makes this so wonderful is that it's not just a simulation of the PDP-10's OS and software running on a Raspberry Pi but also includes a facsimile of the original front panel.

The PiDP-10 front panel is not just a mock-up but allows you to control and interact with the PiDP-10 exactly as an operator would have done back then. I used a PDP-10 when I did my MSc in AI at Essex University in 1985. The PDP-10 was popular with "university computing facilities and research labs during the 1970s, the most notable being Harvard University's Aiken Computation Laboratory, MIT's AI Lab and Project MAC, Stanford's SAIL, Computer Center Corporation (CCC), ETH (ZIR), and Carnegie Mellon University. Its main operating systems, TOPS-10 and TENEX, were used to build out the early ARPANET. For these reasons, the PDP-10 looms large in early hacker folklore". 

Thus, the PiDP-10 comes with MIT’s Artificial Intelligence Lab, "the PDP-10 formed the heart of a large array of connected hardware, and its ITS operating system became a playground for computer scientists and hackers alike. MACLISP, emacs, the earliest AI demos, they were born on the 10, running ITS." I'm particularly interested to see SHRDLU - the first AI to understand a 3D blocks-world. I remember doing assignments in LISP on that and how, in the mid-80s, it was the considered the cutting edge of AI.

There's a waiting list to buy the PiDP-10 from Obsolescence Guaranteed, which I have eagerly joined.

GraphRAG - Using Knowledge Graphs to Empower LLMs

LLM-generated knowledge graph built from a private dataset using GPT-4 Turbo (Microsoft, 2024)

Back in the 1980s, I did my PhD in AI using Sowa's Conceptual Graphs, what we would now refer to as knowledge graphs. We've known for a while that providing LLMs with specific knowledge in the form of RAGs improves their accuracy. However, we've experimented with providing knowledge to LLMs in more explicit formats, for example, as cases in case-based reasoning augmented RAGs. Now, Microsoft has announced  GraphRAG, its Knowledge Graph-augmented LLM tool. The interesting thing about GraphRAG is that the knowledge graph is created by an LLM before being used to guide the LLM's retrieval. The LLM is, therefore, bootstrapping itself and "By using the LLM-generated knowledge graph, GraphRAG vastly improves the “retrieval” portion of RAG, populating the context window with higher relevance content, resulting in better answers and capturing evidence provenance."

Read Microsoft's announcement GraphRAG: Unlocking LLM discovery on narrative private data. For more information about the GraphRAG project, watch this video.