The Foundation Model (LLM) Transparency Index

A new index compiled by the Stanford University Center for Research on Foundation Models (CRFM) rates the transparency of 10 foundation model companies and finds them lacking. The best, Meta’s Llama 2, only scores 54% across 100 different aspects of transparency. As LLMs become more widespread and embedded into our lives, their transparency includes the computational resources, data, and labour used to build foundation models, the specifics of their architectures and their downstream use. You can read about The Foundation Model Transparency Index here. #LLM

AI already makes decisions that may affect you

We are all familiar with AI making daily decisions, such as what shows Netflix may recommend to us next or what to listen to on Spotify. But AI has crept into our lives and is making important decisions that would affect you more severely, such as allowing you to marry the person you want or getting that mortgage on a new home. An article in The Guardian, where UK officials use AI to decide on issues from benefits to marriage licences, highlights the growing risks of unregulated use of AI by government bureaucracies. AI needs to be regulated to stop bureaucratic creep.

Classic TV Debate on AI & Mind

 In this old TV debate from 1984, John Searle (philosophy professor from Berkeley) and Margaret Boden (AI professor from Sussex) debate AI, intelligence, understanding and consciousness. What is remarkable is the intellectual quality of the TV debate. You'd never see a programme like this today on TV, which has been totally dumbed down. Secondly, Searle's argument, namely the Chinese Room, is still just as relevant to ChatGPT as it was to the comparatively dumb AI of the 80s. Can a computer shuffling 1s and 0s according to a program understand anything?

CBR and Large Language Models Report on arXiv

 I've just published a report titled A Case-Based Persistent Memory for a Large Language Model on arXiv. The report explores Case-based reasoning (CBR) as a methodology for problem-solving that can use any appropriate computational technique. This report argues that CBR researchers have somewhat overlooked recent developments in deep learning and large language models (LLMs). The underlying technical developments that have enabled the recent breakthroughs in AI have strong synergies with CBR and could be used to provide a persistent memory for LLMs to make progress towards Artificial General Intelligence.

https://doi.org/10.48550/arXiv.2310.08842

 What I'm reading: Digital Technologies ITP

I attended the launch of NZ's Digital Technologies Industrial Transformation Plan at Google's HQ in Auckland on Monday evening, presented by the Minister, the Hon. Ginny Anderson. I'd like to say it's a gripping read, but it is rather full of well-meaning statements and not much hard action. It's disappointing to see Artificial Intelligence almost left as an afterthought to the "Future Focus Areas."  Near the end of the document, it says they will "continue exploring the merits of establishing a Centre for Data Ethics by 2025." Note, it doesn't say they will establish this centre, just explore its merits. On the whole really very disappointing that AI that looks set to transform society in unprecedented ways gets so little mention.

What I'm reading: Sparks of AGI

Today I'm introducing a new feature of this blog. Every week I will highlight something I've read in the previous week that may interest readers of this blog. These will vary from books to academic research papers, online articles, to newspaper stories. There will always be a brief discussion about why the recommendation is interesting. 

Let's start "what I'm reading" with a recently published paper from Microsoft Research titled: "Sparks of Artificial General Intelligence: Early experiments with GPT-4" by Sébastien Bubeck et al. The title explains why I find the paper exciting, but its strength lies in the methodology the researchers use to evaluate the performance of GPT-4. You needn't be a computer scientist or ML specialist to understand this paper. The paper is summarised by the authors in this manner: 

"We demonstrate that, beyond its mastery of language, GPT-4 can solve novel and difficult tasks that span mathematics, coding, vision, medicine, law, psychology and more, without needing any special prompting. Moreover, in all of these tasks, GPT-4's performance is strikingly close to human-level performance, and often vastly surpasses prior models such as ChatGPT. Given the breadth and depth of GPT-4's capabilities, we believe that it could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system."

That really sums up why this paper is worth a read. If you are impressed by ChatGPT, you haven't seen anything yet!

ChatGPT reviews my predicitions for AI made in 2012

My book, The Universal Machine, for which this blog was initially created, is partly about AI.  All of the history of computing is, from Ada Lovelace, through Turing to the present day.  The last two chapters of the book deal specifically with AI and make some predicitions.  I did not anticipate ChatGPT and the rapid rise of generative AI.  However, I thought "why not ask ChatGPT how prophetic my book was given a decade has passed since its initial publication".  Here is part of my conversation with ChatGPT:

Ian: Would you agree that in light of recent developments in AI, the book, The Universal Machine, was prescient.

ChatGPT: As an AI language model, I do not hold opinions or beliefs, but I can provide information on recent developments in AI and their relevance to Ian Watson's book "The Universal Machine".

Since the publication of the book in 2012, there have been significant developments in AI, including breakthroughs in deep learning and reinforcement learning algorithms, the emergence of new AI applications in areas such as natural language processing and computer vision, and the development of AI-powered autonomous systems in industries such as transportation and manufacturing.

In light of these recent developments, it can be argued that "The Universal Machine" was prescient in its analysis of the potential benefits and risks of AI.  For example, the book highlights the potential for job displacement and the need for new models of education and training to prepare workers for a changing job market, an issue that has become increasingly relevant in recent years.

Moreover, the book's discussion of the risks associated with autonomous weapons and the need for ethical guidelines for AI research and development are topics that have gained significant attention in recent years, with many experts and organizations calling for greater oversight and regulation of AI technologies.

Overall, while the field of AI continues to evolve rapidly, "The Universal Machine" provides valuable insights into the history and future of computing and AI, and remains a relevant and insightful read for anyone interested in understanding the impact of these technologies on society.

Well, I can't disagree with that!