[J]udgment involves choices that are guided by values. These values are acquired through the course of our life experience and are necessarily qualitative: they cannot be captured in code. Calculation, by contrast, is quantitative. It uses a technical calculus to arrive at a decision. Computers are only capable of calculation, not judgment. This is because they are not human, which is to say, they do not have a human history – they were not born to mothers, they did not have a childhood, they do not inhabit human bodies or possess a human psyche with a human unconscious – and so do not have the basis from which to form values.
Ben Tarnoff, in ‘A certain danger lurks there’: how the inventor of the first chat bot turned against AI [theguardian.com], published in The Guardian
A timely article in The Guardian about the life an work of Joseph Weizenbaum, the author of the original Eliza program. Eliza was the first AI for prominence, if not the first, and despite how simple a program Eliza was when we look back from the likes of ChatGPT, it managed to show a fundamental issue: anthropomorphization.
The quote above is part of the article’s summary of Weizenbaum’s book Computer Power and Human Reason. A key tenant of the book is Weizenbaum’s belief that humans are too fast to anthropomorphize AI, to assign human characteristics, especially intelligence to a mere program, a deterministic bit of code written to mimic intelligence. Weizenbaum’s argument is that, a program, no matter how cleaver the programmer, no matter how good the program, can never truly be human. Humans have experiences, that are qualitative and aspect that can never be learned from mere information by a computer that is quantitate in it’s very nature. You can teach a computer the definition of love or heartbreak, but a computer can never experience it. Weizenbaum argues that only a human can, and should, make judgements which necessarily require qualitative experience, while an AI can only ever make computations. He argues that because of this fundamental difference, there are things that computers should not be allowed to do. Or, as Weizenbaum puts it in the introduction to Computer Power and Human Reason, there are limits to what computers ought to be put to do
.
The future that Weizenbaum feared where we outsource judgement to AI has already come to pass in some specific instances. The Guardian article links to a Brookings Institution article [Brookings.edu] on the “widespread use [of algorithmic tools] across the criminal justice system today”. The Bookings Institution describes how computer programs are used to assign risk scores to inmates up for parole, or identify likely crime locations —hello Minority Report— and, of course, the use of facial recognition. While there is still a “human in the loop” in the decision making process today its easy to imagine people, including judges or police, just trusting the machine and in-effect outsourcing their judgement to the AI rather than just the computational tasks.
The Guardian articles is worth the read, it’s more a character study of Weizenbaum and his trajectory from creating Eliza to arguing about the potential pitfalls of AI. The promise of AI, in Weizenbaum’s lifetime faded, as it became apparent that the super intelligent programs promised was out of reach. Both the public and the academic world —and those funding them, like the military-industrial complex— moved on. It’s worth revisiting Weizenbaum’s work in our new hype cycle for AI.
Reading the article reminded me of a paper I wrote in college. AI was one of the subjects I focused on as part of my computer science degree. I built neural networks, language models, and much more. I experienced first hand how easy it was to make something that, at first glance, seemed to embody some intelligence only to find that it became quickly apparent that this was an illusion and that actual intelligence was far away, and seemingly unachievable on the resources of the day. But this was the early days of the internet, more than two decades later the world is different. In the midst of my studies I wrote a paper I titled “Will I Dream” referencing the question that HAL 9000 asked at the end of 2010: Odyssey Two. It was not really a technical paper, more a historical discussion on the dream of achieving “human like intelligence” in a program. I covered ELIZA and several of her descendent, PARRY, the paranoid program, SAM, and others.
I don’t have an electronic copy of the paper anymore, sadly it was on a hard drive that died long ago without a backup. I do have a single print out of it. Maybe I’ll transcribe it and post it here, as outdated as it is.
Looking back at my paper I’m reminded how long the AI journey has been, how the use cases that I covered in 2000, already old then are new again. A good example is FRUMP, a program written in the late 1970’s to read and summarize news stories. Similar to how GPT can be used today to summarize a website. The old goals are the new goals and the hype is reaching a fever pitch again. Will we be disappointed again? Will AI change the world in some fundamental way or will it’s promise fade? Will it be a tool but never the craftsman? Or will it take overall our jobs? Is AI an existential threat or just an amusement, a distraction from the actual existential threats we refuse to face? Meta released a music making AI recently, maybe it can generate a soundtrack for us while the world burns.