AI is impacting us daily. We see tools writing literature and creating art as if designed by their favorite authors and poets. We see whole posts being summarized and replied as if by some magician crafting a delicate string of words that was prompted by less than a paragraph of text. It’s for this cause I am deeply reminded of a thought experiment presented in the 1980s.
14 years ago, I took a course that would finish my Psychology minor for my undergraduate degree in Computer Science. The course was titled Psychology of Consciousness and it was a deep dive on the psychological theories surrounding the phenomenon we call consciousness. Concepts including Cartesian dualism, material monism, idealism, as well as many other concepts deeply related to how we experience consciousness.
The Chinese Room argument relates to AI, and it is a deeply needed thought experiment that might help add another layer to an increasingly complex phenomenon.
In the experiment, a man is requested to produce a string of Chinese words despite not knowing or understanding Chinese from a series of prompts that are written in Chinese.
The man is given a series of books in his native language that show how to manipulate and write Chinese to answer the reply. In no way does the man understand the Chinese characters, he has no understanding of the words being provided him. He is simply shown how to move characters around to converse with the prompt. The man effectively replies to Chinese with Chinese, and for all intents and purposes, a conversation took place that was in Chinese.
The thought experiment (prescient from the 1980s) is an argument that shows that (at current) AI produces language through a series of computerized operators that needs no underlying understanding of the meaning of the language, it needs only to understand how to manipulate language to form meaning. Does that mean AI isn’t useful? Certainly it is useful. But we must always consider this argument. We are talking to a system that we have shown how to manipulate strings of text to respond to us, that is no simple feat, but we are still far from creating the kind of Consciousness that will lead to much larger questions. When that day comes, let’s learn the questions we should be asking.