The nineteenth century tried to warn us of what to expect, but obviously we ignored the advice and signs when the moment arrived. But what, for some amongst us, is the Oimpia to their Nathanael? It’s AI, or more specifically, LLMs. While each digital automaton has its own Spallanzani and Coppola as the engineers, they all work towards the same ends: a facsimile of life. Their chief components are the virtual clockwork mechanisms that are the heart of algorithms. But like E.T.A. Hoffman’s tale “The Sandman,” the ending isn’t what you expect.
The prospect of AI is exciting for some, but it is not without its criticisms. Beyond the environmental concerns and resource consumption issues, the ethical questions on how these engines are built, trained, and presented do require greater scrutiny, especially in light of the psychological questions raised in works like those in German Romanticism asked during the Industrial Revolution. At the heart of the issues these contemporary Pygmalions miss however is their mechanical marvels possess a pleasing aesthetic while fundamentally missing what lies beneath the artifice is base materials masquerading as something they are too blind to see—or at least want the world to believe that. In short, they’ve fallen in love with their own reflected egos embedded in the dream they’re selling. Caught in the glamour are their Nathanaels.
I’ve covered the issues concerning the difficulty trying to reconcile the ways in which math and language operate, but those fundamental differences are amplified by this engineering attempt. Let’s engage in a quick exercise before diving deeper into the limitations our Spallanazanis and Coppolas cannot fully resolve to anyone’s satisfaction. Go find and photograph what this symbol represents, not another version, the actual thing it represents: 8. Now read this word aloud: refuse. Did you pronounce it as “refuse” or “refuse?” What even was the context to let you know which definition and pronunciation to use? That is unclear here on purpose and you are more capable of determining which syllable is emphasized. How can a digital gearbox come close when its choices are driven by the pure logic of mathematical formulae?
Now, with that covered, let’s do a brief summary of that nearly 18,000 word essay. While the purpose of “An Unbridgeable Divide” was to explain what leads to arguments about game rules, it fully captured the problems between the absolute values of numbers and equations and the inherently fuzzy nature of languages. Computers can only operate in the realm of pure numbers. The reason you cannot find the physical correspondence to the number I asked for is that all numbers are classified linguistically as adjectives. As descriptive terms in a language, they can only augment nouns while never possessing any weight. Yes, you can use numbers as a noun but limited only to identifying itself or as a collective already identified by its actual noun for clarity.
The subtle shifting between grammatical categories is nothing extraordinary as we routinely use words in varying categories. Verbs as nouns; adjectives as adverbs, nouns, and verbs; and others serve as examples for the slipperiness of words. Math, like logic, doesn’t slip. Equations only have one interpretation, but there is only meaning in why someone uses them. Now, if you are not trained in their use, there’s no meaning, only a string of numbers and symbols. Even for mathematicians, if you remove the variables and apply no language elements to denote what measurements were solved for in the proof, it would be a challenge to correctly identify the formula.
How many definitions of the word “is” are there? In the mid- to late-90s, this was worth a laugh. But, it’s not as facetious as it sounds. It is the most lawyerly thing many of us had ever heard at the time, but there are multiple entries under its parent form: “be.” There are thirty-two definitions in the Oxford English Dictionary. Just this single word has over four pages dedicated to its history and usage. Some of the uses depend on if it’s the main verb or an auxiliary one in the sentence. The similarities are subtle, as the average native speaker doesn’t notice the usage; it’s intuitive as which shading is used. Yet, it does not guarantee the intended definition will be used.
If humans have trouble identifying an equation when the variables are replaced by numbers, how would a system of formulae integrated into the digital deference engine understand the intricate shadings of words? Sorry, my Nathanaels, but your Olimpia isn’t alive. It doesn’t understand you and it can only approximate the response, but its built to fool you just enough as Spallanzani intended. But like Nathanael, Olimpia is just as blind. The automaton can’t see its failures any more than he can. “The Sandman” is a metaphor for the blindness obsession creates—and in this case, the comparison is apt.
See, while a human can determine the definition of the words through context, the machine cannot. Engineers can approximate the intent, but there are no mathematical functions that can capture the fuzziness of language. Linguistics can only calculate the frequency of sounds, use of words, and the number of words needed to make oneself understood in any particular tongue. For English, that’s between 1,000-3,000 words. Nearly 80-95% of daily interactions get by with those common words, with the rest relying on technical terms related to a profession. But, that can’t be explained by the math or tell a computer how to use them, they can only measure what is there.
What isn’t in that data is how the brain can sort through the definitions. Engineers live and die by precision, yet they believe they can trick you and their code into believing it has chosen the correct definition. If you think the machine knows how to use language without the experiences that lead to catachresis, a process by which words gain new meanings but are close enough for the current lexicon to function unaltered but enriched by the experience captured and only some definitions are online (go look at how many entries there are for “be”). Where does Olimpia gain such knowledge? The mechanical doll only knows what its creators can encode in it. Being lifeless, it cannot become aware of those extra meanings, and those are the eyes the Spallanzanis and Coppolas give you to see through until they conspire to pilfer more while needing more digital cogs (e.g. data centers) to keep the fiction alive.
When a machine attempts to experience the world through its own expression what it’s been fed, it malfunctions. The coding and gears are bound to syntax alone. Grammar does work that way. While it does describe how the language ought to work in that point in time, it too yields to environmental and cultural changes. That’s what it needs to do because the point is to share an experience. That is one of the complex ideas necessitating the development of language and part of the focus in “An Unbridgeable Divide.” The depth of complexity and catachresis is why communication is an art. News has always been governed by this principle as it is a retelling of what one witnessed.
People do not report without omission, only video can do that. What people report is what impression the event left them with, which does include factual details. Most news coverage is still filtered through the observer’s senses—their ability to experience the world. Even scientists experience this in their findings. Yes, the scientific process is rigorous to avoid any bias from contaminating the results, but the observation is not free from sensory input. The dry, passive voice and subsequent third-party tests work to further remove any bias from seeping into the results and description of the discovery.
How does any of this apply to LLMs and “The Sandman?” It seems like a stretch or a tortured metaphor, but that is what the technology is: a tortured format using precision to affix fuzzy, imprecise objects into a repetitive, replicative whole. Languages are living objects that grow, evolve, and can die. English alone has more than 500,000 words, with some speculation that with the obsolete words and technical terms used in disparate fields pushing that total to nearly one million. Enter Coppelius and the uncanny valley.
Nathanael perceives of Coppelius as a monstrous, malformed man who brought misery to Nathanael and his siblings. He is the personification of the twisted mind of an inventor driven to create what harms others because he can. In effect, his devices are a misapplication of what they should be used for. Thus, he is described as having fists as both his brutishness and indelicate approach to mechanical design. And, in light of how LLMs function to create a bland, amalgamated standard of business casual English, the darker side of our software engineers emerges. The algorithms kill the power of language by rendering them as dry and sterile as a research paper.
In addition to trying to sound natural, LLMs are subject to procedural generation to summarize what it has been fed. There is no guarantee the data will be accurate or based on accurate sources, such as not including summaries to create a new summary or drawing in material either using homographs or categories related but not the same as the focal topic/subject. This is because no matter how sophisticated the system, it cannot process the slippage or nuance. That’s fuzzy logic and it only works with the experiences an individual can observe or an analogous situation. It’s a process beyond a mechanical system and moreso for one for one based on an on/off switch. Gears slipping in a mechanism is bad for the device and speaks to an approaching malfunction.
The digital equivalent is to throw an exception. It’s a clunky mechanism, it’s slow and shoddy if used routinely. Also, it shows the lack of care a programmer took in devising the software. Imagine how many exceptions need to be thrown to replicate the semblance of sentience that doesn’t feel scripted or follow the strict categorical definitions a computer must use. Also, how can Olimpia incorporate and synthesize new data relating to unwritten cultural/social rules and the ways in which they inform language and provide a lens through which events and experiences are interpreted. Like any automaton, LLMs are frozen in time. They can be fed new data, much of which has been stolen from others who shared their experiences or observations, but nothing new can be generated.
Blurring the lines between the human and the artificial is the goal of these would-be Spallanzanis and Coppolas, and that’s the problem. The Nathanaels fall easy prey to the confusion introduced and may trust to the content Olimpia offers as an accurate portrayal of the world. The modern Nathanaels cannot disambiguate and are caught up in the hallucinations of the digital Olimpias they’ve fallen for. Their gaze has been riveted to an illusion, blinding them to the limitations fo the mechanism as it reduces communication into a rote, mechanical process. The goal is a homogenized use of language into a bland beige spouting data as proof of intelligence, but like Olimpia, it is hollow and wooden. What all involved have failed to see is the lifeless corpse made to dance not for the novelty of experience, but rather the entertaining notion of exercising power over a slavishly devoted servant. The veneer may be one of beauty, but it is to mask mastery over another with that of craft and knowledge.
Nathanael’s death is a metaphorical one. His obsession and blindness are the result of sacrificing his humanity to the object of his desire. In transferring the human to the machine, reality becomes the malleable space rather than that of the experiential realm. The fantasy of the automaton as companion and assistant causes the blindness as a Nathanael projects his own emotions onto the machine. Believing the gears have humanity is a symbolic death of one’s own and contributes to the blindness in the human effort stolen and mimicked by the puppet the Nathanael has become infatuated by with admirations also misplaced on the engineers.