As the essays throughout this book demonstrate, the Wachowski Brothers designed The Matrix to work at many levels. They carefully thought through the film’s philosophical underpinnings, religious symbolism, and scientific speculations. But there are a few riddles in The Matrix, aspects of the film that seem nonsensical or defy the laws of science. These apparent glitches include:

• The Bioport—how can a socket in your head control your senses? How can it be inserted without killing you?

• The Red Pill—since the pill is virtual, how can it throw Neo out of the Matrix?

• The Power Plant—can people really be an energy source?

• Entering and Exiting the Matrix—why do the rebels need telephones to come and go?

• The Bugbot—what’s the purpose of the bugbot?

• Perceptions in the Matrix—how do the machines know what fried chicken tastes like?

• Neo’s Mastery of the Avatar—how can Neo fly?

• Consciousness and the Matrix—are the machines in the Matrix alive and conscious? Or are they only machines, intelligent but mindless?

This essay addresses these questions and shows how these seeming glitches can be resolved.

THE BIOPORT

Can the machines really create a virtual world through a bioport? And how does it work? The bioport is a way of giving the Matrix computers full access to the information channels of the brain. It is located at the back of the neck—probably between the occipital bone at the base of the skull, and the first neck vertebra. Wiring would best enter through the soft cartilage that cushions the skull on the spinal column, and pass up through the natural opening that lets the spinal cord into the skull. This avoids drilling through bone, and maintains the mechanical and biological integrity of the skull’s protection. A baby fitted with a bioport can easily survive the operation.

The bioport terminates in a forest of electrodes spanning the volume of the brain. In a newborn, the sheathed mass of wire filaments is pushed into the head through the bioport. On reaching the skull cavity, the sheath would be released, and the filaments spread out like a dandelion, gently permeating the developing cortex. Nested sheaths would release a branching structure of filamentary electrodes. As each sheathed wire approaches the surface of the brain, it releases thousands of smaller electrodes. In the neonate, brain cells have few synaptic connections, so the slender electrodes can penetrate harmlessly.

With its electrodes distributed throughout the brain, the Matrix could deliver its sensory signals in either of two places: at the sensory portals or deep inside the brain’s labyrinth. For example, vision could be driven by electrodes on the optic nerves where they enter the brain. Artificial signals would then pass into the visual cortex at the back of the brain, which would handle them as if they had come from the eyes. Correspondingly, outgoing motor nerves would also have electrodes at the boundary of brain and skull. This simple design mirrors the natural state of the brain most closely. It is not, however, the only possibility. Electrodes could alternatively be attached in the depths of the brain, beyond the first stages of the visual cortex. This would greatly simplify the data processing. In normal perception, most of the incoming information isn’t processed; information you aren’t paying attention to is filtered out. If the Matrix were to deliver information directly to the output axons from the sensory cortex—as opposed to the input to the cortex—then it would save itself the job of filling in all those details.

One scene tells us which method the Matrix uses. When Neo wakes and finds himself in a vat, he pulls out the oxygen and food tubes, drags himself out of the gelatinous fluid, and—perceives the world. The fact that he can see and hear proves that the visual and auditory cortices of his brain are working. This wouldn’t be possible if the Matrix had put its sensory data into the deeper centers of his brain. For then his sensory cortex would have been bypassed: it would never have received any stimulation, and would have wasted away. In that case, Neo would wake from his vat and find himself blind and deaf, with no sense of smell or taste, no feeling of touch or heat in his skin, no awareness of whether he was vertical or horizontal, or where his arms or legs were. The Matrix must have input its visual data just where the optic nerve from the eyeball passes into the skull, rather than in the midst of the brain’s vision processing. Likewise, Neo’s ability to walk and use his arms shows that the motor cortex is also developed and functioning. Indeed, even the cerebellum, which controls balance, must be working. So, the Matrix must be capturing its motor signals from the brain’s efferent nerves after they have finished with the last stage of cortical processing, but before the nerves pass out of the skull.

The rebels use the bioport to load new skills into their colleagues’ brains—writing directly into permanent memory. The Matrix itself never implants skills in this way; folks in the virtual world learn things in the usual manner by reading books and going to college. So, why did the architects of the Matrix build into the bioport this capability to download skills? It is actually a byproduct of how the bioport is installed. They could have attached electrodes to just the sensory and motor nerve fibers. That, though, is difficult: the installer must predict where each nerve fiber will be anchored, which is hard to do reliably, given the plasticity of the neonate brain; and it must navigate through the brain tissue to find these sites. A more robust and adaptable method is to lay a carpet of electrodes throughout the whole brain, and let the software locate the sensory and motor channels by monitoring the data flows on the lines.

That spare capacity remains available for others to exploit, and the rebels use it to download kung-fu expertise into Neo’s brain and to implant helicopter piloting skills into Trinity’s. If the Matrix ever learned this technique, it could create havoc for the rebels, implanting impulses to serve its own ends.

THE RED PILL

Morpheus offers Neo the choice of his lifetime, in the form of the famous red and blue pills. But what can a virtual pill do to a real brain? We have seen that the Matrix interacts with the brain only in the sensory and motor nerve fibers. It does not affect the inner workings of the brain, where a real psychoactive chemical would have to act. Minor analgesics such as aspirin would work by having their effect outside the brain centers, canceling out pain inputs from the avatar software.

The blue pill is probably a placebo. Morpheus says only, “You take the blue pill and the story ends. You wake in your bed and you believe whatever you want to believe.” We never know what, if anything, the blue one would do.

So, how does the active pill, the red one, work? Since virtual aspirin can work as a painkiller, the avatar’s software module must be able to accept instructions to cancel out any given sensory input. Evidently, the red pill gives the avatar a blanket command to cancel all such input. It thereby obliterates Neo’s perception of the virtual world, which the Matrix has been feeding to him throughout his life. Instead of sitting on a chair in a hotel room, Neo sees and feels for the first time that he is immersed in a fluid. The perception of this filters through into his perceptions of the Matrix’s own imagery. Neo touches a mirror, and finds it a viscous fluid that clings to his finger and then seeps along his arm, covering his chest and slithering down his throat. A blend of bodily perceptions and mental imagery is typical of what happens when you wake from a dream; external perceptions are distorted to fit the contents of the dream. Your dream of falling off a cliff might fade into falling out of bed. In the film, the liquefied mirror is seen only by Neo, not the others in the room. His real bodily sensations are, for the first time, sweeping into his brain, which struggles to integrate them into the stable narrative he has lived in up to that moment.

Another route out of the Matrix, besides the red pill, would be meditation. The Buddhist practice of vipassa¹ gives adepts penetrating insights into their own mental processes. It rolls back the barrier between conscious awareness and the subconscious. An adept of vipassana, living in the Matrix, would discover the interface between the Matrix’s electrodes and the brain’s wetware. The expert practitioner could override the Matrix’s stream of imagery, and see reality. Morpheus mentions that someone did break free from the Matrix. Perhaps meditation was the key. To attain that expertise, however, would take years of effort. Leading other people to the truth would require a school of meditation to train new recruits for years, to pursue what one individual claimed was the truth, but everyone else dismissed as fantasy. No doubt this is what the Oracle is gently encouraging. But it is not surprising that the red pill was invented as a fast-track route.

Morpheus’s team monitors Neo’s progress. As he realizes that he is immersed in fluid, Neo panics, and his instinct to escape drowning compels him to drag the tubes out of his mouth. Like waking out of a dream, Neo finds the sensible world rushing in on him, and it is remarkable that his manual coordination has been so well preserved by the Matrix system. He grabs the tubes and yanks them out, using weak hands that had never before grasped anything.

When Neo’s exit from the Matrix is detected, a robot inspects him and flushes him out of his pod. Too weak to swim, he must be pulled out of the wastewater pool without delay. How are the rebels to find him? In a power plant vast enough to house the human race, there would be thousands of effluent drains. As Morpheus mentions to Neo, “the pill you took is part of a trace program.” Besides canceling Neo’s sensory inputs, the red pill also puts a unique reference signal onto the Matrix network. When the Nebuchadnezzar‘s computer locates that signal, it can work out Neo’s physical location and order the hovercraft to the appropriate chute. In the tense moment before that reference signal is located, the worried Morpheus says, “We’re going to need the signal soon,” and Trinity exclaims that Neo’s heart is fibrillating as the panic threatens to bring on a heart attack. Apoc finds the reference signal just in time, before Neo’s brain disengages from the Matrix network and the signal vanishes.

THE POWER PLANT

During the armchair scene, we have what is probably the most criticized element in The Matrix story line. Morpheus claims that the human race is imprisoned in a power station, where human bodies are used as a source of bioelectricity. This is engineering nonsense; it violates the fundamental law of energy conservation. The humans would have to be fed, and the laws of physics demand that the energy consumed as food must be greater than the energy generated by the human body. That Morpheus has misunderstood what is going on is underscored by his mention in the same speech of the machines’ discovery of a new form of nuclear fusion. Evidently, the fusion is the real source of energy that the machines use. So what are humans doing in the power plant? Controlled fusion is a subtle and complex process, requiring constant monitoring and micromanaging. The human brain, on the other hand, is a superb parallel computer. Most likely, the machines are harnessing the spare brainpower of the human race as a colossal distributed processor for controlling the nuclear fusion reactions. (Sawyer comes to a similar conclusion elsewhere in this volume—Ed.)

ENTERING AND EXITING THE MATRIX

The virtual world of the Matrix is not bound by physical laws as we know them, but for the virtual world to be consistently realistic, the laws of physics must be followed where they can be observed by humans. Access into and out of a virtual world is a problem, because materializing and dematerializing violate the conservation of mass and energy. Furthermore, whatever was previously in the space occupied by the materializing body must be pushed out of the way; and would be pushed with explosive speed if the materialization is instantaneous. Conversely, on dematerialization, the surrounding air would rush in to the vacated space with equal implosive force. There are no such explosions and implosions in The Matrix, so how do the rebels do it?

In the Matrix computer, software modules represent the observable objects in the virtual world, and these modules interact by means of predefined messages. One such message issued by a virtual human body, or “avatar,” is, “What do I see when I look in the direction V?” A module whose object lies on the line of sight along V will respond with a message specifying the color, luminosity, and texture that the human should see in that direction. If a rebel’s avatar is to be visible to other people who are immersed in the Matrix world, the Nebuchadnezzar‘s computer must pick up those “What-do-I-see” requests and reply with its own “You-see-this” message.

A virtual human body does not send “What-do-I-see?” message to all other modules in the Matrix, or else it would overload the network. It refers to “registers” of modules, which record the virtual objects’ shape, size, and position. Simple geometry then tells it which modules to target. For efficiency, each visible volume of space, such as the room of a building, has its own register.

The key step in materializing a body in a given space is for its module to be inserted into that space’s register. For dematerializing, it is deleted from the register. Once it is registered, anyone looking in that direction will see that module’s virtual body. The Matrix cannot let a software module insert itself arbitrarily into a register, since that could violate the conservation of mass if it led to an object’s materializing in an area that has a conscious observer.

Registers for unobserved spaces are not constrained in this way. If nobody is watching a room and its entrances, then a body can safely materialize in it without observably breaking the simulated laws of physics.

This does not mean that the laws of physics break down as soon as all observers leave a room. The table and chair do not start to float around against the law of gravity when nobody is looking. Rather, the Matrix simply does not bother to run the simulation for a room that nobody is looking at. In its register, it retains details of where each object is, but the room is no longer rendered as visual and tactual imagery.

So, when the Nebuchadnezzar‘s computer wants to materialize a rebel, it must find some unobserved room, and insert the data module for the rebel’s body into the register for that room. Subsequently, if someone else enters the room, he will see the rebel just like any other object in the room. And the rebel can walk out of the room into any other part of the Matrix world in the normal manner. This is how rebels materialize in the Matrix without causing explosions or breaching the integrity of the simulation.

When a rebel exits, the module that simulates her body is deleted from the register. This must happen only when the body is not being observed. There is, however, an intermediate state, “imperception,” which effectively takes the body out of the virtual world even while the data module is still in the register. This is an emergency procedure that the Nebuchadnezzar‘s software uses for fast escapes.

Although the Matrix software cannot insert or delete a module while its object is being observed, it does allow any module to change its appearance. The agents use it whenever they enter the world. An agent never materializes or dematerializes, but changes the appearance of another person’s avatar to match the personal qualities of the agent.

To make a rebel imperceptible, the Nebuchadnezzar‘s computer changes the body’s visible appearance to be transparent; and the body’s mechanical resistance to that of the air. From an observer’s perspective, the body has melted into air. From a software perspective, the data module is still on the register but simulating a body indistinguishable from thin air. Later, when the scene is no longer being observed by anybody, the module will be deleted.

We see this happen only once, when Morpheus leaves the subway. Once the Nebuchadnezzar‘s computer has located his avatar, it sends an instruction to make it invisible. This does not affect the whole avatar at once: the module has to calibrate its appearance to match exactly its surroundings. The first part of the body to receive the instruction is the nervous tissue of the ear, and this at first glows bright white, before settling down to a state of transparency. The rest of the body follows. Its appearance oscillates around whatever is visible in the background, settling down to transparency: where the Morpheus stood, we see the background shimmer momentarily. The solidity of the body then fades: moments after Morpheus’s body has become invisible, the telephone handset that had rested in his hand drops, slowly at first, toward the ground. The observed sequence is consistent not with the sudden deletion of the body’s module, but rather with its changing its appearance.

HARD LINES

Telephones play a key role in entering and leaving the Matrix. But the rebels do not travel through the telephone lines as energy pulses. There is no device at the end of the telephone for reconstructing a human body from data: all you would get is noise in the earpiece. Furthermore, the bandwidth of a telephone line is too narrow to ship an entire human being. Finally, nothing at all ever really travels along the lines in the Matrix world, as they are only virtual.

Instead of being a conduit for transporting dematerialized rebels, the telephone line is a means of navigation. It pinpoints where a rebel is to enter or leave the Matrix.

To enter the vast Matrix requires specifying where the avatar is to materialize. To get an avatar into the Matrix world, the rebels must use some strictly physical navigation. This is done with the telephone network, which has penetrated every corner of the inhabited world with electronic devices, each of which has a unique, electronically determined label. Without knowing anything of human society and its conventions, the physics modules of the Matrix can determine where any given telephone number terminates.

How are the rebels to give a telephone number to the Matrix? They must dial it, but they cannot simply pick up a handset and make a call to a number inside the Matrix world, for any handset in the Nebuchadnezzar is connected to the real world telephone network, not the Matrix’s virtual network. Inside the Matrix, a call must be placed subtly, without observably breaching the simulated laws of electromechanics.

To see how this can be done, we need to know something of the infrastructure of the Matrix. Monolithic computer systems are unreliable, so the Matrix is instead an assemblage of independent modules, each having a unique “network address.” For a module to communicate with another, it will put a data message on the network with the address of the intended destination. Neither module need know where the other one is inside the electronic hardware of the Matrix computer. They might be inches apart, or a mile away.

This scheme is robust and flexible. There is no central hub, and individual modules can be plugged into, or taken out of, the network without disturbance. Conversely, the rebels can easily hack into it. Once they are linked into the network, their equipment can simply pretend to be another module. It can place data messages onto the system, which will be routed just like authentic messages, and be received and read by the addressed module. So, to initiate a telephone call, the crew will place a data message on the network, addressed to any module that simulates an aerial for receiving calls from cell phones. Some such node will pick up and read the counterfeit data message just as if the message had been sent by a bona fide source. On getting this message, the aerial module will carry out its role in handling a telephone call.

The Nebuchadnezzar‘s operator maintains contact with rebels who are in the Matrix even while the hovercraft is moving, so they must use radioports onto the network. The rebels might have installed their own rogue radio receiver—mechanically securing it in some dark corner, and plugging its data cable into a spare socket of a router. More likely, the Matrix itself uses radio as part of its network infrastructure, and the rebels broadcast their counterfeit messages on the same frequency.

Materializing or dematerializing, however, needs a network address, which is gotten as follows. When the Nebuchadnezzar makes a “phone call” into the Matrix, it places on the network a packet saying “Place this call for (212) 123-4567” or whatever the telephone number is, together with the Nebuchadnezzar‘s own network address as a return label, such as 9.54.296.42. When the call is picked up, the Matrix will return a data packet, addressed to the Nebuchadnezzar, saying “Message for 9.54.296.42: call connected to telephone (212) 123-4567.” All the Nebuchadnezzar‘s computer has to do is listen out for its own address, and it will find attached to it the network address of the telephone equipment.

As soon as the answering machine picks up the incoming call, the Nebuchadnezzar will get the network address of that destination.

Essentially the same job must be done when a rebel leaves the Matrix world. In order to disengage the rebel from his or her avatar, the Nebuchadnezzar‘s computer must again get a fix on the avatar’s location within the virtual world. As before, it is not enough to locate the avatar’s virtual body in terms that relate to human culture. It is no use to say that Neo is at 56th and Lexington. Rather, it needs a network address that the Matrix’s operating system can follow. Of course, the Nebuchadnezzar gets it by calling a telephone in the Matrix world, which must be answered for the network address to be passed back to the Nebuchadnezzar. Once that has happened, the avatar’s module can be deleted from the register for that location.

Why don’t the crew navigate their exits with the stylish cell phones that all the rebels carry? Why hunt for a land line (called a “hard line” in the film) under hot pursuit from the agents? The answer is that the cell phones are not part of the Matrix world and do not have network addresses known to the Matrix software. The cell phone is projected into the Matrix world by the Nebuchadnezzar‘s computer, 114 peter b. lloyd along with the avatar’s body and clothes—and the weapons that Neo and Trinity eventually bring in with them. The software that simulates the cell phones is running inside the Nebuchadnezzar‘s computer, not the Matrix’s computer, so the rebels must find a land line—which are somewhat scarce in an era when everyone has a cell phone.

THE BUGBOT

Before Neo is taken to meet Morpheus, the agents insert a robotic bug into him. Trinity extricates the bugbot before it can do any harm. But what was the bugbot for? Given that it operates inside the human body, the bugbot should be as small as possible. Yet, it is clearly much bigger than a miniature radio beeper needed for tracking Neo’s whereabouts. Trinity says that Neo is “dangerous” to them before he is cleaned. We can infer that the bugbot is actually a munition, probably a semtex device that will detonate when it hears Morpheus’s voice, killing both Neo and Morpheus and everyone else in the room.

Just before it is implanted, the bugbot takes on the appearance of an animate creature, with claws writhing. Yet, after Trinity has jettisoned it out of the car window, it returns to an inert form. It is another illustration of the agents’ limited use of the shapeshifting loophole in the Matrix software, that lets an object transform its properties under programmed commands.

PERCEPTIONS IN THE MATRIX

At dinner on the Nebuchadnezzar, Mouse ponders how the Matrix decided how chicken meat should taste, and wonders whether the machines got it wrong because the machines are unable to experience tastes.

A nonconscious machine cannot experience color any more than taste. A computer can store information about colored light, such as a digitized photograph, but it does so without a glimmer of awareness of the conscious experience of color. The digitized picture will evoke conscious colors only when someone looks at it. All other sensations that you can be conscious of will elude the digital computer.

The feel of silk, the texture of the crust of a piece of toast, feelings of nausea or giddiness: these are all unavailable to insentient machines. This being so, Mouse could have doubted whether the Matrix would know what anything should taste, smell, look, sound, or feel like.

But the Matrix doesn’t need to experience the perceptual qualities to get them right. As we have seen, the Matrix feeds its signals into the incoming nerves where they enter the brain, not into the deeper nerve centers. So when you eat (virtual) fried chicken inside the Matrix, the Matrix will activate nerves from the tongue and nose, and the brain will interpret them as taste sensations. What the Matrix puts in will be a copy of the train of electrical impulses that would actually be produced if you were eating meat. Because of the way that the Matrix has been wired into the brain, it has less freedom than Mouse assumed. Whilst the Matrix cannot know tastes itself, it can nonetheless know which chemosensory cells in a human’s nose and mouth yield the requisite smell and taste.

NEO’S MASTERY OF THE AVATAR

For purists of science-fiction plausibility, Neo’s superhuman control over his avatar body is a troubling element in the film. The final triumphal scene, where Neo flies like Superman, has especially come under criticism. But is it completely at odds with what we have inferred about the Matrix? And how does Neo transcend his human limits?

The Matrix interacts with the brain, but the brain in turn affects the body. When Neo is hurt in training, he finds blood in his mouth. He asks Morpheus, “If you are killed in the Matrix, you die here?” and gets the cryptic reply: “The body cannot live without the mind.” But it cuts both ways; ultimately, Neo’s avatar is killed inside the Matrix, causing the vital functions to cease in his real body.

Mental states and beliefs can affect the body in several ways. In the placebo effect, the belief that a pill is a medicine can cure an illness; in hypnosis, imagining a flame on the wrist can induce blisters. In total virtuality, the mind accepts completely what is presented. If the Matrix signals that the avatar’s body has died, then the mind will shut down the basic organs of the heart and lungs. Actual death will inevitably ensue, unless fast action is taken to get the heart pumping again.

In the climactic scene, Agent Smith kills Neo’s avatar within the Matrix. Neo’s brain accepts this fate: it stops his heart and loses conscious awareness. His real brain, however, retains enough oxygenated blood to keep it functioning for approximately three minutes, after which it would begin to suffer irreversible damage and, a few minutes later, brain-death. During this time, the auditory cortex keeps on working and digests what Trinity says, albeit unconsciously. Trinity’s message is comprehended by Neo’s subconscious mind, and a deep realization that the Matrix world is illusory crystallizes in his mind. At an intellectual level, Neo already believed this, but now he knows it at the visceral level of the mind, the level that interfaces with his physiology. Empowered by the insight that his avatar’s death is not his death, Neo regains control of his avatar —not only resurrecting it but attaining superhuman powers: the avatar can stop bullets, and fly into the air.

Neo’s new powers contrast with the rigid compliance with simulated physical laws that the Matrix generally adheres to. It reveals that Neo has gained direct access to the software modules that simulate his avatar body. That raises two questions: Why does the avatar software accept commands to transform itself, when normally it strictly follows a physical simulation? And, how can Neo’s brain issue such commands, which are obviously outside the scope of the normal muscular signals?

The software that simulates the avatar must have a special port, intended for use only by agents, which accepts commands to change the internal properties of the avatar’s body. Agents use this facility to embody themselves in human avatars. Like all software, the avatar will obey such commands wherever they originate, provided that they are correctly formulated. We saw earlier how the Nebuchadnezzar‘s computer used this transformative power to make Morpheus disappear from the subway station. Now Neo’s brain is directly using the same command port.

Commands to transform the body cannot travel on the wires that carry the regular muscular signals from the brain to the avatar module. So, they use some of the many other, seemingly redundant, data lines that terminate throughout the rest of the brain. That those lines are hooked up at all on the Matrix end is a spin-off from the Matrix architect’s use of general-purpose interfaces. When a newborn human baby is connected to the software module that runs its avatar, there is no way to predetermine which wires carry which data streams. So, at the Matrix end, each line is free to connect to any data port of the avatar module. Some data ports emit simulated signals from virtual eyes and other sense organs, and they will connect with the brain’s sensory cortex; others will accept motor commands to carry out simulated contractions of virtual muscles, and they will link up with the motor cortex. In a feedback process that mirrors how the natural plasticity of the brain is molded to its function, useful connections are strengthened and the useless are weakened. As a baby grows into an infant, it gains feedback through using the simulated senses and muscles of the avatar, and therefore its brain builds up the normal strong connections to the conventional input and output channels. But it lacks the abstract concepts needed to use the special port that accepts transformation commands. So the brain’s connection with those lines atrophies. Nevertheless, the hardware for that potential connection remains in place. In Neo’s kung fu training, his brain rediscovers the abandoned data lines, and he starts to issue rudimentary transformations, giving his avatar’s muscles superhuman strength. Only with the deep insight that he gains from being woken after his avatar’s death, does he acquire the mental attitude needed to harness that transformative function fully.

The existence of the transformational back door into the avatar software is a security hole that the architects of the Matrix never imagined would be used by mere humans—but now it threatens the very existence of the Matrix, as Neo exploits the power it gives him.

CONSCIOUSNESS AND THE MATRIX

The last question I will address in this essay is a complex one, and one that continues to be explored and debated in scientific and philosophical circles. Can machines be conscious? In everyday life, the 118 peter b. lloyd machines are so dumb that we can ignore this question, and so we do not have an established criterion for judging whether the intelligent machines of science fiction are conscious. How similar must a machine be to a human for it to be conscious? Humans have a cluster of properties that always hang together: they have conscious perceptions and emotional feelings, they have opinions and beliefs, intuition and intelligence, they use language, and they are alive and warm-blooded, and have a biological brain. We do not, in everyday life, have to separate out those concepts and decide which ones are necessary and sufficient for sentience. The properties all come as a package. In contrast, the lower animals are like us but do not use language and are not as intelligent as we are. So, it is believed that the higher animals probably have basic conscious perceptions—such as colors and sounds, heat and cold—much as we do, but they lack the superstructure of thought. But what about machines that are intelligent and use language, but are not made of biological tissue? Could they be conscious? To respond rationally to this emotive challenge, we need to be clear about the ideas that are involved. The commonest and most damaging conflation is that of “intelligence” and “consciousness.” Alan Turing, in his celebrated paper that introduced the Turing Test, used the terms interchangeably—but mathematicians are notorious for playing fast and loose with their terms. Philosophers, whose trademark is the careful delineation of concepts, have always insisted on maintaining the distinction. Intelligence is the capacity to solve problems, while consciousness is the capacity for the subjective experience of qualities.

As we shall see, intelligence can be attained without consciousness.² A digital computer can be programmed to perform intelligent tasks such as playing chess and understanding language by welldefined deterministic processes, without any need to introduce enigmatic conscious experiences into the software. On the other hand, a conscious being can have subjective experiences—such as seeing the color red, or feeling anger—with needing to use intelligence to solve any problems. An android could be vastly more intelligent than any human and still lack any glimmer of interior mental life. On the other hand, a creature might be profoundly stupid and still have subjective experiences.

Agent Smith is an example of a machine that manifests humanlike behavior—which, if you witnessed such words and gestures in a human, you would immediately regard them as showing conscious emotions and volitions. Indeed, it is the immediacy of the interpretation that is deceptive. When you see someone laugh with joy, or scream in pain, you do not knowingly infer the person’s mental state from those outward signs. Rather, it is as if you see the emotions directly. Yet, we know from accomplished actors that these signs of emotions can be faked. Therefore, you are indeed making an inference, albeit an automatic one. It is a job of philosophy to scrutinize such automatic inference. When you see another human being emoting, your inference is not based wholly on what you see, but also on background information (such as whether the person is acting on the stage). More fundamentally, you are relying on the reasonable assumption that the person’s behavior arises from a biological brain just as yours does. Whenever those premises are undermined, you inevitably revise any inferences you have made from the emoting. If the emoting stops and people around you clap, you realize it was a piece of street theatre, and the person was only acting out those emotions. Or, if the person has a nasty car accident that breaks open his head, revealing electronic circuitry instead of a brain, you realize that it was only an android and you may conclude that it was only simulating emotions.

A key step in the inference is the premise that the emotion plays a role in the causal loop that produces the outward words and gestures. If, instead, we have established that the observed words and gestures are wholly explained in some other way, without involving those emotions—then the inference collapses. The exterior emoting behavior then ceases to count as evidence for an interior emotional experience. If we know that an actor’s words and gestures are scripted, then we cease to regard them as evidence for an inward mental state. Likewise, if we know that the words and gestures of an android or avatar are programmed, then they too cease to support any inference of a mental state.

In an android, or in a software simulation of a human such as an agent, words and gestures are produced by millions of lines of programmed software. The software advances from instruction to instruction in a deterministic manner. Some instructions move pieces of information around inside memory, others execute calculations, others send motor signals to actuators in the body. Each line of code references objective memory locations and ports in the physical hardware. It may do so symbolically, and it may do so via sophisticated data structures, for example, using the tag “vision-field” to reference the stabilized and edge-enhanced data from the eye cams. Nevertheless, nowhere in the software suite does the code break out of that objective environment and refer to the enigmatic contents of consciousness. Nor could the programmer ever do so, since she would need an objective, third-person pointer to the conscious experience—which, being a subjective, first-person thing, cannot be labeled with such a pointer.

Everything that the android says and does is fully accounted for by its software. There is no explanatory gap left for machine consciousness to fill. When the android says, “I see colors and feel emotions just as humans do,” we know that those words are produced by deterministic lines of software that functions perfectly well without any involvement of consciousness. It is because of this that the android’s emoting does not provide an iota of evidence for any interior mental life. All the outward signs are faked, and the programmer knows in comprehensive detail how they are faked.

This point is systematically ignored by the mathematicians and engineers who enthuse about artificial intelligence. You have to go next door, to the philosophy department, to find people who accord due importance to it. Even if, by some unknown means, the android possessed consciousness, it could never tell us about it. As we have seen, everything the android says is determined by the software. Even if, somewhere in the depths of its circuit boards, there was a ghostly glimmer of conscious awareness or volition, it could never influence what the android says and does.

Could it be that the information in the computer just is the conscious experience? This argument is popular with information engineers, as it seems to allow them to gloss over the whole mind-body problem. It is flawed because information and conscious experience have different logical structures. Namely, information exists only as an artifact of interpretation; but experience does not stand in need of interpretation in order for you to be aware of it. If I give you a disk holding numerical data (21, 250, 11, 47; 22, 250, 15, 39. etc), those numbers could mean anything. In one program, they are meteorological measurements—temperature, humidity, rainfall. In another, they are medical—pulse rate, blood pressure, body fat. The interpretation has no independent reality; the numbers have no inherent meaning by themselves. In contrast, conscious experience is fundamentally different. If you jam your thumb in a door, your sensation does not need first to be interpreted by you as pain. It immediately presents as pain. Nor can you reinterpret it as some other sensation, such as the scent of a rose. Conscious experiences have real, subjectively witnessed qualities that do not depend for their existence on being interpreted this way or that. They intrinsically involve some quality over and above mere information.

Another popular argument is to appeal to “emergence.” Higherlevel systems are said to “emerge” from lower-level systems. The simple classic example is that of thermodynamic properties, such as heat and temperature, which emerge from the statistical behavior of ensembles of molecules. Yet the concept of “temperature” just does not exist for an isolated molecule, although billions of those molecules collectively do have a temperature. In like manner, it has been suggested, consciousness emerges from the collective behavior of billions of neurons, which individually could never be conscious on their own. But emergent properties are, in fact, artifacts of how we describe the world, and have no objective existence outside of mathematical theories. An ensemble of molecules may be described in terms of either the trajectories of individual molecules or their aggregate properties, but the latter are invented by human observers for the sake of simplifications. The external reality comprises only the molecules: the statistical properties, such as average kinetic energy, exist only in the mind of the physicist. Likewise, any dynamic features of the aggregate behavior of brain cells exist only in the models of the neuroscientists. The external reality comprises only the brain cells. Yet, as you know, when you jam your humb in the door, the pain is real and present in the moment; it is not a theoretical construct of a brain scientist.

So there are good reasons for believing that machines are not conscious. But—wouldn’t these arguments apply equally to brains? Surely a brain is just a bioelectrochemical machine? It obeys deterministic programs that are encoded in the genetic and neural wiring of the brain. Yet, if our argument that machines are not conscious can also apply equally to brains, then the argument must be flawed— since we know that our own brains are indeed conscious!

The answer is that there are certain processes in brain tissue that involve nondeterministic quantum-mechanical events. And, working through the chaotic dynamics of the brain, those minute phenomena can be amplified into overt behavior. The nondeterminism opens a gateway for consciousness to take effect in the workings of the brain.

As we saw earlier, you can report only the conscious experiences that are in the causal loop that gives rise to the speech acts. If you can report that you are in pain, then the pain sensation must exert a causal influence somewhere in the chain of neural events that governs what you say and write. A step that is physically nondeterministic provides a window of opportunity for consciousness to enter into that causal chain. Since we, as humans, know that we do express our conscious perceptions, we can infer that there must be some such nondeterminism somewhere in the brain. So far, quantum-mechanical events constitute the only known candidate for this. For example, Roger Penrose and Stuart Hammeroff have formulated a detailed theory of how quantum actions in the microtubules of brain cells could play this role. The jury is still out on whether the microtubules really are the locus at which consciousness enters the chain of cause and event.

A conventional, deterministic computer has no such gateway into consciousness. So androids, and virtual avatars, that are driven by computers of that kind, cannot express conscious awareness and their behavior therefore can never be evidence for consciousness. But, if a machine were to be built that used quantum computation in the same way that the brain does, then there is no philosophical reason why that machine could not have the same gateway to consciousness that a living being does. This is not because the quantum module lets the machine carry out computations that a classical computer cannot do. Whatever the quantum computer can do, a classical one can also do, albeit more slowly. Rather, it is the specific implementation of the quantum computer that provides the bridge into conscious processes.

In The Matrix, there is no reason to think that the machines are equipped with the kind of quantum computation needed to access consciousness. Quantum computation is not mentioned in the film, and there is circumstantial evidence that the Matrix and its agents are devoid of conscious thought.

Therefore the agents—which are software modules within the Matrix—are intelligent but mindless automata. For the most part, the agents behave unimaginatively, and we might naively think that this corroborates their lack of awareness. Yet, Agent Smith exhibits initiative and seems, in his speech to Morpheus, to evince a conscious dislike of the human world. But is he genuinely conscious, or only mimicking humans? In fact, Smith gives himself away when he says about the human world, “It’s the smell, if there is such a thing . . . I can taste your stink and every time I do, I fear that I’ve somehow been infected by it.” Smith’s own logical integrity obliges him to doubt the existence of that noncomputable quality that humans talk about: the conscious experience of smell. When Smith says, “. . . the smell, if there is such a thing,” he is exhibiting the mark of the automaton. This is corroborated when he then tells Morpheus that he can “taste your stink,” revealing that Smith simply does not understand the differentiation of senses in the human mind. For a computer, data are interchangeable, but for a human, tastes, smells, colors, sounds, and feels, are irreducibly different. This fact eludes Agent Smith.

Smith is mimicking human behavior as a tactic to trick Morpheus into cooperation. As the interrogation is getting nowhere, Brown suggests, “Perhaps we are asking the wrong questions.” So Smith pretends to talk like a human, to gain Morpheus’s empathy. Needless to say, the tactic fails completely.

1. In the oldest form of Buddhism, Theravada, the two major forms of meditation are Vipassana (the Pali word for “insight”) and its complement Samatha (“tranquility”). Vipassana consists in systematically attending to the individual elements that make up the contents of consciousness. It involves persistently turning away from the ceaselessly arising tide of chatter in the mind. Over time, the chatter subsides, and preconscious activity becomes more readily observed. Laboratory data support claims that long-term practitioners acquire a conscious awareness of brain microprocesses, possibly down to the cellular level. See Shinzeng Young’s works.

2. For an alternative perspective, see Kurzweil’s essay in this volume. —Ed.

© 2003 BenBella Books. Published on KurzweilAI.net with permission.

The Matrix is set in a world one hundred years in the future, a world offering a seemingly miraculous array of technological marvels—sentient (if malevolent) programs, the ability to directly download capabilities into the human brain, and the creation of virtual realities indistinguishable from the real world. For most viewers these developments may appear to be pure science fiction:interesting to consider, but of little relevance to the world outside the movie theatre. But this view is shortsighted. In my view, these developments will become a reality within the next three to four decades.

I’ve become a student of technology trends as an outgrowth of my career as an inventor. If you work on creating technologies, you need to anticipate where technology will be at points in the future so that your project will be feasible and useful when it’s completed, not just when you started. Over the course of a few decades of anticipating technology, I’ve become a student of technology trends and have developed mathematical models of how technologies in different areas are developing.

This has given me the ability to invent things that use the materials of the future, not just limiting my ideas to the resources we have today. Alan Kay has noted, “To anticipate the future we need to invent it.” So we can invent with future capabilities if we have some idea of what they will be.

Perhaps the most important insight that I’ve gained, which people are quick to agree with but very slow to really internalize and appreciate all of its implications, is the accelerating pace of technical change itself.

One Nobel laureate recently said to me: “There’s no way we’re going to see self-replicating nanotechnological entities for at least a hundred years.” And yes, that’s actually a reasonable estimate of how much work it will take. It’ll take a hundred years of progress, at today’s rate of progress, to get self-replicating nanotechnological entities. But the rate of progress is not going to remain at today’s rate; according to my models, it’s doubling every decade.

We will make a hundred years of progress at today’s rate of progress in 25 years. The next ten years will be like twenty, and the following ten years will be like 40. The 21st century will therefore be like 20,000 years of progress—at today’s rate. The twentieth century, as revolutionary as it was, did not have a hundred years of progress at today’s rate; since we accelerated up to today’s rate, it really was about 20 years of progress. The 21st century will be about a thousand times greater, in terms of change and paradigm shift, than the 20th century.

A lot of these trends stem from thinking about the implications of Moore’s Law. Moore’s Law refers to integrated circuits and famously states that the computing power available for a given price will double every twelve to twenty-four months. Moore’s Law has become a synonym for the exponential growth of computing.

I’ve been thinking about Moore’s Law and its context for at least twenty years. What is the real nature of this exponential trend? Where does it come from? Is it an example of something deeper and more profound? As I will show, the exponential growth of computing goes substantially beyond Moore’s Law. Indeed, exponential growth goes beyond just computation, and applies to every area of information-based technology, technology that will ultimately reshape our world.

Observers have pointed out that Moore’s Law is going to come to an end. According to Intel and other industry experts, we’ll run out of space on an integrated circuit within fifteen years, because the key features will only be a few atoms in width. So will that be the end of the exponential growth of computing?

That’s a very important question as we ponder the nature of the 21st century. To address this question, I put 49 famous computers on an exponential graph. Down at the lower left hand corner is the data processing machinery that was used in the 1890 American census (calculating equipment using punch cards). In 1940, Alan Turing developed a computer based on telephone relays that cracked the German enigma code and gave Winston Churchill a transcription of nearly all the Nazi messages. Churchill needed to use these transcriptions with great discretion, because he realized that using them could tip off the Germans prematurely.

If, for example, he had warned Coventry authorities that their city was going to be bombed, the Germans would have seen the preparations and realized that their code had been cracked. However, in the Battle of Britain, the English flyers seemed to magically know where the German flyers were at all times.

In 1952, CBS used a more sophisticated computer based on vacuum tubes to predict the election of a U.S. president, President Eisenhower. In the upper right-hand corner is the computer sitting on your desk right now.

One insight we can see on this chart is that Moore’s Law was not the first but the fifth paradigm to provide exponential growth of computing power. Each vertical line represents the movement into a different paradigm: electro-mechanical, relay-based, vacuum tubes, transistors, integrated circuits. Every time a paradigm ran out of steam, another paradigm came along and picked up where that paradigm left off.

People are very quick to criticize exponential trends, saying that ultimately they’ll run out of resources, like rabbits in Australia. But every time one particular paradigm reached its limits, another, completely different method would continue the exponential growth. They were making vacuum tubes smaller and smaller but finally got to a point where they couldn’t make them any smaller and maintain the vacuum. Then transistors came along, which are not just small vacuum tubes. They’re a completely different paradigm.

Every horizontal level on this graph represents a multiplication of computing power by a factor of a hundred. A straight line in an exponential graph means exponential growth. What we see here is that the rate of exponential growth is itself growing exponentially. We doubled the computing power every three years at the beginning of the century, every two years in the middle, and we’re now doubling it every year.

It’s obvious what the sixth paradigm will be: computing in three dimensions. After all, we live in a three-dimensional world and our brain is organized in three dimensions. The brain uses a very inefficient type of circuitry. Neurons are very large “devices,” and they’re extremely slow. They use electrochemical signaling that provides only about 200 calculations per second, but the brain gets its prodigious power from parallel computing resulting from being organized in three dimensions. Three-dimensional computing technologies are beginning to emerge. There’s an experimental technology at MIT’s Media Lab that has 300 layers of circuitry. In recent years, there have been substantial strides in developing three-dimensional circuits that operate at the molecular level.

Nanotubes, which are my favorite, are hexagonal arrays of carbon atoms that can be organized to form any type of electronic circuit. You can create the equivalent of transistors and other electrical devices. They’re physically very strong, with 50 times the strength of steel. The thermal issues appear to be manageable. A one-inch cube of nanotube circuitry would be a million times more powerful than the computing capacity of the human brain.

Over the last several years, there has been a sea change in the level of confidence in building three-dimensional circuits and achieving at least the hardware capacity to emulate human intelligence. This has raised a more salient issue, namely that “Moore’s Law may be true for hardware but it’s not true for software.”

From my own four decades of experience with software development, I believe that is not the case. Software productivity is increasing very rapidly. As an example from one of my own companies, in 15 years, we went from a $5,000 speech-recognition system that recognized a thousand words poorly, without continuous speech, to a $50 product with a hundred-thousand-word vocabulary that’s far more accurate. That’s typical for software products. With all of the efforts in new software development tools, software productivity has also been growing exponentially, albeit with a smaller exponent than we see in hardware.

Many other technologies are improving exponentially. When the genome project was started about 15 years ago, skeptics pointed out that at the rate at which we can scan the genome, it will take 10,000 years to finish the project. The mainstream view was that there would be improvements, but there was no way that the project could be completed in 15 years. But the price-performance and throughput of DNA sequencing doubled every year, and the project was completed in less than 15 years. In twelve years, we went from a cost of $10 to sequence a DNA base pair to a tenth of a cent.

Even longevity has been improving exponentially. In the 18th century, every year we added a few days to human life expectancy. In the 19th century, every year, we added a few weeks. We’re now adding about 120 days every year to human life expectancy. And with the revolutions now in an early stage in genomics, therapeutic cloning, rational drug design, and the other biotechnology transformations, many observers including myself anticipate that within ten years we’ll be adding more than a year, every year. So, if you can hang in there for another ten years, we’ll get ahead of the power curve and be able to live long enough to see the remarkable century ahead.

Miniaturization is another very important exponential trend. We’re making things smaller at a rate of 5.6 per linear dimension per decade. Bill Joy, in the essay following this one, has, as one of his recommendations, to essentially forgo nanotechnology. But nanotechnology is not a single unified field, only worked on by nanotechnologists. Nanotechnology is simply the inevitable end result of the pervasive trend toward making things smaller, which we’ve been doing for many decades.

Above is a chart of computing’s exponential growth, projected into the 21st century. Right now, your typical $1000 PC is somewhere between an insect and a mouse brain. The human brain has about 100 billion neurons, with about 1,000 connections from one neuron to another. These connections operate very slowly, on the order of 200 calculations per second, but 100 billion neurons times 1,000 connections creates 100 trillion-fold parallelism. Multiplying that by 200 calculations per second yields 20 million billion calculations per second, or, in computing terminology, 20 billion MIPS. We’ll have 20 billion MIPS for $1000 by the year 2020.

Now that won’t automatically give us human levels of intelligence, because the organization, the software, the content and the embedded knowledge are equally important. Below I will address the scenario in which I envision achieving the software of human intelligence, but I believe it is clear that we will have the requisite computing power. By 2050, $1000 of computing will equal one billion human brains. That might be off by a year or two, but the 21st century won’t be wanting for computational resources.

Now let’s consider the virtual-reality framework envisioned by The Matrix—a virtual reality which is indistinguishable from true reality. This will be feasible, but I do quibble with one point. The thick cable entering Neo’s brainstem made for a powerful visual, but it’s unnecessary; all of these connections can be wireless. Let’s go out to 2029 and put together some of the trends that I’ve discussed. By that time, we’ll be able to build nanobots, microscopic-sized robots that can go inside your capillaries and travel through your brain and scan the brain from inside. We can almost build these kinds of circuits today. We can’t make them quite small enough, but we can make them fairly small.

The Department of Defense is developing tiny robotic devices called “Smart Dust.” The current generation is one millimeter—that’s too big for this scenario—but these tiny devices can be dropped from a plane, and find positions with great precision. You can have many thousands of these on a wireless local area network. They can then take visual images, communicate with each other, coordinate, send messages back, act as nearly invisible spies, and accomplish a variety of military objectives.

We are already building blood-cell-sized devices that go inside the blood stream, and there are four major conferences on the topic of “bioMEMS” (biological Micro Electronic Mechanical Systems). The nanobots I am envisioning for 2029 will not necessarily require their own navigation. They could move involuntarily through the bloodstream and, as they travel by different neural features, communicate with them the same way that we now communicate with different cells within a cell phone system.

Brain-scanning resolution, speeds, and costs are all exploding exponentially. With every new generation of brain scanning we can see with finer and finer resolution. There’s a technology today that allows us to view many of the salient details of the human brain. Of course, there’s still no full agreement on what those details are, but we can see brain features with very high resolution, provided the scanning tip is right next to the features. We can scan a brain today and see the brain’s activity with very fine detail; you just have to move the scanning tip all throughout the brain so that it’s in close proximity to every neural feature.

Now how are we going to do that without making a mess of things? The answer is to send the scanners inside the brain. By design, our capillaries travel by every interneuronal connection, every neuron and every neural feature. We can send billions of these scanning robots, all on a wireless local area network, and they would all scan the brain from inside and create a very high-resolution map of everything that’s going on.

What are we going to do with the massive database of neural information that develops? One thing we will do is reverse-engineer the brain, that is, understand the basic principles of how it works. This is an endeavor we have already started. We already have high resolution scans of certain areas of the brain. The brain is not one organ; it’s comprised of several hundred specialized regions, each organized differently. We have scanned certain areas of the auditory and visual cortex, and have used this information to design more intelligent software. Carver Mead at Caltech, for example, has developed powerful, digitally controlled analog chips that are based on these biologically inspired models from the reverse engineering of portions of the visual and auditory systems. His visual sensing chips are used in high-end digital cameras.

We have demonstrated that we are able to understand these algorithms, but they’re different from the algorithms that we typically run on our computers. They’re not sequential and they’re not logical; they’re chaotic, highly parallel, and self-organizing. They have a holographic nature in that there’s no chief-executive-officer neuron. You can eliminate any of the neurons, cut any of the wires, and it makes little difference—the information and the processes are distributed throughout a complex region.

Based on these insights, we have developed a number of biologically inspired models today. This is the field I work in, using techniques such as evolutionary “genetic algorithms” and “neural nets,” which use biologically inspired models. Today’s neural nets are mathematically simplified, but as we get a more powerful understanding of the principles of operation of different brain regions, we will be in a position to develop much more powerful, biologically inspired models. Ultimately we can create and recreate these processes, retaining their inherently massively parallel, digitally controlled analog, chaotic, and self-organizing properties. We will be able to recreate the types of processes that occur in the hundreds of different brain regions, and create entities—they actually won’t be in silicon, they’ll probably be using something like nanotubes—that have the complexity, richness, and depth of human intelligence.

Our machines today are still a million times simpler than the human brain, which is one key reason that they still don’t have the endearing qualities of people. They don’t yet have our ability to get the joke, to be funny, to understand people, to respond appropriately to emotion, or to have spiritual experiences. These are not side effects of human intelligence, or distractions; they are the cutting edge of human intelligence. It will require a technology of the complexity of the human brain to create entities that have those kinds of attractive and convincing features.

Getting back to virtual reality, let’s consider a scenario involving a direct connection between the human brain and these nanobot-based implants. There are a number of different technologies that have already been demonstrated for communicating in both directions between the wet, analog world of neurons and the digital world of electronics. One such technology, called a neuron transistor, provides this two-way communication. If a neuron fires, this neuron transistor detects that electromagnetic pulse, so that’s communication from the neuron to the electronics. It can also cause the neuron to fire or prevent it from firing.

For full-immersion virtual reality, we will send billions of these nanobots to take up positions by every nerve fiber coming from all of our senses. If you want to be in real reality, they sit there and do nothing. If you want to be in virtual reality, they suppress the signals coming from our real senses and replace them with the signals that you would have been receiving if you were in the virtual environment.

In this scenario, we will have virtual reality from within and it will be able to recreate all of our senses. These will be shared environments, so you can go there with one person or many people. Going to a Web site will mean entering a virtual-reality environment encompassing all of our senses, and not just the five senses, but also emotions, sexual pleasure, humor. There are actually neurological correlates of all of these sensations and emotions, which I discuss in my book The Age of the Spiritual Machines.

For example, surgeons conducting open-brain surgery on a young woman (while awake) found that stimulating a particular spot in the girl’s brain would cause her to laugh. The surgeons thought that they were just stimulating an involuntary laugh reflex. But they discovered that they were stimulating the perception of humor: whenever they stimulated this spot, she found everything hilarious. “You guys are just so funny standing there” was a typical remark.

Using these nanobot-based implants, you will be able to enhance or modify your emotional responses to different experiences. That can be part of the overlay of these virtual-reality environments. You will also be able to have different bodies for different experiences. Just as people today project their images from Web cams in their apartment, people will beam their whole flow of sensory and even emotional experiences out on the Web, so you can, à la the plot concept of the movie Being John Malkovich, experience the lives of other people.

Ultimately, these nanobots will expand human intelligence and our abilities and facilities in many different ways. Because they’re communicating with each other wirelessly, they can create new neural connections. These can expand our memory, cognitive faculties, and pattern-recognition abilities. We will expand human intelligence by expanding its current paradigm of massive interneuronal connections as well as through intimate connection to non-biological forms of intelligence.

We will also be able to download knowledge, something that machines can do today that we are unable to do. For example, we spent several years training one research computer to understand human speech using the biologically inspired models—neural nets, Markov models, genetic algorithms, self-organizing patterns—that are based on our crude current understanding of self-organizing systems in the biological world. A major part of the engineering project was collecting thousands of hours of speech from different speakers in different dialects and then exposing this to the system and having it try to recognize the speech. It made mistakes, and then we had it adjust automatically, and self-organize to better reflect what it had learned.

Over many months of this kind of training, it made substantial improvements in its ability to recognize speech. Today, if you want your personal computer to recognize human speech, you don’t have to spend years training it the same painstaking way, as we need to do with every human child. You can just load the evolved models, it’s called “loading the software.” So machines can share their knowledge. We don’t have quick downloading ports on our brains. But as we build nonbiological analogs of our neurons, interconnections, and neurotransmitter levels where our skills and memories are stored, we won’t leave out the equivalent of downloading ports. We’ll be able to download capabilities as easily as Trinity downloads the program that allows her to fly the B-212 helicopter.

When you talk to somebody in the year 2040, you will be talking to someone who may happen to be of biological origin but whose mental processes are a hybrid of biological and electronic thinking processes, working intimately together. Instead of being restricted, as we are today, to a mere hundred trillion connections in our brain, we’ll be able to expand substantially beyond this level. Our biological thinking is flat; the human race has an estimated 10²⁶ calculations per second, and that biologically determined figure is not going to grow. But nonbiological intelligence is growing exponentially. The crossover point, according to my calculations, is in the 2030s; some people call this the Singularity.

As we get to 2050, the bulk of our thinking—which in my opinion is still an expression of human civilization—will be nonbiological. I don’t believe that the Matrix scenario of malevolent artificial intelligences in mortal conflict with humans is inevitable. At that point, the nonbiological portion of our thinking will still be human thinking, because it’s going to be derived from human thinking. Its programming will be created by humans, or created by machines that are created by humans, or created by machines that are based on reverse-engineering of the human brain or downloads of human thinking, or one of many other intimate connections between human and machine thinking that we can’t even contemplate today.

A common reaction to this is that this is a dystopian vision, because I am “placing humanity with the machines.” But that’s because most people have a prejudice against machines. Most observers don’t truly understand what machines are ultimately capable of, because all the machines that they’ve ever “met” are very limited, compared to people. But that won’t be true of machines circa 2030 and 2040. When machines are derived from human intelligence and are a million times more capable, we’ll have a different respect for machines, and there won’t be a clear distinction between human and machine intelligence. We will effectively merge with our technology.

We are already well down this road. If all the machines in the world stopped today, our civilization would grind to a halt. That wasn’t true as recently as thirty years ago. In 2040, human and machine intelligence will be deeply and intimately melded. We will become capable of far more profound experiences of many diverse kinds. We’ll be able to “recreate the world” according to our imaginations and enter environments as amazing as that of The Matrix, but, hopefully, a world more open to creative human expression and experience.

SOURCES

BOOKS

Kurzweil, Ray, The Age of Spiritual Machines: When Computers Exceed Intelligence (Penguin USA, 2000).

© 2003 BenBella Books. Published on KurzweilAI.net with permission.

As the sophistication of graphics software marched forward andthe cost of computing slid downward, the annual SIGGRAPH film andvideo show became the crucible in which technologists, filmmakersand artists were introduced to one another: the baton of computergraphics design was passed from those who wrote the programs requiredto create imagery to those who perceived and exploited the incrediblecommunicative potential of this fledgling medium.

Along the way, the natural predilection to create computer graphicsin the image of ourselves has led to a striking body of creativeendeavor, ever growing in realism and resolution, which now knocksat the door of imperceptibility; that is, the rendering of humanperformances indistinguishable from flesh and blood performances.Some of the steps in that evolution are traced here from a personalperspective, and some speculations on future developments are presented.

Initial attempts at simulating human body motion took several formsat Digital Effects, a company I co-founded in 1978 along with collegeassociates. Hierarchical skeletons were created and keyframe-animatedto move in a bipedal fashion, but without any IK (inverse kinematics)solutions available, the results were stilted and difficult to edit.Rotoscoping live-action footage of a subject festooned with witnesspoints at the joints and digitizing the positing of the points onfilm allowed us to imbue our characters with more lifelike motion,but the process yielded primarily 2-D information that could notbe used from all angles.

At that time, 3-D modeling software had been designed for architecturalapplications and was not capable of modeling the human form in asatisfactory manner. Early attempts at creating and linking facialexpressions using software solutions in computer animation werevery disappointing and unconvincing at even the most basic levelsof lip synchronization.

3-D rotoscoping at Robert Abel and Associates yielded the firstmotion-captured animation in the commercial project, Sexy Robot,under the technical direction of Frank Vitz, who used two 35mm camerasto triangulate the 3-D positions of witness points on a live subject.

During the period of 1985-1986, a good deal of seminal researchand development in character animation was conducted at Abel aswell as Digital Productions and Omnibus Computer Graphics, all threeof which joined together and imploded under the weight of theirlargesse.

First Synthespian

While at Digital-Omnibus-Abel (to become known as DOA) I met DianaWalczak, a recent college graduate who was searching for ways tocombine science and art. We formed a partnership based on our mutualinterest in developing computer-generated characters and came upwith sculpture-based solutions to the problem of modeling the humanform and creating facial animation.

Diana sculpted a human form in clay and metal armature that wascast in hydrocal, from which individual body parts could be createdand digitized using a magnetic digitizing device called the 3SpaceDigitizer by Polhemus. The body parts were lined with thin tapeto define the optimum topology in polygons and digitized by handusing a magnetic sensor.

Thesebody parts were then assembled digitally into a skeletal hierarchyto form Nestor Sextone, our first Synthespian. Nestor’s joints wereformed by interpenetrating solids-due to the fact that softwaredid not yet exist that would allow for flexors at the joints-whichgave us seams similar to that of a plastic action figure. For facialanimation, a neutral face was cast in hydrocal, which allowed Dianato make multiple clay copies that could be sculpted into variousphonemes of speech and facial expressions.

Larry Weinberg, a programmer from Digital Effects and Omnibus wholater would write Poser, contributed software that would allow usto link the various digitized facial expressions together by re-orderingthe polygons. With multiple faces re-ordered into exactly the samepolygonal topology, we could interpolate from one to another, enablingus to create scripts that could simulate lip synchronization withour soundtracks. Using keyframe animation and Larry’s facial animationsoftware, Sextone made his screen debut for SIGGRAPH 1988 in a filmof 30 seconds duration in which he campaigns for the presidencyof the Synthetic Actors Guild.

Intriguedby the potential of motion capture to link natural human motionto our synthetic characters, we created Don’t Touch Me, amusic video piece that premiered at SIGGRAPH 1990 in which singer/songwriterPerla Batalla was optically motion captured (by Motion Analysisin Santa Rosa, CA) to drive a singing synthespian called Dozo. Bythis time, we had suitable flexing software for simple joints likeelbows and knees, but the multi-axis requirements for the shoulderjoint meant a solution was still several months of development away.

Facial animation was again created by linking digitized sculpturesof various facial expressions and this technique yielded superiorresults to any of the software solutions of the time that soughtto model the musculature of the face. Software solutions would requiremany years of development before they would overcome the qualityand believability of the sculpture-based technique, which allowedfor the preservation of facial volume and the illusion of the preservationof facial muscle integrity during motion.

This is the result of the fact that Diana’s keyframe sculpturesall had appropriate muscle definition and maintained that definitionwhile interpolating from one to the next.

Ourfirst stereoscopic synthespians were created for In Search ofthe Obelisk, a theme park trilogy for the Luxor Hotel in LasVegas, designed by Doug Trumbull. Using optical motion capture oflive dancers, we created the illusion of glass synthespians dancingon a hovering beach that floated over the audience.

Since we used ray tracing to refract the background through thebodies of the dancers, the stereoscopic image perceived by the audiencewas accurately rendered with slightly different refractions fromthe point of view of the left eye as compared to the right, yieldinga very realistic illusion reminiscent of the optical propertiesof a glass object when viewed stereoscopically.

Forthe feature film Judge Dredd, digital stunt doubles werecreated to solve a technical problem: many of the shots in the climacticchase sequence required Sylvester Stallone and Rob Schneider toappear to ride on a flying motorcycle that weaves around other flyingvehicles and skyscrapers. The close-ups were shot on a green screenstage with a gimbaled prop of the motorcycle and composited intomocon (motion control) footage of the huge model of the city. Othershots required the motorcycle to fly toward camera from a long distanceand maneuver in a complex flight path as it whizzed past camera.

These shots were not able to be photographed due to limitationsin the length of the green-screen camera rig as well as the reluctanceof the producers to allow a large, heavy, motion-controlled camerarig to careen within a few feet of their lead actors. We used magneticmotion capture (Ascension Technologies’ Flock of Birds) to obtainthe body dynamics of the motorcycle riders during the various changesin attitude of the motorcycle.

Playing back the previsualization on video, the subject (in thiscase Diana Walczak) was affixed with magnetic trackers and was wobbledaround on a gimbaled motorcycle mockup in sync with the previz playback.The way her body moved in response to the motion of the bike was captured and applied to photoreal synthetic versions of Stalloneand Schneider. For the faces, we used CyberScans for the first timeand the results were satisfactory since the camera never lingeredon the faces at close range.

Organic shape-shifting

A later project for the feature film X-Men involved the characterMystique, played by Rebecca Romijn-Stamos. Mystique is a shape-shifterwho transforms from her scaly blue form into other characters andback again using a combination of live action photography and CG animation.Director Bryan Singer was looking for a transformation that wouldstand apart from the typical morphing that had risen like a plaguethrough the visual arts, becoming a constant technique used in advertisingand in films to change one object into another using simultaneous2D shape transformation with dissolving texture vertices.

We designed a technique that would allow for a dimensional transformationthat would begin at various locations and spread across and aroundthe limbs in an organic, infectious fashion accented by 3-D scalesbursting through the surface and settling down like a shaking dog’scoat to form the scales on her body. In most cases we used CyberScandata of the outgoing actor matched to the 3-D position of the actorin the shot as a matting element to transform into an all-CG Mystique.

This technique required eighteen stages of production to createthe multilayered complex transformation and very careful matchingof CG skin, clothing and hair to the live action footage. Althoughthe mandate was to make the CG Mystique appear photoreal, her blue,scaly body was very different from that of a normal person, yieldingconsiderable visual leeway

For the Revolution Films production of the Jet Li film, TheOne, Jet Li battles his identical doppelganger from anotherdimension. For many shots, a simple split screen or a Patty Duke-styleover- the-shoulder shot would suffice, but for high-speed kung fubattle sequences in which punches and kicks had to land and be feltby the audience, digital face replacement was the technique of choice.

The separation of a facial performance from a physical performancehad been accomplished before in Jurassic Park in the shotwhere the velociraptor leaps up from below to attack a child character.

The adult stunt double’s face was replaced with that of the childactor, but that was simply a composite of photographic elements.In The One, the complex high-speed motion of the subjectsduring the fight sequences-coupled with the requirement that thetwo subjects sometimes appear to move at different camera framerates-required us to develop a fully CG face-replacement solution.

The stunt double, a kung fu expert with a very similar body typeto Jet Li, was outfitted with a plastic mask that was milled froma CyberScan of Jet Li’s face. The mask was equipped with retro-reflectivewitness points and the camera was outfitted with a fluorescent,circular light around the lens to ensure that the markers wouldshow up on film.

Jet Li plays a police officer pursued by his evil alterego from a parallel universe who seeks to kill him and becomeThe One. Advanced face replacement techniques allow Lito fight his twin. Both faces are visible and fully expressivein close-ups.

The fight sequences were choreographed so that the force of theimpacts would impart proper reaction in the two participants. Usingthe known positions of the facemask markers, we could determinethe precise orientation of the stunt double’s face on each frame,allowing us to track a CG face over top of his mask.

Using CyberScan data of Jet’s face, along with high-resolutionphotographs, we created and rigged a detailed 3-D Jet Li face withblendshapes that would allow us to simulate different facial expressionsduring the fight. The CG face was then animated to give us the appropriateexpression for each sequence, matted into the shot covering up themask and blended into the stunt double’s natural color around theface. Because it was not possible to photograph Li’s face in theproper dynamic orientation with the proper expression for a givenmoment of a fight, a CG face was the only solution.

The resulting technology, which allows us to separate the physicalperformance from the facial performance, has far-reaching implicationsfor the future of filmmaking. First of all, stunt sequences thatnormally would be staged in such a way that the face of the stuntdouble is never facing camera can now be staged according to theneeds of the director, and the actor’s face can be inserted accuratelyand believably. More broadly, the facial performance of an actorwho is incapable of the physical aspects of a performance can becomposited into the footage of a stunt double to multiply the rangeof an actor’s possible roles. Recent projects making use of ourtechnology include inserting an actor’s face onto stunt doubleswho are surfing and riding motorcycles.

Animation trumps mo-cap

More interesting from our standpoint is the creation of whollyCG characters and their application to entertainment projects. UniversalStudios came to us with the mandate to create the best theme parkattraction in the world based on the Spider-Man characters, andwe spent three years in production on The Amazing Adventuresof Spider-Man, a multimedia, stereoscopic, moving motion-simulatorattraction that was to become the flagship of their new theme park,Islands of Adventure in Orlando, Florida.

The Amazing Adventures of Spider-Man was createdfor Universal's billion-dollar Islands of Adventure themepark in Orlando. It's the first ride in history to combine stereoscopic3D film projected onto giant screens with the latest in motion-basedvehicle technology. This virtual-reality adventure immersesriders in a comic-book battle between Spider-Man and membersof the sinister syndicate as riders move through a 1.5-acreset environment.

Working with our head software designer, Frank Vitz, we developedsoftware that would compensate for the viewing position of the movingaudience, who would sit in six degrees-of-freedom motion simulatorstraveling on a track past 13 large reflective screens. The imagerywas projected in stereoscopic eight-perf 70mm film.

A great deal of attention was paid to matching the physical setsin the ride to the imagery projected onto the screens so that thelines were blurred between the real world and the virtual, projectedworld. In fact, many of the sets adjacent to the screens were dressedwith CG textures that originated from our virtual sets and werescanned onto eight-foot-wide canvas murals so that imagery and sightlines would match up and blend the two worlds into one.

From a design standpoint, our goal was to take the audience intoa comic book world that combines the hard key-lighting and saturated-colorstyle of comic art with enough textural detail to feel like a realplace. It was a balancing act between stylization and realism thatresulted in a unique and exciting environment in which to stagethe epic struggle between Spider-Man and a gaggle of super-villainsled by Dr. Octopus, one that swirls around the audience whom Spider-Manmust protect.

We tested and abandoned motion capture for the project based onthe fact that the superhuman performances of the Marvel characterscould be better realized by talented animators using keyframe techniquesrather than by animators trying to extend the physical range ofmotion-captured athletes.

Our first totally original synthespian project was made possibleby Busch Entertainment, who gave us virtual “carte blanche”to design a ride from the ground up for a new area at Busch Gardensin Williamsburg, VA. With only one word of direction, “Ireland,”from the client, we wrote a story called Corkscrew Hill thatwould exploit the physical parameters available: two 60-person Reflectonemotion bases in two identical warehouse spaces.

The Corkscrew Hill computer-animated stereoscopicepic ride experience takes audiences on an adventure to OldIreland, populated with humans and mythical creatures. In thepre-show, the audience shrinks to fit in a magic box. Then theyenter a motion base and are strapped into their seats for themain show: one continuous-point-of-view shot from the box ascharacters carry it on a wild adventure on Corkscrew Hill.SensAble's FreeForm System was used to sculpt characterheads. Pieces of character models were joined with Paraformsoftware. Maya was used for modeling, animation, and rendering.Large-format digital projection was engineered by Electrosonic.

We specified very large reflective screens and anopen cockpit design for the attraction and—working with theaudio-visual engineers at Electrosonic in Burbank, CA—we cameup with a digital projection system that would give us film resolutionon a large screen despite the fact that digital projectors werecurrently not up to the task.

Byrotating four Barco DLP projectors 90 degrees and edge-blendingdown the middle (using two projectors for each image), we couldget stereoscopic image pairs onto the 30 x 40 foot screens at 2048horizontal by 1280 vertical resolution. Since the brain fuses theleft and right images into a single mental image, any image artifactsfrom the projection were lost in the mental blending process, resultingin excellent stereoscopic imagery. We choreographed a camera movethat takes us on an adventure through ancient Ireland, encounteringIrish townspeople, a magic flying horse, banshees, a troll, a witchand a griffin.

Thiseight-minute attraction allowed us to create a completely syntheticworld and populate it with mythical creatures and characters witha visual style akin to that of a storybook. Again, we opted forkeyframed character animation instead of motion capture, which oftenseems pedestrian when applied to CG characters. When keyframing,an animator enters into and becomes the character, breathing originallife into it that cannot be obtained through motion capture, whichis in effect the three-dimensional “xeroxing” of a physicalperformance.

In the same way that a caricature of a person looks more like thesubject than would a tracing off a photograph, or a good sculptureof a person looks more like them than a life cast, a stylized CGcharacter created by a talented keyframe animator looks more believableand lifelike than one created with motion capture and CyberScans.

The limits of photorealism

Looking to the future, one must examine one’s goals in creatingCG life forms. There are those who hold up photorealism as the ultimategoal: to create a synthespian indistinguishable from a live actor.This idea has intrigued, taunted and tormented programmers for 30years, going back to the films Westworld in 1973 and Lookerin 1981. The broad base of development required to accomplish thisfeat has been gaining momentum at an exponential rate as more applications,competition and funding enter the arena.

There exists a trade-off between what level of realism is possibleversus how much computing time can be spent on each frame. We suppliedvery efficient body databases to Ray Kurzweil’s Ramonaproject, which was presented at the Technology Entertainment Design(TED) conference in February 2001. This real-time performance tookadvantage of recent developments in hardware rendering that alloweda fairly sophisticated human figure to be rendered and displayedat 30 frames per second. Through the use of real-time motion captureand voice synthesis, Ray was able to inhabit his female alter ego,Ramona.

The performance was designed within the limitations of the technology,in that the “camera” did not venture too close to Ramona’sface, where the “efficiency” of our data would becomea liability in terms of image quality. As the camera approachesthe subject, the resolution requirements skyrocket, and to rendera photorealistic close-up on film requires orders of magnitude morecalculation than can be supported by real-time rendering engines.

As Ray points out, computing speeds are increasing at exponentialrates, but current technology still gets slammed to the mat whenit is applied to creating a synthespian who appears real in everydetail. The problem is that we spend so much of our time studyingthe nuances of facial expression in our colleagues, friends andfamily, so we have become quite expert at spotting flaws. Thereare many subtle details in a real face, including how the complexmuscle system perturbs the skin surface, how light scatters insidethe skin, and how surface pores, blemishes and other minute detailslook and react to light.

A spectacular amount of money was poured into solving these problemsin the all-CG feature film Final Fantasy: the Spirits Withinand the results did not pay off at the box office. Many projectshave been proposed that would use CG characters to bring deceasedactors to the screen for a posthumous encore, but the technologyis not yet ready for this task, and many of us cringe at the prospectof this sort of application. The recent release of S1m0nereiterates the basic problem these sorts of projects face: we canget about 90 percent of the way to photorealism in CG actors, butthe last ten percent is extremely expensive and time-consuming incomparison to photographing real actors.

In Final Fantasy, the hair, skin, cloth dynamics and lightingare all in that 95% range that just doesn’t make it to photoreality,except in stills. In motion, the illusion lacks the subtlety ofmicro-motion and micro-detail of live action photography, and theresults are unsettling and distracting from the storyline.

In Simone, the producers opted to use a live actor who wasdigitally altered to be just slightly idealized through image processing.Coupled with a few shots of a CyberScanned 3-D model being revealedlike an orange peel wipe-on, the processed footage told the storyadequately and carried the story point of a believable CG humancost-effectively.

Tohave used a CG character throughout would have been many times morecostly and it is unlikely that the audience would believe that theCG character could be mistaken for a real actor. Albeit a valiantattempt, the film presumes a mythical world where Hollywood producersand the general public have no knowledge of the history and progressof visual effects, computer animation and digital compositing.

Animator as actor

The exciting areas to be explored are those where the animator becomesanalogous to the actor. By animators using a robust set of tools andtechniques, performances of the quality and richness currently createdby the finest actors will be made possible in a medium free of theconstraints of live-action photography. These will be characters,roles, and plots that exploit our intimate familiarity with the humanform and its subtleties, but don’t attempt to recreate photorealisticrenderings of it.

When painters developed the skills to recreate realistic images,a golden era of realism followed. But when photography came alongand replaced the role of the painter as visual documentarian, paintersresponded with expressionism and abstraction, modes of image-makingonly possible in the era of post-realism.

In the same way, after the CG industry is able to reproduce realityin its most intricate detail, the next step will be to build uponthat foundation a new and exciting future of non-realistic style.But rather than being limited to the confines of a painted canvasor a physical sculpture, the realm of imagination becomes the onlyouter limit.

Beyond the capability to achieve photorealism, there is a muchmore compelling goal of creating entertainment that takes placebeyond what and where we can photograph. The writer and directorare now the creative overlords, equipped with unlimited theatricalpossibilities in terms of locations, characters, storylines andvisual style. The entire world of science fiction and fantasy-basedliterature can be shot “on location” without limitation.New stories heretofore inconceivable will be created and broughtto the world of the visual arts and entertainment.

In this work, the emphasis will be on the concept. The writer anddirector stand at the door of a new space that has thus far beenexplored by precious few—and marvel at the possibilities.

Sextone for President Written and Directed byJeff Kleiser and Diana Walczak © 1988 Kleiser-Walczak
Don’t Touch Me Directed by Diana Walczak andJeff Kleiser © 1989 Kleiser-Walczak

In Search of the Obelisk Computer Animation byKleiser-Walczak. Produced by The Trumbull Company for Circus CircusEnterprises, Inc.

X-Men © 2000 Twentieth Century Fox. Allrights reserved. Image courtesy Kleiser-Walczak

The One © 2001 Revolution Studios DistributionCompany, LLC. Property of Sony Pictures Entertainment, Inc. ©2001 Columbia Pictures Industries, Inc. All rights reserved. Imagecourtesy Kleiser-Walczak.

The Amazing Adventures of Spider-Man © 1999Universal Studios Escape. A Universal Studios/Rank Group Joint Venture.All rights reserved. Image courtesy Kleiser-Walczak.

Corkscrew Hill Original ride film Written andDirected by Jeff Kleiser and Diana Walczak © 2001 Busch EntertainmentCorporation. All rights reserved. Image courtesy Kleiser-Walczak.

Final Fantasy Property of Sony Pictures Entertainment,Inc. © 2001 Columbia Pictures Industries, Inc. All rights reserved.

S1M0NE © 2002 Darren Michaels/New Line Productions

I met my first computer in 1968, when I came from London to take up a one-year visiting professorship at UC San Diego. Four years later I spent a couple of years at the AI Lab at Stanford, and while I was there I started work on a program designed to generate original art. I called it AARON because I assumed I’d be writing a series of programs and it seemed like a good idea to start with a name from the beginning of the alphabet.

There never was a series of programs. I’m still in San Diego and I’m still working on AARON today.

Here’s another way to begin the story.

I started in on a painting career in 1952 and in the succeeding years I showed my work in one of London’s leading galleries, where a six-week exhibition would be seen by two or three hundred people, and in various museums and international shows, where the audiences may have numbered a couple of thousand. That was still the case when I met my first computer in 1968. But it had changed by1985, when I represented the US in the World Fair in Tsukuba (fig. 1). A million people went through the US Pavilion in six months and saw AARON making drawings on one of my drawing machines.

In 1995 I did an exhibition at the Computer Museum in Boston, this time using a painting machine (fig. 2). The size of the walk-in audience was conventionally modest, but the PR people reported that through television programs, syndicated newspaper articles, magazine articles–the entire gamut of media coverage–my work had reached about thirteen million people.

Figure 1. World Fair, 1985.

Figure 2. Computer Museum, 1995.

AARON entered a new public phase early in 2001. In the preceding months, Ray Kurzweil’s people had done an excellent job of preparing AARON for a new life on the web and AARON–the program itself, not its output–became available as shareware (fig 3). Within a few weeks there were tens of thousands of downloads and the number was increasing by two or three hundred a day. That’s the same number that I said would have seen a conventional gallery exhibition thirty years ago.

Figure 3. Screen image, 2002.

These numbers aren’t going to surprise anyone in an industry where exponential growth has been the norm for the past decade. Yet I’m only talking here about the growth in the size of a potential audience, not what it takes to reach that audience. The difference is important: art has a cultural role that can’t be served unless it reaches its audience. For that reason, issues of dissemination have never been far behind issues of production for the artist; the two are not as independent of each other as one might imagine.

For artists today using conventional means of production–painting, sculpting–the current technology-driven changes in the available means of dissemination and the resultant growth of potential audience means very little. There is less chance of someone stumbling across one of fifty million web-sites than there is of someone walking into a commercial art gallery by mistake. But artists using the web as a medium rather than a new mode of dissemination may face a similar predicament. What does it mean to use a public medium without a public?

I made a decision at the outset affecting both production and dissemination: on the one hand to produce art by writing a program that produced art and, on the other hand, to exhibit the program producing art rather than exhibiting only the things it produced. Had I not made that decision then in all likelihood I’d still be disseminating my work to two or three hundred people at a time in conventional commercial galleries.

For reasons that I hope to make clear, I’m going to be talking as much about the reasons we do things as about the things we do, so I have to ask why I made that decision. Not because I knew what would happen thirty years later, obviously. It was simply that, in a world where few people knew anything about computers and most of them had it wrong, I didn’t want to present myself as a magician with a magic box. I wanted to clarify, not to mystify. I was acting out of a persistent belief that the health of art is strongly determined by its relationship to the culture it serves and that supports it and I was groping for an appropriate form for that relationship.

But notice that nothing in that decision had any direct bearing on what I wanted the program to do. We’re a long way here from aesthetic considerations.

That decision, made thirty years ago, has left me with a number of unanswered questions, one of which I’m going to try to answer today. As you will see, both the nature of art and the nature of the potential audience figure in my attempted answer.

To introduce the question: I had an email correspondence a while back with someone who said he thought I should make AARON’s source code available on the web. I responded that I’d never hesitated to tell people anything they wanted to know about how the program works so that they could write their own programs, but that AARON itself was my intellectual property and I couldn’t see any reason to give it away. He responded–echoes of linux–that if AARON could be picked up by programmers all over the world we might have generations of AARONs producing original art into the distant future. If I kept it to myself, he said, AARON would come to an end when I stopped working on it.

I replied that he had a point–which he certainly did!–but that I didn’t care for his conclusion. My preference would be to have AAARON modify its own performance over time the way a human artist modifies his own performance over time so that AARON itself could be generating original art into the distant future.

Where’s the problem? In its current state AARON could go on generating original images forever. Yet both the question and my answer presuppose not just the possibility, but the desirability of a different state; one that could be achieved, as it is by human artists, only through continuous modification.

Well, how about re-writing the program so that it randomly changes the values assigned to some of its variables every so often? With hundreds of variables to change, that should cover the search space pretty thoroughly and come up with some interesting possibilities.

That is certainly not what I mean by modifying its own performance over time the way a human artist modifies his own performance over time. The human artist certainly does not wander blindly and randomly through a search space of possibilities, stubbing his toes from time to time on interesting things to do. If you think of “interestingness” as a context-free property–which I do not–then there are always many “interesting” things to do, almost all of which are ignored by the artist just as almost all of the legal moves in a chess game are ignored by the chess master. They fall outside the steadily narrowing window through which the artist surveys the increasingly constrained game to be played out in the future.

The work of any major artist will give you the feeling that he always knew exactly where he was going, but then, any path will look like a path once it has been trodden. Try predicting today what he will do tomorrow and you’ll get a very different understanding of what “moving forward” means. The artist has a notion of direction, not of destination, and he himself can never predict, other than what he certainly will not do, what tomorrow will bring.

As to whether what he does tomorrow will have the cultural role he wants for it; that must depend to a large extent upon the enormously varied demands and the expectations of the culture. The only thing I know about that for sure, after fifty years in the field, is that the aesthetic value of a work constitutes only a part of what the culture wants from the artist, and the size of that part appears to be inversely proportional to what it costs.

There’s an old art world story about a man who has a Picasso original hanging in his living room until one day a friend tells him that it isn’t an original, it’s a print; and he takes it out of the living room and hangs it in the toilet. Less apocryphally and at the other extreme of exclusivity, a Warhol print–a print, not a painting–sold at Sotheby’s last year for four and a half million dollars, bringing to the buyer the same aesthetic value it had when it was first sold–when the artist was still alive and able to make more–probably for a couple of thousand. What was the buyer buying for the other four million plus?

Exclusivity, fashion, whatever… mostly, what the buyers of art are buying is the artist’s reputation, his status. And while status may be awarded temporarily on the basis of a single work or a single exhibition–as Andy Warhol famously said, everyone can be famous for fifteen minutes–it is withdrawn quite soon if the artist is not seen to be moving forward, not developing; if, like AARON in its current state, he merely produced an endless succession of original images without ever changing the terms of reference.

So: not simply change over time; purposeful, self-directed change over time.

Conventionally, we would probably say that purposeful change must necessarily require a program’s assessment of whether or not its output had satisfied some criteria. My own contention, at which I have already hinted, is that the aesthetic criteria that relate to the appearance of individual works are only the final link in a chain of criteria leading far back into the reasons the individual does what he does and believes what he believes. The central question for me, then, is not whether a program can self-modify in order to satisfy internal criteria; it is whether enough of that chain of criteria can ever be internal to a program for it to manifest the self-directed development we expect of human artists.

I don’t think I have an answer to that question. What I do have, however, is the thirty-year history of a continuously developing program and some insight into what was involved in directing it. By way of illustration, I’d like to review a few key points in the program’s history where I can locate particularly important shifts in direction.

Most of this will concern recent developments, but I need to begin by looking at the basis of AARON’s drawing strategy, which was established very early on, because there’s a lot about recent developments that won’t make much sense unless I do.

Briefly, then; the form AARON took initially followed from what seemed rather obvious; that making images for human consumption meant addressing the cognitive processes that enable human beings to attach meaning to marks on flat surfaces. So I concentrated initially (fig. 4) on giving the program the ability to differentiate between closed forms and open forms, figure and ground and so on; the standard things I could get from books. After a few years I sensed that the books weren’t telling me everything I needed to know and I started to examine what actually happens in young children in the first year or two of drawing. They start with a simple motor activity–scribbling; backward and forward, round and round–until one day one of the round and round scribbles migrates outwards to become an enclosing form (fig. 5). Some time later, the scribble disappears, leaving the closed form (fig. 6) as an empty but infinitely malleable container for whatever the individual wants to represent.

Figure 4. Computer generated drawing, ink on paper, 20 x 30, 1975.

Figure 5. Child’s drawing.

Figure 6. Child’s drawing.

I never succeeded in simulating the child’s performance very well, but simply in terms of the diversity of forms I could see being generated by this two-step strategy–build a core figure, then draw a line around it–the results were astonishing (fig. 7). Compared to any other way I could think to generate forms of that complexity, it was practically a computational free lunch and, even had there not been other reasons for adopting it, I suspect that it would have become AARON’s dominant mode.

Figure 7. Tapestry, 40 x 60, 1983.

I think it’s important to point out that the strategy has nothing in common with current computer graphics methods, which, like the photography they simulate, deal with the light reflected off the surfaces of things. AARON knows nothing of surfaces. The method for generating its enclosing forms is general enough that it can handle any core figure the program can construct from its data and it has undergone only a single modification in twenty years. That means that as the program moved on to figurative imagery, most attention has been required by the design of the core figures–again, nothing to do with surfaces–that replaced the scribbles in the earlier work.

By the early ‘nineties (fig. 8) I had given AARON some rudimentary knowledge of a few things in the world–figures and plants and very little else–and it was making recognizable images; initially from a sort of two-and-a-half dimensional mode in which it could make plausible drawings of the body from a single viewing position but couldn’t do genuinely 3-dimensional things like drawing an arm crossing in front of the body: and then, later (fig. 9), with a much richer, fully three-dimensional description of the human figure comprised of data representing the parts, as well as procedural knowledge about how the parts articulate, how much the proportions can plausibly vary and something about posture.

Figure 8. Oil on canvas, 54 x 77, 1988.

Figure 9. Oil on canvas, 54 x 78, 1997.

The physical data represented key points within the body–articulation points, muscular attachments and so on–and the lines joining them then constituted a set of internal core figures equivalent to the child’s scribble. Each part of the body–upper arm, lower arm, torso, thigh and so on–was rendered as a closed form which resulted from drawing an encompassing line around each core.

That was fine as far as it went, but of course the human viewer doesn’t think of the body as a jointed doll, a collection of discrete parts; visually, the shoulder is an area of transition between the arm and the body, not a boundary, and while we may think of a nose as a conceptual unit it doesn’t offer much in the way of boundaries. In the two-and-a-half dimensional mode (fig. 8) noses were merely markings on the closed form of the head used to indicate the direction the head was facing. But the three-dimensional version was able to give a convincing account of the position of the head simply by the configuration of the outline, and the nose was drawing attention to itself by not being there.

In fact, it was the need to deal with noses that finally triggered the one modification of the underlying drawing mode, which was to have the program leave out parts of the outline–for any part, not just for noses–appropriate to the posture and the angle of view (fig. 10). That sounds simple; but since the functions that generated the outline were quite independent of the data from which the core figure was constructed, the modification required a total overhaul of how the data was represented in the program and a set of new functions to interpret the new form of the data.

Figure 10. “Theo,” oil on canvas, 24 x 34, 1992.

But hold on, you may say, what’s wrong with drawing the outline of a nose the way you draw the outline of the head? Good question. The answer is that there’s nothing wrong with it; 20^th Century art invented a number of drawing conventions that stretched the visual basis of representation a lot further than that. But it’s an important question; because if the program had no reason to think there was anything wrong either with a nose-less head or with a nose as a discrete unit stuck onto the front of the face, then there would be no reason why it should want to do anything about it. Once again, the relevant criteria rested on my own sense of stylistic consistency. Unless I could have given the program a rather sophisticated understanding of stylistic consistency–and there are few words in the vocabulary of art more difficult to understand than the word “style”–AARON could not have found anything wrong with what it had been doing, much less decide what to do about it.

I’ll come back to the issue of drawing, but now I want to turn my attention to color, which has occupied me for most of the past decade. For all of the images I’ve shown you so far I did the coloring myself, either directly on one of AARON’s drawings or by transferring one of its drawings onto canvas and coloring it there. I did it simply because I felt that the images needed color, yet from the very beginning there was a mild but persistent sense of absurdity about the fact that I seemed to be working for the program when I thought the program should be working for me.

So the major change that was initiated in the early ’90’s, leading to AARON’s current role as an independent and capable colorist, actually had a long gestation period in which the pressure was building, not by virtue of something the program was doing wrong, but by virtue of something the program wasn’t doing at all. Why hadn’t I tried to deal with the problem earlier? Quite simply, I didn’t know how. Now I do know how and I also know why I hadn’t known how.

Before that point, I’d thought I was just another expert unable to tell the long-suffering knowledge-engineer about his expertise, though in this case the expert and the knowledge engineer were the same person. It turned out not to be that at all; it was that I had failed to bring into focus something that I now consider so fundamental as to be almost a law of nature. It is that behavior is always shaped–both enabled and constrained–by our physical and our mental resources. Even if I could have described how I did coloring myself–and I had tried long enough to know it was quite unlikely that I could–that description would necessarily have been in terms of my own resources, and they were resources the program didn’t have.

I mean that the human painter has a sophisticated visual system that is capable of registering very small variations in color; and one that permits continuous feedback, reporting the way every new brush stroke modifies the overall color signature of a painting. The result of working within the capabilities provided by that resource is that, almost without exception, human colorists proceed by what I’d call expert trial and error, continuously modifying what has been put down as each new element is added. In fact, almost nobody has ever made a painting by making all the coloring decisions in his head and then writing down the instructions for someone else to execute. Our resources do not include any marked ability to build a reliable, stable internal representation of a color scheme that has never been actualized for the eye to see. In fact, I doubt that there’s much correspondence between the way we imagine color and the way we see it; which in itself would constitute a major reason for the colorist’s trial and error strategy.

AARON doesn’t have a visual system and it isn’t clear that a visual system would help much with respect to coloring for the human viewer unless its color responses closely matched those of the human system. What the program does have is precisely the resource that the human colorist lacks, which is the ability to build a stable internal representation; the ability to design a color scheme “in imagination” as it were, without ever seeing anything on the screen or on the canvas.

The realization that AARON’s resources were different from, but not necessarily inferior to, the human colorist’s resources proved to be the key that unlocked a door for me. Not the solution, obviously, because I had still to devise a rule-based strategy robust enough and flexible enough to provide good performance in an unpredictable range of compositions. But at least I was able to stop asking the entirely irrelevant question about how human colorists use color, and concentrate upon two pertinent questions: firstly, what do human colorists use color for?: and, secondly, what are the properties of color that can be identified and manipulated to any desired end?

The first question–what do we use color for?–has almost as many answers as there are inventive colorists; which isn’t as many as you might think. But it seemed to me that the first and perhaps the only mandatory obligation on the use of color, binding upon all colorists, is that it has to clarify the physical reading of the image. All the rest–emotional content, decorative value, whatever–have to fall into place behind that first obligation. In terms of representational art, that means that color has to help determine the difference between objects and spaces, help to establish boundaries and say where things are in the implied three-dimensional world.

The second question has a simpler answer: there are only three primary properties of color that can be manipulated. One is the hue–where the sample falls on the spectrum. One is it’s purity–how much energy falls within how narrow a part of the spectrum. And one is it’s brightness–regardless of its hue and its purity, where the sample would fall on a scale from black to white. Since the eye functions primarily as a brightness descriminator the most important property of color is not its hue, not its purity, but simply how light or dark it is.

Armed with some clear idea of what the program should be using color for and what it had to work with I was able to formulate a reasonably robust rule-set for determining the profiles of colors to be used for the individual areas of a painting to give adequate visual separation at the boundaries. And by ’93 (fig. 11) I was using not only AARON’s drawings, which hadn’t changed at all during this phase, but also its color schemes, as photographed off the monitor, to make paintings (fig. 12, 13).

Figure 11. “Clarissa,” oil on canvas, 42 x 54, 1992.

Figure 12. Screen image, 1993.

Figure 13. Oil on canvas, 54 x 78, 1993.

The last part of this story is a bit more complicated.

As I emphasized at the beginning, no artist is very far from the problems of dissemination, exhibiting his work. And it didn’t take much, all the time I’d been working on color on the monitor, to recognize that a single monitor in an art gallery doesn’t make much of an exhibition. Given that I’d solved the problem with respect to drawing by building several generations of drawing machines, the answer now–the much too obvious answer–was to build a painting machine.

I won’t go into detail about the hardware, a large xy device that carried a small robot arm on its beam, complete with a simple hand. It could mix its colors from a palette of water-based dyes by having the hand open the taps on the dispensers for measured times. And it could pick up and use brushes of different sizes.

After making a drawing with waterproof black ink the machine would move a cup from one dispenser to another to mix the required color, deposit the cup where the hand could reach it as it moved around the table, pick up a brush and start filling in the shapes. When it had finished with a color it would empty the cup and wash it out ready for the next color, then wash out the brush and start over again. It took about six hours to complete a painting about four feet by six and in a show at the Computer Museum in Boston it did one painting a day for seven weeks.

On the software side, the first problem was to characterize the dyes mixtures, about eleven hundred of them, in terms of the three fundamental properties. But characterizing them was one thing, mapping them onto the rgb specifications generated for the screen another thing entirely. The guns that activate the screen are effectively light sources, so that mixing is additive, whereas physical materials act as filters and remove light. Mixing dyes is subtractive mixing. (fig. 14) You get yellow on the screen by adding red and green, while if you mix red and green pigments you get dirty brown.

Figure 14. Additive mixing and subtractive mixing.

There was another difference, too, that proved to be much more troublesome.

As I’ve said, the most important property of color is its brightness, which is manipulated on the screen by increasing or decreasing the output of the three guns. Assuming the dyes are being applied over white paper, you increase brightness by dilution, while you can’t decrease brightness at all except by adding a darker dye and thus changing the color.

By the time I’d made and measured the samples of the dye mixtures themselves I had neither the resources nor the inclination to make enough dilution samples for each of those eleven hundred samples to be useful. I made some not very sound assumptions about how the dyes could be grouped and I provided dilution data for the groups rather than for the individual mixtures and the output during the show (fig. 15) varied from very good to clearly incompetent. I realized later that there was a simpler way of establishing dilution, but by then the program was already changing direction.

Figure 15. Machine painting, ink and dyes on paper, 53 x 70, 1995.

There were three painting machines in all; the first served only to tell me how little I knew about engineering; the second was the one used in the Computer Museum and the third, the much improved model I’ve been showing you, was filmed for several television programs but never left my studio until it went to its final home in the Museum of Computing History late last year. As part of a questionable balance-sheet of energy expended versus advantage gained, I had to recognize that the better the machine got, the more it risked directing attention away from the program instead of toward it. That risk was already evident in the Computer Museum show, where people were interested in the painting but delighted by the machine washing out its own cups–housework being the high point for robotics, obviously–and it was becoming increasingly apparent as I found more and more people referring to AARON as a robot artist; as I got more and more requests for pictures of AARON rather than pictures by AARON

This is not to say that the painting machine represented a dead end and had no significant role in AARON’s development. For one thing, I hadn’t gone to the trouble of building the machine to have it do simple-minded raster-scan filling and the algorithm I devised for generating the brush movements has become AARON’s standard way of filling in color.

And, to come back to drawing–or, rather, to the relationship between drawing and coloring–the most important gain was the realization that, given black lines to hold the colors in place, the choice of color wasn’t nearly as critical as I had thought it should be. On the machine the lines served a critical physical function; it simply wasn’t possible to have edges established by the colors themselves, since the water-based dyes would run into each other and confuse more than they clarified. Once I was no longer using the machine I could, in principle, do anything I wanted to do. What I wanted–what I had wanted as a painter for several decades–was to have color play a critical role, not merely a supporting role, in the structure of images. The black lines of AARON’s drawing were preventing that happening.

The current version of the program does not draw lines (figure 16) and has gone some way toward that goal. But it was not enough simply to leave out the lines; once they were eliminated, the edges of forms had to be established by the colored brush marks themselves. However, my painting algorithm controlled the path of the brush, not the edge of the brush, and whichever brush the program chose there would always be some concavities in the outlines too small for the brush to get into, which had been left blank in the earlier versions and on the painting machine . The problem wasn’t intractable, but it did involve substantial modifications to the underlying algorithm.

Figure 16. Screen image, 2002.

As to the coloring part of the change; as I indicated earlier, constraining areas of color within black boundary lines minimizes the degree to which the reading depends upon the colors. When you have two patches of color in contact with each other, on the other hand, there are optical effects generated by the color contrast at the edges that were not there before. And in this case the difference was extreme enough that I had to make substantial changes, not to the values assigned to the colors, but to the rules that assigned those values.

That brings us up to date with respect to AARON’s history and now I want to make some general observations.

Firstly: each phase of AARON’s development has involved modifying, but never abandoning, what has gone before. So, for example, AARON still structures its forms the way it has always done, though the delineation no longer shows in the output.

Secondly–and this may be simply another way of saying the same thing–there have been no false starts in AARON’s development. That doesn’t mean that I’ve been clever enough always to pick the right thing to do, but simply that this is a game that has to be played with the cards I’ve dealt myself. Given that human artists don’t suddenly change their style from one day to the next or one decade to the next, we might assume it to be a characteristic of directed development for the human. For the program, I would assume it to be rather difficult characteristic to emulate and, whatever other difficulties might arise, it certainly couldn’t emulate it unless it had appropriate access to previous states of the game.

That rules out AARON in its present form, if not programs in general, because AARON doesn’t have any archival record of its work. It doesn’t have one for the simple reason that I’ve never known what should go into one. As I suggested earlier, if the program is to move forward by assessing its efforts against various criteria, then it needs a representation of those aspects of the work that bear upon the criteria, including the decision history that resulted in the particular example.

Lots of problems here, the principle one being that AARON’s work is intended for human use and its criteria must consequently reflect what the human viewer responds to in an image. That is not the result of a single decision, it rests upon the interaction of thousands of decisions. Even if I were to accept the conventional view that the program would need to be trained by a human expert, it would have no way of knowing in detail what the human expert is responding to when he declares that one image is better than another, because the human expert doesn’t know,

As you can imagine, I’ve frequently been told that that’s exactly the kind of thing neural nets are supposed to do. That leads to the third general observation, which is that all of the changes of direction I’ve described, and all the others I haven’t described, have involved sizeable modification to the program itself, often down to the invention of completely new algorithms. I’ve never done anything that affected the program’s development significantly by adjusting the values of variables.

The fourth observation I want to make, and perhaps the most crucial with respect to my contention, is that no change of direction was ever made because there was anything wrong with the program’s output. There was nothing wrong with the output of AARON’s earliest phase and there is nothing wrong with the colored drawings that are now being generated on tens of thousands of machines around the world. Well, I don’t mean there was nothing wrong exactly; I mean there was nothing wrong aesthetically with the individual images; no failure to satisfy aesthetic criteria internal to the program that might have prompted it to make changes. To restate my original contention; the criteria that have been most in evidence in requiring changes of direction are at the upper end of the chain and external to the program itself.

I’d like to finish up this survey, then, not by looking at individual changes, but by trying to say a little about why AARON exists at all and why it took the path it did.

In all the museum shows I did with AARON using drawing machines, I offered its drawings for sale: $25 for a signed original; an hundreds were sold in places dedicated to showing things that ordinary people could not dream of owning. I’ve discussed the reason for building the drawing machines: I wanted to show what was happening, not just a few examples of the things that fell out in the process. But why on earth did I want to sell drawings at two orders of magnitude less than I could sell my own drawings for in the conventional art world?

You may be inclined to suppose it was an experiment in marketing, but clearly that couldn’t have been the case. I had no reason to suppose that I could sell a hundred of AARON’s drawings for every one of my own. With no precedent to look to, there were no reasons to suppose anything at all, except perhaps that making an anti-elitist move like this one might very well endanger my standing in the conventional art world. Which, of course, it has. After all, I could hardly expect approval for saying that a machine could do what we normally assume only talented human artists can do.

Why would an artist who had spent twenty years of his life making luxury objects for wealthy people feel compelled to make a socio-political move of this sort? I never had anything against wealthy people buying luxury objects–including the ones I made–and I still don’t. On the contrary, and regardless of why they do it, wealthy people are actually fulfilling a rather important role on behalf of the culture; we wouldn’t have museums full of great art unless someone had commissioned it and bought it. I do have to confess to having been quite fed up with an official art establishment that existed, as all establishments do, to maintain what has been established: which, in London at the end of the ‘sixties, included me. But being fed up doesn’t necessarily lead to computing, especially for someone with nothing of the sort in his background and at a time when computing meant sitting up all night punching IBM cards.

I’ll skip any amateur psychoanalytical speculation. Whatever the reasons for AARON’s existence and its history, it’s surely obvious that they are not to be found in the individual works produced by the program, but in my own background, my own history. What, then, would be involved for a program to have its own history of self-directed development?

Now I’m not much of a programmer and writing a program so that it can rewrite itself is way beyond my capacities; most of the time I don’t even know how to rewrite things myself without much stumbling and a great deal of debugging. If my more capable colleagues want to assure me that one can indeed write programs that can figure out new algorithms for themselves, rewrite themselves, debug themselves and assess the results by themselves, I’m quite prepared to believe them.

I am not yet prepared to believe, however, that a program can have the reasons for doing so: that it can embody enough of some equivalent to the chain of criteria that determines what human artists do that it could be capable of the kind of self-directed development I’ve been describing.

On the other hand…

My confidence in the logic behind this conclusion needs to be qualified, for it’s quite possible that I’m not thinking about this in the right way at all. So I think I must end by acknowledging that there may be other ways of thinking about things; and, in the process, clear up what may seem to have been an oversight. I mean that AARON is one of the very few programs in existence with any claim to creativity and I haven’t used the word once. Isn’t creativity, after all, precisely at the core of this discussion?

Creativity is a word I’ve never used if I could avoid it and I’ve certainly never made any such claim on AARON’s behalf. On the contrary, while friends and well-wishers like Margaret Boden and Bruce Buchanan have continued to cite AARON as a prime example of programmed creativity, I’ve maintained steadfastly that it’s nothing of the sort, citing precisely the fact that the program is incapable of the kind of development we would require of a major human artist. Of course, that could itself be regarded as a definition of sorts; one that not too many human artists could satisfy, by the way, and one that may not map easily onto other domains, which spread a long way down from the Einsteins and Michelangelos once we leave the more refined strata of academic research.

If I had to place AARON somewhere on a spectrum with creative napkin folding at one end and general relativity at the other, I would certainly place it well above creative napkin folding, so perhaps its time to soften my hard-nosed position, that AARON is not creative at all, and concede that it’s creative with a small ‘c’ but not Creative with a big ‘C’. That little dance step brings a nagging question into focus, however, and I don’t think we know how to answer it. Is there, in fact, a continuous spectrum from creative napkin folding to general relativity? Is creativity with a small c a small example of Creativity with a big C, or are the two entirely different? If they are entirely different, then we haven’t even begun on the search for programmed Creativity–with a big C–and I certainly don’t have any clue as to how to begin.

If there is a continuum, then that’s another story. I’m reminded of a conversation a long while ago with my good friend Ed Feigenbaum. I’d written something for an exhibition catalog text along the lines that machines couldn’t think, but they were capable of making extremely complex decisions. Ed was quite upset and insisted that machines could think. I objected that he was voicing an article of faith rather than a demonstrable fact and he replied that when machines make much more complex decisions than they were then able to make, we would all say they were thinking and we wouldn’t give the matter another thought.

And so it has come to pass. And it may well be that as AARON becomes more complex, as the range of its abilities and its knowledge of the things in the world and the determinants to what it should do about them becomes much more complex than is now the case and without regard to the nature of the complexity, perhaps even I will be prepared to say that it’s Creative–with a big C–and not give the matter another thought.

I have every reason to hope so, given that I don’t know how else to proceed.

On the other hand… who knows what tomorrow will bring…?