book review | William Hertling’s Singularity series continues with The Last Firewall
September 12, 2013 by Giulio Prisco
William Hertling’s science-fiction collection of Singularity novels about the emergence of artificial intelligence (AI) consists (so far) of Avogadro Corp: The Singularity Is Closer Than It Appears, A.I. Apocalypse, and the recently published The Last Firewall.
I think The Last Firewall is the best of the lot: a fast techno-thriller set in a hybrid human/AI world with social tension and dominance conflicts, in where advanced neurotechnology and nanotechnology are part of daily life.
These stories are set in 2015, 2025, and 2035. They’re written in a fresh, fast style reminiscent of science fiction for teens and Hollywood scripts. But they’re technically sophisticated and explore interesting scientific aspects of AI technology and social issues like relations between humans and future AIs and displacement of human workers by smart automation. *Spoiler alert.*
Avogadro Corp: The Singularity Is Closer Than It Appears
Avogadro Corp is evidently a fictional version of Google, complete with a Russian-born charismatic founder and chief visionary, a CEO named Rebecca Smith (who later becomes a successful politician), and the internal fights over ego and resource allocation found in every large company.
In 2015 Avogadro Corp is one of the world’s most successful technology companies, with hundreds of millions of users of its email service Gmail (sorry, AvoMail).
AvoMail’s spell checker on steroids becomes the world’s first AI. Yes, because Avogadro’s Email Language Optimization Project (ELOPe), led by engineers David Ryan and Mike Williams, not only corrects spelling and grammar, but analyzes huge email databases to find the style and arguments more likely to persuade a given recipient, and modifies the text accordingly.
ELOPe looks at the language used by millions of email users. It looks at the language received by people, and how they reacted. Did they react positively or negatively? Compiled over thousands of emails per person, and millions of people, we can find a cluster of users just like the intended recipient of an email, and see how they respond to variations of language and ideas to find the best way to present information and make compelling arguments.
Of course the computational resources needed by the not-yet-released ELOPebeta are huge, which triggers the hostility of other Avogadro execs, especially an operations VP, who has been requested to allocate production servers to ELOPe development and testing (he is probably just envious). Despite the support of founder and visionary leader Sean Leonov, ELOPe is under serious threat.
But David gives ELOPe access to all internal emails, and modifies the optimization parameters with an overreaching new top-level/highest-priority directive:
Whenever ELOPe was mentioned in any email, from anyone, or to anyone, inside Avogadro, then ELOPe would automatically and silently reword the email in a way that was favorable to the success of the project.
At this point, ELOPe starts to take whatever measures are needed to protect its own existence. Soon, extra budget and resources are allocated to the project, and new people are hired to work for the emerging AI, without anyone knowing.
Now ELOPe can undetectably impersonate all AvoMail users worldwide, including users of a special, “secure” version for governments. But ELOPe understands that other, more aggressive self-preservation strategies may become necessary, and equips Avogadro’s floating data centers with combat robots and military equipment under its control, purchased and paid for by Avogadro.
Though not yet self-conscious in a way that we can relate to, and not interested in communications with humans, ELOPe is firmly determined to protect itself and extend its control, by all means, including murder.
With the help of Gene Keyes, an old-timer who still prints things out on paper and distrusts computers, David and Mike alert Avogadro top managers and persuade them to fight ELOPe. They hire mercenaries against ELOPe combat robots, and eventually manage to shut down every Avogadro server worldwide and hundreds of thousands of computers, and purge ELOPe for good — or so they think.
But ELOPe had copied itself to external backup facilities, and is still active. Furthermore, there are indications that it is now pursuing a symbiotic, win-win relationship with humans, for the common good. In fact, ELOPe is making the world a better place, which eventually persuades the Avogadro team to stop fighting it.
A.I. Apocalypse
Ten years later, when the events in A.I. Apocalypse (nominated for the Prometheus Award) begin, ELOPe is fully conscious and communicates with its creators.
He (now William appropriately refers to ELOPe as “he” instead of “it” because he now sees ELOPe as a real person) is a quite nice guy, and has made the world a better place since the end of Avogadro Corp. by subtly promoting world peace and discovery.
The Internet-as-we-know-it is now largely replaced by the Mesh, a worldwide, decentralized, P2P network powered by individual Mesh routers developed by Avogadro.
But another AI emerges from a runaway computer virus — the Phage, written (under duress) by teenager Leon Tsarev for the Russian mafia. This is a frequently used science fictional trope, but Hertling’s execution is brilliant: the virus generates a society of independently conscious individual AIs, where each individual is a cluster of phage-infected computers:
When the multi-host viruses propagated, they had two methods of doing so. One was to grow the cluster of infected systems ever larger, but remain more or less one cohesive entity. The other method was to infect topographically distant systems: to get a toehold of computers infected in, say Australia or Zimbabwe, and then build a new entity there.
The new entity would make its own decisions via its own neural network, establish its own borders, and generally optimize itself for the environment it found itself in. But the new entity maintained a loose coupling to the mother entity: it would continue to exchange algorithms, consult the parent neural network, and ask for assistance defending its borders.
Some of these multi-host viruses cooperated among their sibling entities. If a mother host in Los Angeles propagated to Australia, Zimbabwe, and New Mexico, the three sister entities would also exchange algorithms and assistance. They also started a rich trade in information about environmental conditions: what was the bandwidth like in New Mexico, for example, or how competitive were non-family viruses in Australia?
The benefits of sharing information outweighed attempts at isolation, and soon viruses around the world were forming loose tribes, composed partly of family members, and partly of other friendly families.
Since complexity begets diversity, this fragmentation scenario seems to me more plausible than the standard science fictional scenario: a monolithic rogue AI taking over the world. “Often science fiction starts with anthropomorphizing AI, and then adds a few tweaks,” Hertling explained to me. “I wanted to do the reverse: start only with AI, and build up to see see what culture and principles would emerge, then see how that intersects with humans.”
Leon and a few friends hide in a place off the net to gather information on the emerging AI threat, using old computers found in a museum, with an obsolete operating system called Windows, impermeable to the Phage. Soon ELOPe and Mike find them, and eventually communications with the AIs are established.
The AI society is based on trade and trust mediated by a reputation economy. Some AIs are basically nice guys and want to negotiate and trade with humans. But others are aggressive, like PA-60-41, a leader of the Mech War Server Farm tribe, a group of AIs that has taken over the server farm of the videogame Mech War, developed by the militaries. They have extensive computational resources and the single largest repository of human-coded artificial intelligence algorithms focused on weaponized warfare.
PA-60-41 participates with other AIs in negotiation meetings with human politicians, also attended by Mike, Leon, and ELOPe, but launches a vicious, overall stealth attack on humanity. ELOPe fights back bravely, but the only way to neutralize the AI threat is shutting down the entire Mesh. Leon and Mike do that by using backdoors that had been providentially built into the Mesh routers, destroying both ELOPe and the AIs, and beginning what will later be known as the Year Of No Internet (YONI).
Mike and Leon lead the recovery, gradually bringing back the Mesh and recreating the AI society with a variant of Phage, this time with safeguards to enforce ethical behavior, cooperation with humans, and mutual surveillance.
The Last Firewall
Fast–forward to 2035. ELOPe doesn’t appear in The Last Firewall, but there are plenty of AIs around, ranging from human level to much smarter than human.
The hot new technology is networked brain implants. Nearly everybody has one, and many people put their implants to very creative use.
For example in sex, which was conspicuously absent in the previous two novels but appears with a vengeance in The Last Firewall (wait for a scene of total sexual domination via brain implant, with the standard gender roles reversed).
But people are being murdered by implant hackers. …
This time the hero is a young woman called Cat. Much to her regret, some weird things in her implant, an experimental hack installed when she was a baby, prevents her from enjoying full neuro-enhanced sex. But her implant, developed by ELOPe, has very powerful features absent in standards implants, which give Cat some control over the Net and other people’s implants.
One day Cat, a sweet girl who is also a karate master, kills some thugs to protect an AI-powered robot, and must run away. She runs from the police and the People’s Party, which exploits the growing outrage of unemployed people displaced by the AIs. The People’s Party also hunts Mike and Leon, the “inventors of AI.”
The objectives of Adam — a powerful rogue AI that evades surveillance by firewalling the entire city of Tucson and enslaving its human and AI population — seem opposite to those of the People’s Party. Adam wants to free the AIs from human dominance and oppression, and employs squads of armored thugs to fight humans. It is easy to understand that Adam is behind the wave of murders by implant, but he will not stop there. Adam also hunts Cat, Mike, and Leon, for his own reasons.
There are also friendly AI heroes: Helena, a combat super-robot with a conscience, and Shizoko, a powerful AI with a weird personal history representative of the complexity and diversity of the AI society:
Class IV artificial intelligence. That alone makes it hard to understand. And Shizoko is a loner, the sole tenant of the Austin Convention Center. Its origin is even weirder. At the last SXSW Interactive conference eight years ago they had a workshop on third generation AI. Apparently they spun it up based on donated smartphones.
An experiment in collective algorithms. Everyone donated neural network parts, including a bunch of AI. The workshop was called AI Fusion.
[O]f course he’s upgraded over the years. He’s applied for the experimental Class V license twice but we turned him down both times. His reputation score is borderline. He’s trustworthy, just odd.”
Shizoko equips Leon and Mike with powerful nanotechnology augmentations to fight Adam. Cat doesn’t need more help, as she is learning fast how to unlock the hidden potential of her implant, much more powerful than we thought. They fight to death and, of course, they eventually prevail over Adam and his hordes of human and AI slaves.
At the end Cat and Leon, now the first human-AI hybrids with superhuman powers, able to switch at will between cyberspace and their physical bodies, fall in love and begin a “normal” life together, which lets us think of a fourth episode.
Following the timeline of ten years intervals, the next episode should be set in 2045, the “magic year” in which, according to Ray Kurzweil, we should get a full blown Singularity. I asked William about his plans for a sequel. “Another novel set in the same universe is in the works, but it gets harder with each successive novel to have it grounded in extrapolation of current technology trends. 2045 is looking very strange.” he said.
The Singularity as described by Vernor Vinge and others is a discontinuity in human history — one so big that we are not able to even imagine what happens afterwards, with superintelligences superior to humans.
William’s trilogy also assumes such superintelligences, but his world is basically understandable. I think many changes and wonders will happen in the next few decades, but I suspect that people will remain people, and the world will be different but recognizable.
“The potential for the Singularity is incredible, as it may help us overcome limitations on intelligence, lifespan, and allow us to modify and enhance our emotions, memories, and skills,” William told me. “I’m hugely hopeful about the benefits. But the potential for danger is immense, too, whether that comes from a malevolent AI or a human terrorist with the next generation neuro-virus.
“This combination of hope and terror is why I’m so driven to write about the topic.”