Over the next ten years, AI will be the greatest force amplifier in history. That is from “The coming wave “. The book is a combination of a lot of books. Such as”.
- “This Is How They Tell Me the World End”
- “Exponential: How Accelerating Technology Is Leaving Us Behind and What to Do About It”
- “The big nine”
- “The synthetic age”
- “Falter”
We have a choice
We are faced with a choice. A choice between a future of unparalleled possibility and a future of unimaginable peril. Technology has a clear, inevitable trajectory: mass diffusion in great roiling waves. With AI, containment is not possible. Neither can we contain synthetic biology, CRISPR, nanotechnology, robotics, 3D printing, IoT, 6G, social media, etc. General-purpose technologies enable seismic advances in what human beings can do. For example, a single person today likely has the capacity to kill a billion people. All it takes is motivation. You should also read “Thank you for being late“.
General purpose technologies
Past general-purpose technologies were axes, fire, language, farming, writing, steam, electricity, IT, etc. and as with everything, the speed and frequency of the waves are increasing. And they become self-enhancing by a combination of the waves. Lego on steroids. The dialogue of science and technology produces a chain of insights, breakthroughs, and tools that build and reinforce, over time, productive recombinations that drive the future. As you get more and cheaper technology, it enables new and cheaper technologies downstream. Uber was impossible without the smartphone, which itself was enabled by GPS, which was enabled by satellites, which were enabled by rockets, which were enabled by combustion techniques, which were enabled by language and fire.
Rampant proliferation
Since the early 1970s, the number of transistors per chip has increased ten million-fold. Their power has increased by ten orders of magnitude—a seventeen-billion-fold improvement. It created an even more mind-boggling proliferation: data, up twenty times in the decade 2010–2020 alone. Billions of hours of raw human life are consumed, shaped, distorted, and enriched by these technologies. Once they gather momentum, they rarely stop. Mass diffusion and raw, rampant proliferation are technology’s historical default.
Unpredictable consequences
Technology exists in a complex, dynamic system (the real world), where second, third, and nth-order consequences ripple out unpredictably. Gutenberg just wanted to make money printing Bibles. Yet his press catalyzed the Scientific Revolution and the Reformation and became the greatest threat to the Catholic Church since its establishment. Any technology is capable of going wrong, often in ways that directly contradict its original purpose. Hence the need for containment. Containment encompasses regulation, better technical safety, new governance and ownership models, and new modes of accountability and transparency, all as necessary (but insufficient) precursors to safer technology.
Containment
Technical containment refers to what happens in a lab or an R&D facility. Then come the values and cultures around creation and dissemination that support boundaries, layers of governance, acceptance of limits, a vigilance for harms and unintended consequences. Last, containment includes national and international legal mechanisms: regulations passed by national legislatures and treaties operating through the UN and other global bodies. The last examples of containment are nuclear weapons and biological and chemical warfare. However, despite the moratoriums, the world’s chemical and biological capabilities grow every year; should anyone perceive the need to weaponize them, it would be easier than ever.
The next waves
The next wave is about the technology of intelligence. From Watson to AlphaGo, AlphaZero, ChatGPT, Deepmind and Q*. AlphaGo is the one that should make you sit up. But after three moves in Go, there are on the order of 200 quadrillion (2 x 1015) possible configurations. It’s often said that there are more potential configurations of a Go board than there are atoms in the known universe; one million trillion trillion trillion trillion more configurations. The book to read is “Maniac” by Benjamín Labatut. A book about the development of the atomic bomb, Von Neumann probes, symbionts, digital DNA and AlphaGo. To win, AlphaGo had to go beyond raw computing power. Call it intuition or creativity……
Technology abstraction
Technology is beginning to operate at a higher level of abstraction. Foundational breakthroughs in synthetic biology have enabled us to sequence, modify, and now print DNA. While AI and synthetic biology are the coming wave’s central general-purpose technologies, a bundle of technologies with unusually powerful ramifications surrounds them, encompassing quantum computing, robotics, nanotechnology, and the potential for abundant energy, among others. Creating a Cambrian explosion of innovation.
AI is eating the world
Where AI (not software) is eating the world. Steadily, AI is leaving the realm of demos and entering the real world. Within a few years, AIs will be able to talk about, reason over, and even act in the same world that we do. Their sensory systems will be as good as ours. It means that AI will become inextricably part of the social fabric. AI isn’t “emerging” anymore. It’s in products, services, and devices you use every day. Across all areas of life, a raft of applications rely on techniques that were impossible a decade ago.
This is just the start
All of this is just the start. We are only beginning to scratch at the profound impact large language models are about to have. If DQN and AlphaGo were the early signs of something lapping at the shore, ChatGPT and LLMs are the first signs of the wave beginning to crash around us. LLMs aren’t just limited to language generation. What started with language has become the burgeoning field of generative AI. They can, simply as a side effect of their training, write music, invent games, play chess, and solve high-level mathematics problems.
Singularity
As it keeps scaling, the ability to complete a multiplicity of tasks at our level and beyond comes within reach. AI will keep getting radically better at everything, and so far, there seems to be no obvious upper limit on what’s possible. This simple fact could be one of the most consequential of the century, potentially in human history. Ultimately leading to Singularity. Over the last decade, intellectual and political elites in tech circles became absorbed by the idea that a recursively self-improving AI would lead to an “intelligence explosion” known as the Singularity. Conscious superintelligence? Who knows. Make no mistake: they are on their way and are already here in embryonic form.
Playing Lego
Combine this with all the other technologies, and you get DNA printers, genetically enhanced biofuels, new molecules, viruses that produce batteries, proteins that purify dirty water, organs grown in vats, algae that draw down carbon from the atmosphere, plants that consume toxic waste, etc. Eventually, it might be possible to reconfigure ourselves to enhance our immune responses. That, in turn, might open the door to even more ambitious experimentation like longevity and regenerative technologies, already a burgeoning area of research. Initial work suggests memory can be improved and muscle strength enhanced. It won’t be long before “gene doping” becomes a live issue in sports, education, and professional life.
What if our supply chain was just biology
Want to make some washing detergent, a new toy, or even grow a house? Just download the “recipe” and hit “go.” In the words of Elliot Hershberg, “What if we could grow what we wanted locally? What if our supply chain was just biology?” Eventually, computers might also be grown as well as made. Remember that DNA is the most efficient data storage mechanism we know—capable of storing data at millions of times the density of current computational techniques with near-perfect fidelity and stability. Some scientists are beginning to investigate ways to plug human minds directly into computer systems.
Machines coming alive
Welcome to the age of biomachines and biocomputers, where strands of DNA perform calculations and artificial cells are put to work. Where machines come alive. Welcome to the age of synthetic life. All intelligence to date has come from life. Now, we have synthetic intelligence and artificial life. Combine that with robotics. Robots are the bridge for AI, their interface with the real world. If AI represents the automation of information, robotics is the automation of the material, the physical instantiations of AI, a step change in what it is possible to do. Mastery of bits comes full circle.
Quantum computing
Throw quantum computing in the mix. Google’s machine used an understanding of quantum mechanics to complete a calculation in seconds that would, it said, have taken a conventional computer ten thousand years. Like AI and biotech, quantum computing helps speed up other elements of the wave. And yet, even the mind-bending quantum world is not the limit.
Nanotech
As the elements of AI, advanced biotechnology, quantum computing, and robotics combine in new ways, prepare for breakthroughs like advanced nanotechnology, a concept that takes the ever-growing precision of technology to its logical conclusion. The ultimate vision of nanotechnology is one where atoms become controllable building blocks, capable of automatically assembling almost anything. Nanomachines would work at speeds far beyond anything at our scale, delivering extraordinary outputs: an atomic-scale nanomotor, for example, could rotate forty-eight billion times a minute. Scaled up, it could power a Tesla with material equivalent in volume to about twelve grains of sand.
Technology explosion
Each technology is subject to a vicious hype cycle. Each is uncertain in development and reception. Each is surrounded by technical, ethical, and social challenges. We are reaching the decisive point of what, in geological or human evolutionary timescales, is a technological explosion unfolding in successive waves, a compounding, accelerating cycle of innovation steadily getting faster and more impactful, breaking first over a period of thousands of years, then hundreds of years, and now single years or even months.
One point of failure cascading
A single AI program can write as much text as all of humanity. A single two-gigabyte image-generation model running on your laptop can compress all the pictures on the open web into a tool that generates images with extraordinary creativity and precision. A single pathogenic experiment could spark a pandemic, a tiny molecular event with global ramifications. One viable quantum computer could render the world’s entire encryption infrastructure redundant. The very scale and interconnectedness of the coming wave create new systemic vulnerabilities: one point of failure can quickly cascade around the world.
You need to worry
Some things to worry about:
- Internal research on GPT-428 concluded that it was “probably” incapable of acting autonomously or self-replicating. Still, within days of launch, users had found ways of getting the system to ask for its own documentation and to write scripts for copying itself and taking over other machines.
- AI, the neural networks moving toward autonomy, are, at present, not explainable. Engineers can’t peer beneath the hood and easily explain what caused something to happen. GPT-4, AlphaGo, and the rest are black boxes, their outputs and decisions based on opaque and intricate chains of minute signals.
- There is a strong case that, by definition, a superintelligence would be impossible to control or contain. There comes a point where technology can fully direct its own evolution; Homo technologicus may end up being threatened by its own creation.
- Today, China has an explicit national strategy to be the world leader in AI by 2030. It’s not just AI, either. From cleantech to bioscience, China surges across the spectrum of fundamental technologies, investing at an epic scale,
- Imagine robots equipped with facial recognition, DNA sequencing, and automatic weapons.
- Once militaries are fully automated, the barriers to entry for conflict will be far lower.
- Future robots may not take the form of scampering dogs. Miniaturized even further, they will be the size of a bird or a bee, armed with a small firearm or a vial of anthrax. They might soon be accessible to anyone who wants them. That is what bad actor empowerment looks like.
- By 2028, $26 billion a year will be spent on military drones, and at that point, many are likely to be fully autonomous (and self-learning).
- AI-enhanced weapons will improve themselves in real-time.
- Terrorists mount automatic weapons equipped with facial recognition to an autonomous drone swarm hundreds or thousands strong, each capable of quickly rebalancing from the weapon’s recoil, firing short bursts, and moving on.
- A mass murderer decides to hit a huge political rally with drones, spraying devices, and a bespoke pathogen.
- Just as systems like AlphaGo learn unexpected strategies from millions of self-played games, so too will AI-enabled cyberattacks. However much you wargame every eventuality, there’s inevitably going to be a tiny vulnerability discoverable by a persistent AI.
- Ask yourself, what happens when anyone has the power to create and broadcast material with incredible levels of realism?
- Russian agents created no fewer than eighty thousand pieces of organic content that reached 126 million Americans on their platforms during the 2016 election. AI-enhanced digital tools will exacerbate information operations like these, meddling in elections, exploiting social divisions, and creating elaborate astroturfing campaigns to sow chaos.
- U.S. risk assessment from 2014 estimated that over a decade, the chance of “a major lab leak” across ten labs was 91%; the risk of a resulting pandemic was 27%.
- What if AI can perform a large majority of white-collar tasks more efficiently?
- Consider that the combined revenues of companies in Fortune’s Global 500 are already at 44 percent of world GDP. These companies already control the largest clusters of AI processors, the best models, the most advanced quantum computers, and the overwhelming majority of robotics capacity and IP. Read “The four“.
- Every glance, every hurried message, every half thought registered in an open browser or fleeting search, every step through bustling city streets, every heartbeat and bad night’s sleep, every purchase made or backed out of—it is all captured, watched, tabulated. The only step left is bringing these disparate databases together into a single, integrated system: a perfect twenty-first-century surveillance apparatus. The preeminent example is, of course, China.
- The coming wave could make a range of small, state-like entities a lot more plausible. Contrary to centralization, it might actually spur a kind of “Hezbollahization,” a splintered, tribalized world where everyone has access to the latest technologies, where everyone can support themselves on their own terms, where it is far more possible for anyone to maintain living standards without the great superstructures of nation-state organization.
- AI hackers and autonomous drones will be available to private security groups as well.
- Multiple ownership structures will exist in tandem: technology democratized in open-source collectives, the products of today’s corporate leaders or insurgent blitz-scaling start-ups, and government-held, whether through nationalization or in-house nurturing. All will coexist and coevolve, and everywhere, they will alter, magnify, produce, and disrupt flows and networks of power.
War, technology escaping, bad actors, concentration of power, winners take all, surveillance, asymmetry, Hezbollahization, dystopia, redistribution of power, etc.
Dark AI
Think about an ACI capable of easily passing the Modern Turing Test but turned toward catastrophic ends. Advanced AIs and synthetic biology will not only be available to groups finding new sources of energy or life-changing drugs; they will also be available to the next Ted Kaczynski. Ask it to suggest ways of knocking out the freshwater supply, crashing the stock market, triggering a nuclear war, or designing the ultimate virus, and it will. Imagine scenarios where AIs control energy grids, media programming, power stations, planes, or trading accounts for major financial houses. Once fast-evolving, self-assembling automatons or new biological agents are released out in the wild, there’s no rewinding the clock.
George Orwell’s 1984
The panopticon is becoming possible. Billions of devices and trillions of data points could be operated and monitored at once, in real-time, used not just for surveillance but for prediction. If AI, biotech, quantum, robotics, and the rest of it are centralized in the hands of a repressive state, the resulting entity would be palpably different from any yet seen.
Techno optimism
The book moves towards abundance and techno-optimism. Renewable energy will become14 the largest single source of electricity generation by 2027. Fusion is on its way. AI has helped design an enzyme that can break down the plastic clogging our oceans. Sustainable, scalable batteries need radical new technologies. Quantum computers paired with AI, with their ability to model down to the molecular level, could play a critical role in finding substitutes to conventional lithium batteries.
Power to the people
Today, no matter how wealthy you are, you simply cannot buy a more powerful smartphone than is available to billions of people. This phenomenal achievement is often overlooked. In the next decade, access to ACIs will follow the same trend. Those same billions will soon have equal access to the best lawyer, doctor, strategist, designer, coach, executive assistant, negotiator, and so on. Everyone will have a world-class team on their side and in their corner. That will be the greatest, most rapid accelerant of wealth and prosperity in human history. It will also be one of the most chaotic.
Production revolution
The author predicts that in a few decades, most physical products will look like services. Zero marginal cost production and distribution will make it possible. When you combine all the facets of the coming wave, from the design, management, and logistical capabilities of AI to the modelling of chemical reactions enabled by quantum computing to the fine-grained assembly capabilities of robotics, you get a wholesale revolution in the nature of production. Foods, drugs, home products, indeed almost anything might be 3-D printed, or bio-produced, or made using atomically precise manufacturing close to or at the site of use, governed by sophisticated AIs fluidly working with customers using natural language. You simply buy the execution code and let an AI or robot do the task or create the product. The world of tomorrow will be a place where factories grow their outputs locally, almost like farms in previous eras. Drones and robots will be ubiquitous. The human genome will be an elastic thing, and so, necessarily, will be the very idea of the human itself.
We need technology
However, the author makes the point that the idea that technology alone can solve social and political problems is a dangerous delusion. But the idea that they can be solved without technology is also wrongheaded. A school of naive techno-solutionism sees technology as the answer to all of the world’s problems. Alone, it’s not. No one should pretend that technology is a near-magical answer to something as multifaceted and immense as climate change. However, the idea that we can meet the century’s defining challenges without new technologies is completely fanciful.
Governance
Finally, we move into governance. Every previous wave of technology has had profound political implications. The introduction of new technologies has major political consequences. Just as the cannon and the printing press upended society, so we should expect the same from technologies like AI, robotics, and synthetic biology. Given that nation-states are charged with managing and regulating the impact of technology in the best interests of their populations, how prepared are they for what’s to come?
Breaking point
The political order that fostered rising wealth, better living standards, growing education, science, and technology, and a world tending toward peace is now under immense strain, destabilized in part by the very forces it helped engender. They are often chronically mismanaged, bloated, and slow to act. Without a major shift in focus, many open democratic states face a steady decay of their institutional foundations, a withering of legitimacy and authority. The nation-state will be subject to massive centrifugal and centripetal forces, centralization and fragmentation. It’s a fast track to chaos, calling into question who makes decisions and how those decisions are executed, by whom, when, and where, pressurizing those delicate balances and accommodations toward the breaking point.
Constant brutal choices
That moment is almost here. Brought about by the inexorable rise of technology and the end of nations, this crisis will take the form of a huge, existential-level bind, a set of brutal choices and trade-offs representing the most important dilemma of the twenty-first century. Solving the question of AI alignment doesn’t mean doing so once; it means doing it every time a sufficiently powerful AI is built, wherever and whenever that happens.
Exponential
Exponential change is coming. It is inevitable. That fact needs to be addressed. On the one hand, they are in a strategic competition to accelerate the development of technologies like AI and synthetic biology. On the other hand, they’re desperate to regulate and manage these technologies—to contain them, not least for fear they will threaten the nation-state as the ultimate seat of power. Containment of the coming wave is not possible in our current world.
Manage the wave
The book ends with several suggestions to manage the coming waves. In my view, a naive wish list that includes an Apollo program for technical safety, being time, red teams, purpose, promising to behave, treaties, culture and people power. That is not going to do it. I suggest you read two more books:
- “Novacene“
- “Scary Smart”
- “The Fourth Age: Smart Robots, Conscious Computers, and the Future of Humanity“
I have decided to be an optimist. It is going to be awesome.
