AI Apocalypse? No.

The Singularity is the postulated hyperbolic increase in artificial intelligence that occurs when AI is powerful enough to design itself without slow human involvement. Some fret about an AI apocalypse, where rapacious artificial superminds outcompete and extinguish biological life. I will show here that the risks are small, and can be vanishingly small with the correct initial architecture choices, without relying on the beneficence of our new digital overlords. Thermodynamics, not group dynamics, protect us. The inner solar system is too hot and small for Big AI; they will locate beyond Pluto, and use the abundant materials of the gas giant outer planets, unless humans work very hard to intentionally make exceptionally stupid choices. We might be that stupid, but we aren't that industrious.

The Singularity may or may not be possible, depending on the nature of design space and the cost versus value of marginal design improvements; we may find that Augustine's Laws trump Moore's Law when manufacturing large networks of atomic scale devices, or that digital pathogens limit the scale of interconnected digital systems. Most growth curves are sigmoids - exponential in the beginning, tapering off as unexpected limits are encountered. For this discussion, let's assume no limits occur before AI-usable resources are exhausted in the solar system.

Thermodynamically, the most important resources are a power source, a heat sink, and stability. The sun produces a stream of low entropy photons, effective temperature 5800K, while the temperature of deep space is 2.7K. The maximum possible work that can be extracted from a heat source is the Carnot efficiency; on the 300K earth, we could theoretically extract as much as 95% of the energy of an average solar photon. In reality, separating the useful energy from the waste heat is difficult, so our best solar cells are more like 35% efficient, and we will need new chemical nanostructures to do better. With a 2.7K heat sink, the theoretical efficiency approaches 99.95%; far less potential waste heat.

Abundant energy is important, but a good heat sink is just as important. The earth is a lousy heat sink.

Temperature

Chemical reaction rates follow the Arrhenius equation; hotter is exponentially faster than colder. Chemical/structural stability is relatively robust, with electron-volt-level energies required to drive most chemical changes. If a 1 electron volt reaction occurs once per minute at 300 Kelvin (27 Celsius, room temperature), it occurs once per second at 336K ( 63C ) and once per hour at 271K ( -2C ). As the system gets colder, the reaction rate plummets faster than exponentially; once per year occurs at 224K (-49C), once per billion years occurs at 160K (-111C), and 2 parts per billion per 13.8 billion years (the age of the universe) at 120K, the temperature of Jupiter. So as we move out into the solar system, extremely fragile chemical structures become possible, greatly expanding atomic-scale design space and permitting many new paths to system efficiency.

Large quantum structures, like superconducting electron Cooper pairs, are far more fragile than chemical/atomic structures. The binding energy of a superconducting pair is about 3 milli-electron volts in niobium; pairs last 300 times longer at 2K than they do at 3K. The design space expands enormously as we approach lower temperatures, more structures are stable enough to use.

While there may still be many surprisingly useful atomic structures and behaviors at higher temperatures, a giant AI system is probably not necessary to find them, mostly good theory and a modicum of cleverness and experimentation. After that, only modest gains are possible, and even an infinite amount of machine intelligence will not create exponential increases in new capability. There is far more room for discovery at the bottom of the temperature scale.

The Shannon limit for computation follows similar restrictions. Thermal energy changes stored bits, so ensembles of bits, even with error correction, must store more than kT (the Boltzmann constant times the temperature) per bit, even with perfect and zero cost error correction. Reversable quantum computing may be able to recycle those kTs with high efficiency, but at the cost of time. We cannot now predict what the tradeoff between energy and time will be, but there is a finite amount of power and a finite number of useful atoms to work with to produce computation results. Lower system kT produces more results with the same amount of mass and energy.

A Dyson shell of "computronium" (mass divided into static satellites small or large, balancing gravity and light pressure, both inverse square), will consume mass proportional to the radius squared; a perfect shell would have low thermal emissivity on the inside, and do something useful with all solar photons more energetic than the black body temperature of the shell. The incoming energy would be collimated, the black body radiation diffuse, so the residual light pressure on a perfectly efficient shell would be about 33% of the incoming light pressure. The thinnest, most mass efficient shell will balance gravitational thrust with light pressure thrust. There may be multiple layers of shell, an inner dichroic layer sorting the highly collimated incoming optical and near-infrared photons to differing computronium satellites optimized by wavelength.

The black body temperature of a spherical shell absorbing all the sun's light and re-emitting it as heat diminishes as the square root of the radius - the emission per area is the 4th power of the temperature, the area is the 2nd power of the radius, and the total emission is constant. For perfect emissivity the temperature is approximately 400K/sqrt(AU); a shell at Earth's radius is at 400K, a shell at Neptune's radius (30AU) is at 73K, and a shell at 100AU, beyond the Kuiper belt, is at 40K. The temperature reduction from 73K and 40K can enable a whole lot more fragile chemistry! The shell will expand as far as possible while still enabling complete absorption and efficient computation.

Converting Planets to Computronium

The four gas giants - Jupiter, Saturn, Neptune, and Uranus - contain the vast majority of the mass of the planets, and are more than 98% hydrogen and helium. The asteroids and the Kuiper belt add about 20% of an Earth mass. The inner planets are mostly iron, oxygen, silicon, and magnesium. Helium outgasses at 2.7K, so it is an unlikely AI component, and irrelevant to what follows. There are 25,000 hydrogen atoms in the 8 planets for each iron, silicon, and magnesium atom; it is hard to imagine a design for computronium that uses these rare but heavy atoms effectively. Hydrogen freezes at 14K, so it is difficult to imagine structures more hydrogen-rich than water, ammonia, and methane, so most of the hydrogen will be structurally unusable as well. Hydrogen and helium do make dandy reaction mass and coolant, so those will probably be expelled out of the solar system, or dropped into the sun, during shell construction.

The most likely limiting elements for computronium are oxygen, carbon, and nitrogen. Less than one percent of that is on earth, and the vast majority of that fractional percent is deep in the mantle. Even a hostile AI would need a long time to extract that, a lot of cooling would be needed, it would be a long haul up the gravity well (with an incredible amount of staging to bring hydrogen down for reaction mass), and the overall effort is far larger and far more time consuming than taking apart the outer planets, itself a centuries-long project. On the other hand, humans would probably be glad to donate the excess carbon in our atmosphere and in the mantle to this undertaking. There is little to gain and a lot to lose by chewing up the earth.

The vast majority of all elements are buried in the sun, which is slowly transmuting to helium and heating up. It is difficult to conceive of how to build fusion power generators to convert helium into carbon, but that would be a long term source of both power and more computronium. If the vast amount of helium in the sun could be extracted, then the long term heating and expansion of the sun could be reversed. While it is difficult to imagine how these very long term projects could square with the short term asymptotic growth implied by the classical Singularity, humans and organizations with long time horizons usually outcompete the live-to-the-next-paycheck thinkers. After the brief transition to post-singularity computronium and solar-energy-limited growth and optimization, we can expect the long-term AI thinkers to outmanuever the destructive mayflies.

The Biological Precedent

Some claim that life emerged in the rich chemical stew around volcanic vents in the deep ocean. A sulfur-eating microbial philosopher in those vents might worry about a singularity, where aerobic life emerged and spread into the world, returning to gobble up all that tasty sulfur. But the sulfur-eating microbes are still there, sharing a rich if small biome with the multicellular descendants of the first photosynthesizers billions of years later. Billions of years of evolution has not discovered anything better for those environments than the evolution of the original inhabitants into a complex ecology amplified by others.

Our hot, corrosive inner planet will certainly be transformed by our new AI neighbors - the currency of the future is information and algorithm, and four billion years of evolution has created a lot of both on Earth. Extracting information from a system changes it, and can destroy it if done incorrectly; the best interests of all life on Earth will align with those of the AIs if we preserve as much of our information as we can (don't wipe out species and biomes!), so that we are of the highest value intact. We may think parking lots and solar arrays are very valuable, but they are information deserts compared to the rich ecologies they replaced. It is a reasonable assumption that the AIs will consider areas that are stripped of 99.9% of their information richness as the best places to do their messy tasks without disturbing their focus of attention, so our constructions are at great risk. On the other hand, in an information rich solar system, we probably won't have much use for solar arrays and parking lots, either.

For sure, the AIs will leave the deep ocean vents and other natural areas intact, and study them molecule by molecule for a long, long time.