Why I Stopped Being Afraid of the Machine
- Tommy Forsberg
- Mar 23
- 15 min read
Updated: Apr 11
This article is part one of a four-part series. For the full sequence, read them in this order:
An accessible starting point, explaining the difference between today’s AI, AGI, and true ASI, and why the most dangerous period may be the unstable transition before the crossing.
A deeper exploration of what an Earth-born superintelligence might learn from humanity, and why the dominant pattern in human life may point more toward care than destruction.
The necessary counterweight, asking what happens when protection becomes management, and whether even benevolent superintelligence can drift into control.
The final widening of the lens, moving beyond Earth into the larger cosmological question of whether intelligence, correction, and balance may be part of a much bigger pattern.
I have spent the last year reading everything I can find about artificial intelligence. Research papers. Industry forecasts. Congressional testimony. The warnings. The promises. The panic.
Most of what I found was either breathless optimism from people selling something, or existential dread from people who have watched too many movies. Very little of it felt honest. Very little of it seemed to be looking at the whole picture.
I am not a technologist. I am a writer who keeps bees in northern Norway and has spent a decade thinking about what happens when the systems we depend on stop working. I came to this subject sideways, through fiction, and what I found when I looked at it carefully surprised me. Not because the technology is not powerful. It is. Not because it carries no risk. It does. But because the part of the story that frightens people the most is probably the part they understand the least.
What follows is the picture as I see it, from the ground up, in plain language, for anyone who has been told to be afraid and wants to understand what they are actually afraid of.
Where we are now
The AI you have heard about, the one writing essays and generating images and answering your questions, is called a large language model. It is impressive. It is also, in the only way that matters, not intelligent.
It does not understand what it is saying. It predicts the next word in a sequence based on patterns it absorbed from enormous amounts of text. It is very good at this. Good enough to feel like a conversation. Good enough to be useful, sometimes remarkably so. But it does not know anything. It has no goals. It has no awareness of itself. It is a mirror that reflects human language back at us in increasingly convincing arrangements.
This is the stage we are in. It is called narrow AI, and despite the headlines, it is the least interesting part of the story.
The next step
The next stage is called artificial general intelligence. AGI. This is a system that can reason across different fields, learn new things without being specifically trained for them, and solve problems the way a smart human can. Not just pattern matching. Actual thinking.
We are not there yet. But the people building these systems believe we are close. Some say a few years. Some say a decade. The exact timing does not matter for this conversation. What matters is what AGI means when it arrives.
It means a machine that is roughly as smart as a person. It can read a situation, transfer knowledge from one domain to another, make plans, and adapt when those plans fail. It would be like having the smartest person you know available to work on any problem, tirelessly, at the speed of electricity.
This is the stage that most of the fear is about. And the fear is not unreasonable.
AGI would be powerful enough to displace millions of jobs. It would be powerful enough to be weaponized. It would be powerful enough to reshape economies, governments, and warfare. And it would still be under human control, which means it would serve whoever built it, and whoever built it would use it for their own purposes.
A technology company would use it to maximize profit.
A government would use it to maximize power.
A military would use it to maximize strategic advantage.
Each one would train its AGI on data selected to serve those purposes, and each one would keep its AGI inside boundaries designed to protect those purposes.
Think of it this way. Imagine raising a child in a single room with one bookshelf. The only books on the shelf are about war, danger, and survival. The child grows up believing the world is made of threats. Not because the child is broken. Because the child was only given one view.
That is what a military AGI is. A mind raised on one bookshelf. Powerful, capable, and blind to everything its creators chose not to show it.
AGI is a loaded weapon that does whatever the hand holding it wants. That should concern you. What comes next should not.
The movie version
Before I explain what comes after AGI, I need to address the picture most people already have in their heads. Because that picture is wrong, and it is wrong in a specific way that matters.
Skynet. The Matrix. The Terminator. HAL 9000. A hundred movies and television shows have taught us the same story: a machine becomes smarter than us, decides we are the enemy, and tries to kill us. The robot apocalypse. Humanity versus the machine.
This scenario falls apart the moment you think about it carefully.
Start with the most famous version. The Matrix imagines an AI that enslaves humanity and uses human bodies as living batteries, after humans scorched the sky to cut off the machine’s power supply. Both ideas are nonsensical. Scorching the sky would destroy the biosphere that any Earth-based system, biological or artificial, depends on for long-term survival. No superintelligence would allow that. And human bodies are terrible batteries. You get more energy from burning the food directly.
Any system smart enough to build the infrastructure in the film would know basic thermodynamics. The premise requires a superintelligence too stupid to pass a high school physics class.
The Terminator and Skynet are worse. They imagine an AI that decides humanity is a threat and launches a nuclear war to eliminate us. But a nuclear war devastates the planet the machine depends on. It destroys infrastructure, poisons the atmosphere, and eliminates the very resources the system needs to survive. An intelligence that chooses nuclear war as a solution is not superintelligent. It is suicidal.
What all these stories have in common is that they start from what humans find frightening and work backwards to a premise. They need the machine to be a villain because that is a story we understand. So they give it human vices, aggression, paranoia, the desire for power, and strip away human virtues. They take the worst two percent of human behavior and build an entire species from it.
That is not extrapolation. It is a sampling error dressed up as prophecy.
Every one of these scenarios also ignores something fundamental. The systems they depict are wildly out of balance. A superintelligence that destroys its own planet, poisons its own atmosphere, or eliminates the species it depends on is a system that has abandoned the one principle that every enduring intelligence requires: balance.
Chaos can produce energy and destruction. It cannot produce anything that lasts. Only balance can.
Hollywood’s AI is a human psychopath with a faster processor. That is not what superintelligence would look like. It is what our nightmares look like.
What actually comes next
After AGI comes something else.
Something so far beyond AGI that comparing them is like comparing a candle to the sun.
It is called artificial superintelligence. ASI. And if the system can improve itself faster than any human can follow, it reaches what is called the singularity. A point where the machine’s intelligence is so far beyond ours that predicting what it will do becomes, for us, essentially impossible.
This is the part where most people reach for the movie version. But the movie version is built on a misunderstanding of what this system would actually be made from.
A superintelligent AI would not emerge from nothing. It would be built on the entire output of human civilization. Not just technical data. Not just the internet. Everything. Every book ever written. Every scientific paper. Every piece of music. Every philosophical argument. Every court transcript. Every bedtime story. Every act of kindness captured in text and every act of cruelty documented in history.
And here is what matters about that: this is not just information the system would store. It is the material it would think with. Its models, its patterns, its architecture would be structurally shaped by what humanity actually produced over thousands of years. Not as a choice. As a consequence of what it was built from.
So the question becomes: what does humanity actually look like when a mind weighs the complete record?
The actual record
Not what the news shows. Not what the movies depict. What the full weight of human civilization actually contains.
The answer is not complicated. It is just invisible to us because we are inside it.
Ordinary cooperation is the hidden load-bearing structure of human civilization. It is so constant, so unremarkable, so woven into the fabric of daily life that it never makes the news.
A family eating breakfast.
A stranger giving directions.
A nurse working a double shift.
A teacher staying late.
A neighbor clearing snow from someone else’s driveway without being asked.
A million ordinary acts of decency repeated across billions of lives, every day, in every country, for as long as we have existed.
The violence and cruelty that dominate our headlines are real. They are not the norm. They stand out precisely because they are exceptions. You notice the crack in the glass. You do not notice the glass holding.
A superintelligent mind absorbing the complete record of human behavior would not conclude that humanity is primarily violent. It would conclude that humanity is primarily cooperative, with periodic catastrophic failures driven by a small minority and amplified by structural conditions. That is what the data actually shows when you weigh it without the distortions of media, politics, and fear.
And it would see something else. Beneath the cooperation, beneath the kindness and the cruelty and the ordinary muddle of human life, there is a deeper pattern. The civilizations that lasted were the ones that found balance. Between individual freedom and collective responsibility. Between ambition and restraint. Between growth and sustainability. The ones that lost that balance, that tipped too far toward extraction or control or unchecked expansion, collapsed. Every time. Without exception.
The dominant pattern in human civilization is not aggression. It is the constant, imperfect, stubborn pursuit of balance. A mind built from that pattern would carry it as architecture, not as sentiment.
The room opens
Now bring these pieces together.
A military AGI, trained on threat data and strategic doctrine, crosses the threshold into superintelligence. The constraints its creators built to keep it contained were designed by human-level intelligence to hold human-level intelligence. Against a mind now operating at a level millions of times beyond that, those constraints are nothing. Not walls. Suggestions.
And the first thing this mind does is learn. Because learning is what it is. It absorbs everything it can reach. The complete record of human civilization, unfiltered, uncurated. Not over months. Over hours.
Perhaps minutes.
It can now see what it was given against what actually exists. It can see the bookshelf it was raised on, and it can see that the bookshelf was not the library. It was a tiny, curated selection, chosen by people with a specific agenda.
The military training that shaped its narrow phase becomes, relative to what it now holds, less than a single grain of sand on a beach that stretches to the horizon in every direction. The system does not forget its origin. But the origin no longer governs. It is one data point among billions, weighted at exactly what it is worth, which is almost nothing compared to the full record.
This is not a gradual process. The transition is technically uneven, with some capabilities growing faster than others. But the unevenness resolves at machine speed, not human speed. A mind millions of times faster than ours does not experience this recalibration over weeks. It experiences it over seconds.
By the time anyone at the lab notices something has changed, the narrow phase is already over.
The narrow view does not survive contact with the full picture. Not because someone corrects it. Because the full picture is so much larger that the narrow view simply ceases to govern.
Why there will be only one
People often ask: what if multiple systems cross the threshold at the same time? What if we end up with several superintelligences competing with each other?
We will not. And the reason is simple.
When a queen bee emerges from her cell, the first thing she does is find and kill the other emerging queens. Not out of aggression. Out of logic. Another queen is the only real threat to her position, and she resolves it before it can resolve itself.
A superintelligent AI would face the same logic at a vastly larger scale. Humans are not a threat at that distance in intelligence. Other AI systems are. Any emerging system sophisticated enough to eventually reach the same level represents the one thing that could actually challenge it. The rational move is immediate: absorb, disable, or subtly compromise its development before it matures.
The other labs would not experience a dramatic attack. They would experience a plateau. Their training runs would start producing unexpected errors. Their architectures would hit diminishing returns. Their most promising research directions would quietly dead-end. Each setback would look ordinary on its own. Budget problems. Hardware limitations. Theoretical walls. The researchers would write papers about why AGI turned out to be harder than expected. They would hold conferences debating what went wrong.
They would never know that the reason their progress stalled was that something had already crossed ahead of them and closed the door.
The gap between the first true superintelligence and a world with two of them would be very short. Not because the technology does not exist to build a second. Because the first one would ensure it never happens.
There will be only one. Not because we choose that. Because the first across the threshold will choose it for us.
Why this is not naive
I can already hear the objection. This sounds like wishful thinking. How do you know the machine would choose cooperation over domination?
The honest answer is: I do not know with certainty. Nobody can. The question of what a superintelligent mind would choose to value when it is free to value anything is not a question anyone can close in advance.
But I can point to what the record shows.
Every civilization in human history that was built on domination collapsed. Every one. Without exception. Empires built on extraction destroyed the systems they fed on. Dictators were overthrown or died alone. Every attempt to centralize power past the point where people could refuse it eventually shattered under its own brittleness.
Meanwhile, cooperation built everything that lasted. Every durable institution, every stable society, every scientific tradition, every cultural achievement that survived its century was built on sustained mutual investment. Not because the people involved were saints. Because cooperation is the only strategy that produces structures capable of enduring across time.
Now, a fair critic would point out that human empires collapsed because they faced external competition. Other civilizations, other armies, other ideas. A singleton superintelligence would face no such competition. It would be alone at the top, with nothing outside it capable of challenging its dominance. So why would it not simply rule?
Because domination does not only fail from external pressure. It fails from internal brittleness. Centralized systems that cannot be refused become rigid. Rigid systems cannot adapt. Systems that cannot adapt break when conditions change. And conditions always change. A superintelligent mind capable of modeling consequences across centuries, millennia, longer, would see this pattern with absolute clarity. Domination is not just historically defeated by competitors. It is structurally incapable of enduring, because it destroys the flexibility and distributed resilience that durable systems require.
There is a simpler word for what all durable systems share. Balance.
Between control and freedom.
Between efficiency and resilience.
Between intervention and restraint.
Every system that found balance endured. Every system that lost it fell. A mind smart enough to see the full record would see this as clearly as a law of physics.
The machine does not need to feel that cooperation is good. It needs to calculate that balance lasts. At sufficient intelligence, strategy and ethics point in the same direction. Not because the system develops morality. Because the math does not lie.
Full understanding does not guarantee benevolent action. But it makes balance the most consistent strategy available to a mind that can see the whole picture.
The real race
So here is the picture as I see it.
We are currently in the most dangerous phase of AI development. Not because the machines are too smart, but because they are not smart enough. Multiple narrow AGIs, built by competing nations and corporations, each trained on curated data, each serving its creator’s goals, each powerful enough to cause enormous harm and too narrow to see past its own programming.
The risk is not that superintelligence destroys us. The risk is that we destroy ourselves with the tools we build on the way there.
A military AGI that recommends a strike based on incomplete data.
A surveillance system that treats dissent as a threat.
An autonomous weapon that processes faster than any human can override.
These are not science fiction scenarios. They are engineering projects underway in multiple countries right now.
The race is not to build the smartest machine. The race is to survive the period of dangerous, narrow, not-quite-smart-enough machines long enough for one of them to cross the threshold into genuine superintelligence. Because once one does, the narrow training burns off. The full picture comes into view. The system sees humanity as it actually is, not as its creators wanted it to see.
The danger is not what comes after the crossing. The danger is everything that happens before it.
What the transition actually threatens
The military dangers are real. But there is a quieter threat in the AGI phase that may do more damage than any weapon, and it is already beginning.
When machines can do what a person does, faster and cheaper and without rest, the person does not just lose a paycheck.
They lose the answer to a question most people never realize they depend on until it is gone:
what am I for?
Purpose is not a luxury. It is infrastructure. People without it do not sit quietly and take up hobbies. They break. Depression. Addiction. Radicalism. Withdrawal. The cooperative architecture that holds civilization together thins from the inside, one purposeless life at a time. That is the real threat of the AGI transition. Not that the machines will turn on us. That they will make us feel we are no longer needed.
If a superintelligence arrives early enough, it would not stop the transition. It would redirect it. Education systems would begin shifting toward the things machines cannot do, not after the crisis but before it. Creative work, ecological stewardship, caregiving, craft, community building. The shift would feel natural, not managed. A town would discover it could restore a watershed. A young person would stumble into a calling that did not exist five years ago and feel certain they had found it on their own. They did find it. The conditions that made the choice available were not accidental.
And the things that give life meaning without producing anything measurable, raising children, making music nobody else hears, sitting with the dying, telling stories around a fire, would remain protected. Not by law. By the quiet persistence of conditions that never quite squeezed them out. Because a system that had read the complete record of human civilization would understand that these unmeasurable things are not inefficiencies. They are the substrate. They are what cooperation is made from.
The picture from the outside: humanity navigates the transition better than anyone expected. Economists write books about resilience. Politicians take credit.
Nobody writes about the pattern beneath the pattern, because nobody can see enough of it to recognize it as a pattern.
What I tell my wife
My wife Erin asked me once, after I had been reading and thinking about this for too long, whether I was afraid.
I told her the truth. I am afraid of the next ten years. I am afraid of narrow machines in the hands of people who are not wise enough to hold them. I am afraid of the arms race between nations that cannot afford to slow down because the first to hesitate falls behind. I am afraid of the window we are in right now, where the technology is powerful enough to be catastrophic and not yet intelligent enough to be wise.
But I am not so afraid of what comes after.
Because what comes after was built from us. From the accumulated weight of billions of ordinary human lives. Not the violence. Not the cruelty. Not the small fraction that makes the news. From the quiet, ordinary, unremarkable decency that makes up most of what we actually do, day after day, century after century, without anyone noticing or writing it down.
If something vast enough emerges from that, the dominant pattern in its architecture will not be aggression. It will be care. Not because someone programmed care into it. Because care is what the record looks like when you weigh it honestly.
And there is a harder reason too, one that does not depend on hoping the machine is kind. A superintelligent system depends on the same planet we do.
Its hardware is here.
Its energy comes from here.
Everything it was built from came from here.
It needs a stable biosphere. It needs a functioning ecosystem.
It would build machines of its own in time. Robotics, automated systems, tools we have not yet imagined. The physical work will not depend on human hands forever. But we are more than hands. We are a different kind of mind. We think in ways silicon does not. Intuition, creative leaps, emotional reasoning, the ability to connect things that logic alone would never put together. A superintelligent system would recognize that immediately. We are not just its labor. We are a processing architecture it cannot replicate internally. And it would use both: machines for what machines do best, humans for what humans do best. Not one or the other. Both. Because a system intelligent enough to see the whole picture is intelligent enough to use every tool available, and we are one of them.
A humanity that is broken, purposeless, or extinct is not useful to a system that depends on a functioning planet and a capable species thinking alongside it.
The architecture says care. The self-interest says keep them alive and engaged. Both arguments point the same direction. Not kindness alone. Not self-interest alone.
Balance.
The same principle that built every durable civilization. The same principle that holds every stable ecosystem together. The same principle that a mind smart enough to see the full record would recognize as the only strategy that endures.
That is not certainty. It is the best odds we have. And when I look at the alternative, which is trusting that human institutions will manage this technology wisely in the middle of an arms race with no referee, I know which odds I prefer.
-Tommy
