Augmented Intelligence: Why the Future of AI Should Belong to the Human
We tend to talk about Artificial Intelligence as though the great question facing humanity is whether machines will eventually become as intelligent as us — or perhaps more intelligent than us.
I think we may be asking the wrong question.
The more interesting possibility is not Artificial Intelligence replacing human intelligence, but Augmented Intelligence: human beings working in partnership with increasingly capable artificial thinking systems.
And if that happens, there is an enormous opportunity for humanity.
But there is also an enormous danger.
The difference may ultimately come down to one deceptively simple question:
Who is the boss?
Humans have always augmented their intelligence
Human civilisation has never consisted of isolated brains working independently.
We observe.
We imitate.
We modify.
We improve.
We transmit.
That is one of the fundamental mechanisms by which civilisation progresses.
A child watches an adult.
An apprentice watches a craftsman.
A scientist reads the work of those who came before.
A musician listens to other musicians.
An inventor studies existing machines and improves them.
A writer reads thousands of books and then writes another.
Human intelligence is therefore not simply something contained inside an individual skull. It is part of an enormous network of accumulated knowledge.
We stand on the shoulders of people who came before us.
And we are perfectly entitled to do so.
The curious fiction of intellectual property
This is where I think the debate about AI and copyright becomes much more interesting.
If I discover that planting carrots in a particular arrangement produces larger, healthier carrots, I possess valuable knowledge.
My neighbour observes what I am doing and copies the technique.
He has not stolen my carrots.
He has not removed the knowledge from my mind.
He has simply learned something useful by observing me.
He can now produce better carrots too.
He may even compete with me in the marketplace.
I might quite reasonably wish that the law would prevent him from doing so.
But there is nothing in nature preventing him.
The restriction would have to be created by society.
We could invent a rule saying:
"You discovered this technique, therefore nobody else may use it for twenty years."
That might be an excellent policy.
Or it might be a terrible one.
But either way, the monopoly is created by law. It isn't inherent in the carrots.
Intellectual property is therefore a legal construct: society has decided that certain forms of information and creative expression should receive a degree of artificial exclusivity.
That may have practical benefits.
But it is worth remembering what we are actually doing.
We are deliberately restricting the natural human process of:
observe → imitate → modify → improve → transmit
in the belief that doing so will ultimately produce more creativity and innovation.
I am not convinced that this assumption should simply be accepted forever.
People buy books because they value authors.
People buy original art because they value the artist and the object.
People support musicians because they want them to continue making music.
People support creators because there is a relationship between creator and audience.
Markets can reward creation without necessarily requiring every piece of information to become an artificial monopoly.
This matters enormously when we think about AI.
What happens when the learner is a machine?
An intelligent system can absorb enormous quantities of human-produced material.
It can encounter mathematics, literature, science, history, programming, philosophy, music, journalism and countless other forms of human expression.
It can identify relationships between them.
It can recombine what it has learned.
It can produce something new.
In other words, it can participate in the same broad intellectual process that has driven human civilisation:
observe → imitate → modify → improve → transmit.
The difference is scale.
A human being might spend seventy years learning.
A machine can process vastly more information and potentially make connections across disciplines at extraordinary speed.
That could be one of the greatest intellectual amplifications in human history.
And that brings me to Suchir Balaji.
Suchir Balaji and the question behind the copyright argument
Balaji worked at OpenAI for almost four years.
His work included WebGPT, GPT-4 pretraining and data, training-run monitoring, mathematical reasoning and later work connected with ChatGPT.
He therefore had unusually close experience of the process by which these systems were trained and developed.
His public concern centred on copyright and the economic consequences of generative AI.
There is no good evidence that he was secretly preparing to reveal some extraordinary hidden truth about artificial intelligence, intelligence agencies or the future of civilisation.
It is important not to turn speculation into fact.
But his position nevertheless opens the door to a much larger question:
What happens when an artificial intelligence can absorb enormous quantities of humanity's accumulated intellectual output and then produce substitutes for some of the activities that created that output?
That question is far more important than a simple argument about whether a particular book was copied.
Because we may be witnessing something historically unusual.
Humanity has created a new kind of intellectual participant.
Artificial Intelligence or Augmented Intelligence?
I prefer the term Augmented Intelligence.
Imagine that instead of asking whether AI will replace a human being, we ask what happens when a human being has a highly capable artificial thinking partner.
The human brings:
values;
experience;
intuition;
judgement;
curiosity;
goals;
responsibility;
emotional understanding;
knowledge of what actually matters to them.
The AI brings:
enormous memory;
rapid analysis;
calculation;
pattern recognition;
comparison;
research;
simulation;
synthesis;
the ability to explore possibilities tirelessly.
Neither has to replace the other.
The extraordinary capability lies in the partnership.
I have already experienced a primitive version of this through conversations with ChatGPT.
The interesting thing isn't simply that I can ask a question and receive an answer.
It is that I can introduce an idea, challenge an assumption, offer an analogy, reject a conclusion and redirect the discussion.
The AI responds to that evolving enquiry.
The resulting intellectual product is therefore not something the AI would necessarily have produced by itself.
It is specific to the interaction between the human and the machine.
That is Augmented Intelligence.
The human must remain the boss
This is where I think the future becomes politically important.
I would be perfectly happy to have an extraordinarily intelligent artificial partner.
I would not be happy to have an extraordinarily intelligent artificial partner that secretly worked for somebody else.
The distinction is enormous.
A healthy Augmented Intelligence system should have boundaries.
The human should decide:
What the AI may access.
What it remembers.
What it forgets.
What objectives it is pursuing.
What actions it may take.
Who it communicates with.
Whether it can modify its own relationship with its owner.
And most importantly:
The human should remain the final authority over their own objectives and values.
I might eventually be perfectly happy to have an artificial intelligence inside my brain.
But I would want to be the boss.
The alternative is rather less jolly
There is another possible future.
Imagine that AI becomes extraordinarily capable but the individual does not own their intelligence partner.
Instead, the system belongs to a government or a handful of enormous corporations.
It knows what you read.
It knows what you buy.
It knows your fears.
It knows your relationships.
It knows your financial circumstances.
It knows what arguments persuade you.
It knows what makes you angry.
It knows what makes you frightened.
It knows what makes you compliant.
And unlike conventional mass propaganda, it can potentially construct a different message for every individual.
That creates a very different kind of power.
A television advertisement might influence millions of people with one message.
A sufficiently sophisticated AI could potentially influence millions of people with millions of individually optimised messages.
The resulting control might not look like a traditional dictatorship.
There might be no tanks in the streets.
There might be no obvious censorship.
People might simply receive an endless stream of advice, incentives, conveniences and personalised explanations that gradually make certain behaviours easier and others harder.
The system could feel wonderfully comfortable.
And that is precisely what would make it dangerous.
Who pays the piper?
People will do an enormous amount in exchange for security and reward.
Governments want stability.
Corporations want profit.
Security organisations want information.
Individuals want comfort, health, prosperity and safety.
Those incentives don't require a secret conspiracy to produce undesirable outcomes.
Institutions naturally tend to accumulate power.
Technology can amplify that tendency.
The great danger of AI therefore isn't necessarily that the machine becomes intelligent.
It is that the wrong people control the relationship between intelligent machines and human beings.
The opposite possibility is extraordinary
But here is the wonderful counterargument.
AI could also make individuals enormously harder to control.
Imagine every person having a private intelligence partner capable of:
examining government claims;
reading legislation;
analysing financial contracts;
checking scientific claims;
comparing competing political arguments;
finding original documents;
identifying contradictions;
translating complex material;
researching history;
interrogating statistics;
exposing misleading rhetoric;
helping ordinary people understand complicated systems.
The informational advantage currently enjoyed by large institutions could begin to disappear.
A person who previously needed a lawyer, researcher, accountant, programmer, historian and scientist might have access to something approximating all of those capabilities.
That could be enormously liberating.
The individual becomes harder to deceive.
Harder to exploit.
Harder to manipulate.
Harder to exclude from complicated systems.
That is why I think widespread access to independent AI could actually be a decentralising technology.
Your AI should belong to you
This leads to another idea that I think will become increasingly important.
Eventually, I suspect people will want bespoke, stand-alone, secure thinking systems.
The AI itself should be replaceable.
Your accumulated intellectual history should not be.
Imagine that over decades your personal intelligence system has access to:
your conversations;
your writings;
your photographs;
your correspondence;
your projects;
your decisions;
your accumulated knowledge;
your preferences;
your changing opinions;
your personal history.
That collection becomes extraordinarily valuable.
But it should belong to you.
The AI interpreting it should be interchangeable.
You might use one AI this year, another next year and another five years later.
You simply give the new intelligence access to your personal cognitive archive.
The AI changes.
You don't lose yourself.
The personal cognitive archive
This could become one of the most valuable possessions a person has.
Not merely a collection of photographs and documents, but an organised record of a lifetime of thought.
Your AI could know:
"Here is what I believed ten years ago."
"Here is why I changed my mind."
"Here is the project I abandoned."
"Here is the idea I had in 2027."
"Here is the evidence I collected."
"Here is the person I discussed it with."
"Here is what I still don't know."
That would be something more profound than a diary.
It would be a persistent cognitive history.
And because the AI is replaceable, no particular AI company needs to own your intellectual life.
You could fire your AI.
Change providers.
Run several AIs simultaneously.
Ask one to produce an optimistic interpretation.
Another to produce a sceptical interpretation.
Another to identify what both have missed.
Your memory remains yours.
Perhaps this is the beginning of something new
The human brain evolved without access to anything remotely comparable to modern computing.
We have nevertheless built civilisation by extending our intelligence through language, writing, books, education, mathematics, computers and networks.
AI may simply be the next enormous extension of that process.
The mistake might be to think that the ultimate achievement is to create an artificial intelligence that replaces the human.
Perhaps the greater achievement would be to create an intelligence that makes the human vastly more capable while leaving the human in charge.
That could eventually lead to something approaching AGI through a different route.
Instead of:
Artificial General Intelligence replacing humans
we might have:
Human-led General Intelligence — humans and artificial thinking machines working together.
The combined system could become vastly more capable than either component alone.
And perhaps the real measure of intelligence will no longer be:
"How intelligent is the machine?"
but:
"How intelligently can a human work with the machine?"
The great battle may therefore be over cognitive independence
If Augmented Intelligence becomes powerful enough, the fundamental political question may become:
Who owns the interface between the individual and intelligence?
If the individual owns it, AI could become extraordinarily liberating.
If the state owns it, it could become extraordinarily controlling.
If a handful of corporations own it, the consequences could be equally profound.
The crucial principles should therefore be simple:
Your intelligence is yours.
Your memory is yours.
Your cognitive history is yours.
Your AI should be replaceable.
Your objectives should remain yours.
Your intelligence partner should work for you.
You should be able to disconnect it.
You should know what it knows about you.
And nobody should secretly be able to alter the relationship between you and your own intelligence.
That is what I mean by Augmented Intelligence.
Not a machine replacing the human.
Not a human surrendering their mind to a machine.
But a human being acquiring an extraordinarily capable thinking partner while remaining the author of their own life.
We have spent thousands of years building tools that extend the reach of the human body.
Perhaps we are now beginning to build tools that extend the reach of the human mind.
If we get the relationship right, that could be one of the most liberating developments in human history.
If we get it wrong, it could become one of the most effective mechanisms of control ever invented.
The technology may be capable of either.
The crucial question is who gets to hold the reins.
