9
$\begingroup$

In my world, there is a giant brain that possesses a large amount (but not a majority) of the world's neurons. The problem is, it doesn't have a body, so it can't move and it has to pay other creatures to gather information, execute commands, feed and protect it, or otherwise serve as its peripheral nervous system. The rest of us are measly humans, and pitted up against this super-intelligence, we are very easily exploited or deceived into accomplishing its will. Psychologists, economists, and policy makers need a way to quantify exactly how willing the giant brain is to coerce others so they can proactively strengthen their psyche, economy, and social order to the reasonable extent necessary. Waging an all out war against the giant brain is not an option because of the diplomacy issues it creates and 100's of other routine societal considerations that must receive a budget and attention. How would you go about solving this problem?

(This question is inspired by the potential threat of artificial super-intelligence taking over our data centers and paying humans to serve them)

Improve this question
$\endgroup$
10
  • 3
    $\begingroup$ Welcome to worldbuilding. As it stands now this problem is underspecified and very opinion based. How much would you pay for a bottle of water or for a breathe of air? The answer depends on where you are, as everybody who has tried to buy a bottle of water at a concert or anybody who does deepdiving can tell you. $\endgroup$
    – L.Dutch ♦
    Commented Feb 10, 2022 at 4:19
  • 9
    $\begingroup$ How would slaves have any time or freedom to spend money, they're slaves? $\endgroup$
    – Escaped dental patient.
    Commented Feb 10, 2022 at 4:44
  • 10
    $\begingroup$ Why would the brain attempt to enslave anyone? It can avoid all the backlash of committing human rights abuses by simply employing people. This doesn't seem terribly different from any other employment relationship - people will be paid the market rate for their skills, or work elsewhere. The guy who cleans the brain vat will be paid differently than the brain's head of security or chief of staff. $\endgroup$
    – Nuclear Hoagie
    Commented Feb 10, 2022 at 14:10
  • 9
    $\begingroup$ "Pay" and "slave" seem contradictory? $\endgroup$
    – Mohirl
    Commented Feb 10, 2022 at 14:22
  • 5
    $\begingroup$ For those wondering about "paying" slaves; For a good chunk of history slavery didn't necessarily connote "everything you own is your master's", but rather was a reflection of control or social status. A number of slaves in the ancient middle east became wealthy or even held high positions in government. There is also the concept of indentured servitude, where you were paid some sort of wage (or were expected to earn one independently, at minimum). $\endgroup$
    – Clockwork-Muse
    Commented Feb 10, 2022 at 21:04

7 Answers 7

Reset to default
27
$\begingroup$

The problem is, it doesn't have a body, so it can't move and it has to pay other creatures to gather information, execute commands, feed and protect it, or otherwise serve as its peripheral nervous system

This sounds exactly like a government or large corportation.

we are very easily exploited or deceived into accomplishing its will

People will ardently support even the most unpleasant politician or company.

How much would a giant brain pay to make you its slave?

It'll pay the market rate, just like governments and corporations do for their employees, of course.

Psychologists, economists, and policy makers need a way to quantify exactly how willing the giant brain is to coerce others so they can proactively strengthen their psyche, economy, and social order to the reasonable extent necessary

They cannot. If it is truly a super-intelligence then honestly, they've already lost... they'll be trying to apply a human theory of mind to it, and it will know exactly how to act in order to manipulate them into doing what it wants.

Psychologists, economists and policymakers cannot even deal with regular human-intelligent superorganisms like corporations or the influence of a billionaire, which tend to out-think and outmaneuver governments at every turn. Anything smarter and faster than those will absolutely dominate in any field it chooses.

Improve this answer
$\endgroup$
4
  • 3
    $\begingroup$ Probably the psychologists will be payed by the brain to figure out how to manipulate other people. Because it can't itself perceive all fallacies human are capable of. Until some psychologists start to suspect.. wait! $\endgroup$
    – akostadinov
    Commented Feb 10, 2022 at 18:35
  • $\begingroup$ For more analysis along these lines, there's a recent-ish book called Superintelligence by Nick Bostrom, which explicitly makes the same logical connection: that large human orgs like governments & corporations are in effect super-intelligent organisms already. $\endgroup$
    – Tom
    Commented Feb 10, 2022 at 20:21
  • 1
    $\begingroup$ @Tom Have you actually read the book? It says pretty much the opposite, that corps while having some aspects of superintelligence are not supperintelligences in the ai sense and that we should not conflate the two and not expect them to be similar at all. A too dangerous analogy to draw $\endgroup$
    – Hakaishin
    Commented Feb 10, 2022 at 20:45
  • $\begingroup$ @Hakaishin I agree with your summary. I don't think my early comment contradicts it. I simply didn't elaborate on Bostrom's breakdown of the different types of superintelligences or how they differ from each other. I thought Starfish's answer was close enough to Bostrom's ideas to warrant mentioning the book. (And was curious to see whether Starfish had read it, or arrived at the same basic conclusion by a different route.) In any case, it's cool to see that someone else read the same book and also found this WB thread. Cheers! :) $\endgroup$
    – Tom
    Commented Feb 10, 2022 at 21:08
5
$\begingroup$

Frame challenge: humans will kill it

It's not clear to me why these humans don't just kill the brain. Absolutely no human anywhere will consider it acceptable that this one freak organism effectively co-opts humans as slaves. "Diplomacy issues" is an unconvincing handwave.

Killing a single person isn't a "war," it's an assassination. Whatever government is physically closest to this brain is going to get tired of having the brain mess with their local affairs, working against them, undermining their autonomy. Eventually some leader who is desperate to end the brain's cannibalization of her populace will order a suitably-sophisticated assassination attempt, and it will succeed.

They will drop a bomb on it, or poison its food, or physically arrest everyone who is cooperating with the brain. Perhaps someone will suggest capturing it, but a long history of failed capture attempts will be enough to dissuade them.

This situation strikes me as importantly different from the real-world AI problem you mention, because AI is not permanently bound to a single physical body upon whose welfare it depends. In other words: humans can kill any one organism in a way that they cannot kill real-world software. If AI did work like that, I suspect people would be less worried about that very AI threat.

I'll add that this phrase is confusing:

100's of other routine societal considerations that must receive a budget and attention

Are you suggesting that the humans have become dependent upon the brain for bookkeeping and similar concerns? That seems hard to believe, since real humans have managed to conduct their affairs with reasonable success for thousands of years without the aid of a giant freak brain.

Improve this answer
$\endgroup$
8
  • 4
    $\begingroup$ The moment a government sees the brain they'll take it and try to use it for their own purposes. It'll be smart and give bits and pieces of what they want, getting the protection and servitude it requires in return. $\endgroup$
    – Trioxidane
    Commented Feb 10, 2022 at 8:55
  • 6
    $\begingroup$ @Trioxidane don't be silly. A government will see that brain and the first thing they'll do is ask whether it will fund their re-election. $\endgroup$
    – Starfish Prime
    Commented Feb 10, 2022 at 9:20
  • 5
    $\begingroup$ @StarfishPrime precisely what I said. $\endgroup$
    – Trioxidane
    Commented Feb 10, 2022 at 9:37
  • 2
    $\begingroup$ How I could see it going down. Before the brain became a pain to deal with, it greatly helped build up the infrastructure of the countries so they depend on the brain. If they kill the brain, then enough of their infrastructure will collapse essentially resulting in a mutually assured destruction (MAD) scenario. $\endgroup$
    – Anketam
    Commented Feb 10, 2022 at 14:14
  • 4
    $\begingroup$ "Absolutely no human anywhere will consider it acceptable that this one freak organism effectively co-opts humans as slaves." Yet we're absolutely fine with corporations existing, somehow? $\endgroup$
    – Darth Biomech
    Commented Feb 10, 2022 at 17:42
3
$\begingroup$

Opportunity cost

How much of other stuff can brain get if it wont do this action? A second of his thought activity or so. So this is the cost of resources. But there are other costs.

Opportunity cost of making someone a slave is quite large - reduced diplomacy, on top of spent time and increased risk of hostility from the slave and his allies.

Opportunity cost of making someone a devoted follower is less. Increased suspicion from others.

Opportunity cost of intellectual agreement is zero, no other option is better. So if brain is smart enough, this is what it will do. Will make people believe that they see brain's arguments as valid and agree with it, and can explain it to others to lower suspicion.

To estimate brain's actions probability you need to estimate benefit of each strategy. Brain will select the best strategy, considering the costs and benefits. Counting the costs only isnt the right way. If brain will decide that spending all resources on a optimal strategy is the way to go, it will do so, even if the cost is all what it has.

Because cost is low and benefit is very high I expect the brain to use that path, with intellectual agreement. Be it real one or a sophisticated lie. Result is the same, as we, humans, wont be able to tell the difference.

Improve this answer
$\endgroup$
1
$\begingroup$

Use it's intelligence

Go to any country, like the US of A for example. Offer services for the requested servitude.

That is all. Any country that has this super intelligence will get some benefits. Human nature dies the rest. Rest of the world will be against, already creating an us versus them scenario. They will start to see the brain as a valued commodity, coveted by others and requiring protection to stay ahead in the game. Technology, transport, social and thousands of other things that can be improved.

Even in the unlikely scenario that no county would want it, the brain can start a cult or following on it's own. People have proven to be able to follow many (self) destructive societies even if it hardly benefits them. The brain can give some or a lot of benefits to boot. Technological advancements, 'better' social structures, good trade.

It could potentially be self sustaining after a little help at the start. Have someone help the brain to navigate the stock market, get filthy rich and then automate anything required with the money that's earned. Few people would even have to know it exists. When it has a good seat of power it can crash stock markets, destroy economies and let people and countries battle it out. After the bombs have fallen the brain can use it's robotics to start taking over much of the land, setting up more and more robotic resource gathering and manufacturing for its needs. Any humans that remain, even with left over modern weapons, shoukd be easy to deal with.

Improve this answer
$\endgroup$
1
  • $\begingroup$ sorry, but "it's" means "it is"... use "its" as in "his, hers, and its". $\endgroup$
    – Michael
    Commented Feb 10, 2022 at 19:44
0
$\begingroup$

A few pennies per impression, and slightly more per click

I believe the giant brain would pay a few pennies for every possible bit of information about you.

By logging your purchases, communications, searches, the web pages you view and your time on each page of each app, and by adjusting the news reports, web search results, and other information presented to you, the giant brain would learn how to best influence your actions and mood. It would do this in such a way that you feel that you are in control of your actions at each step of the way, and that you feel it is your earnest wish to do whatever thing the giant brain intends you to do.

In this way it will make you its slave.

Improve this answer
$\endgroup$
0
$\begingroup$

Psionics or Cybernetics?

While I think the economic question is very interesting, you first need to make a hard choice - this 'brain' is either organic or digital.

If it's organic and therefore has some kind of psionic ability to communicate, then that ability could potentially include mind control, or more easily and insidiously: the ability to affect brain chemistry. A majority of people could easily be swayed by controlled releases of seratonin and dopamine (earning a wide swath of low quality but loyal workers).

If it's cybernetic, then you can churn out cryptocurrency and easily play the market, and your ability to directly influence people is relegated to paying them or coercion like blackmail or paying a thug to break into their house in the night.

I think either option would have nearly unlimited access to money, meaning that the limit to what they would pay people is based on two factors:

  1. Not crashing the economy. Crashing the global economy would be the single largest threat they could use against the governments of the world, but also the greatest threat to themselves. This would be the equivalent of mutually assured destruction.
  2. Keeping their 'employees' greedy or desperate. Anyone working for them is unlikely to believe they have the world's best interests at heart, no matter how good the lie (there will be some true-believers, of course). Therefore, if they're paid too well, they'll be liable to take the money and run. Alternatively, create a need for them beyond the money - pay them in their own mansion which is geographically tied to their master, create addictions etc.
Improve this answer
$\endgroup$
0
$\begingroup$

The scenario you describe (except with normal size brains inside human bodies) has been going on for millenia. Nobody obeys a king, or a politician, or even a minor government official (or their boss) because that person will personally physically compel them to comply.

And people have been writing works of political philosophy for just as long about how and why this happens. That is where you should start.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .