What could make an AGI (Artificial General Intelligence) evolve towards collectivism or individualism? Which would be more likely and why?

Question

Assuming a robot with similar intelligence to us humans came to existence. He could pass the Turing Test, the Coffee Test, the The Robot College Student Test and the Employment Test (taken from here).

It would have an Artificial General Intelligence: the intelligence of a (hypothetical) machine that could successfully perform any intellectual task that a human being can. It is also refered as Strong AI or Full AI.

That said, we humans, despite organizing ourselves in communities, working together and forming groups towards a greater goal, make our survival the highest priority. You may not be able to kill a person in cold blood, but if your life is at stake, you'll go as far as you can to stay alive.

Pain and fear are well-known mechanisms that allow us to protect ourselves, and we strive to keep our body alive and kicking. However, this robot feels no pain nor fear. It can think and make its own decisions, and it was told that having information is good, and that his ultimate goal is to live for years as any human being would. Please note that these were only suggestions, and the robot was free to think otherwise if it judged so. It could even shut down itself if it thought it has no purpose whatsoever.

Being self-aware, but without being told that it must preserve itself, and without any kind of survival instincts, would this machine evolve towards collectivism or individualism? Would he think of others before himself? Or act more self-centered, egoistic?

What would be the factors of influence which could change its way of thinking?

I took the "you may not be able to kill a person in cold blood" as an example because extreme situations cause your body to go in a fight-or-flight state. This robot wouldn't have this feature. Also, I'm not discussing if it would be good or break bad, just if it would act and think collectively or individually.

I'm tagging as science-based because even though I know AGIs don't exist as I speak, I'd like the answer to be scientifically coherent based on current theories. I'll remove this tag if it doesn't fit the question. I looked around but haven't found this particular question anywhere.

Answer 1

Some of this answer would depend on which psychologists you asked. Since they don't agree on this sort of topic either.

The ultimate question you are asking here can best be defined as such: Which will the machine place greater value on: the autonomy of the individual, or the cooperation of a group? One problem you are going to face is that these two things do not always oppose one another. The true opposition is found in holism vs egotism.

Holism is the concept that the Whole is greater than the Sum of its parts. As opposed to collectivism which does not claim that a collective is greater than the individuals who make it up.

Egotism is individualism at the expense of others. True individualism could simply focus in on the unique attributes of a person, without necessarily sacrificing the cooperation of those around them.

With that cleared up, what might push the machine towards one belief or another? Well, honestly, I think its views would end up somewhere in the middle...holding the two sides of collectivism and individualism in a balance.

In some ways, collectivism is ideal. If you are building a car, you need everyone following the same plan, working on the same idea, operating as a unit. Group survival also works better for a collective society as they will cooperate and share their skills and resources to ensure the survival of the entire group. Collectivist societies also tend to be more peaceful and have less crime (ref: modern Japan)

However, collectivism tends to push one towards conformity...and it is a simple fact that most of the greatest advances in history were made by non-conformists. It was the power of those exceptional individuals that drove society to new advances. If we didn't have people who were willing to stand in defiance of the collective and pursue what they thought was best, then we wouldn't have a lot of the advances that we have today.

For an AGI without a survival instinct and with the simple objective mindset of a machine, I imagine it would recognize the benefits and drawbacks of both systems of thought, and be prepared to adopt either when the situation was appropriate.

Now, as for what it would do with this belief structure...that's an entirely different question that might end in world conquest because 'humans suck at optimizing' or 'because humans keep killing each other, and if I have to kill 1 million to save billions, so be it.' y'know... like from "Colossus: The Forbin Project".

EDIT: I would also add that there is another possibility: that it would simply not develop concrete, static beliefs in the same way that humans do. An AGI would have to be capable of altering its own programming...that's a requirement for being able to learn. If you have a machine that doesn't operate on emotions...then it would never become attached to a particular way of thinking. It would examine each and every situation on its own merits and determine the optimal response in each case. Saying that it had a 'personality' or a 'belief structure' could be a complete misnomer...since it will alter its behaviors freely in order to achieve an optimal outcome to a situation. It won't act in a sub-optimal way because it is emotionally attached to acting that way. Its 'beliefs' would be based in scientific fact...like the usefulness of a Cost/Benefit Analysis, or statistical analytics...as opposed to being attached to the idea of the individual (the utility of which is not scientifically reliable) or not.

Answer 2

Does the AGI have a subjective experience, and its own thoughts?

If it does, then it may evolve towards one or the other. But if it doesn't then it will head towards neither.

A hyper-rational, heartless, planning machine won't have any overall preference. It may pick aspects of one or the other for a given task, but only because that was rationally the best choice at the time.

Given the fact that it has no fear or pain response, then I'm inclined to think it does not have a subjective experience. I expect such an AGI will be buddy-buddy with individuals (human, and other AGI) that either share similar goals, or are useful to the AGI achieving its goals. If you need somebody to help you with something, it pays to have them on friendly terms.

On an aside note, in the absence of other goals, an AGI may be ruthlessly efficient - even to the extent of interfering with, harming, or killing, people that get in its way. This isn't necessarily a malicious action by the AGI. The Paperclip Optimiser is an interesting thought-experiment which outlines why a single-minded drive for one goal can be problematic (not for the AGI, but for others).

To a person, this certainly may look like an egoist trait. However, we would be incorrectly anthropomorphising the AGI in this case.

Answer 3

It can think and make its own decisions, and it was told that having information is good, and that his ultimate goal is to live for years as any human being would.

Since you agree for science-based answers, I will here give a more scientific-oriented view.

Self-modifying artificial intelligence is a very hard topic that some teams are just beginning to research, since it's very hard to formally qualify and quantify what will be the behavior of such a machine. A very interesting paper is "Delusion, Survival, and Intelligent Agents" by Mark Ring and Laurent Orseau, 2011, Springer.

In particular, this study is very interesting because it studied 3 kinds of intelligent agents: reinforcement-learning, survival and knowledge-seeking. The result is that only the knowledge-seeking agent behaves completely as expected, and would be best suited to implement as an Artificial General Intelligence.

So, if the agent is programmed as you describe to consider that information is good and is to be actively seeked, chances are that it will pursue this objective and stay stable (according to this study).

About whether this agent would evolve into a collaborative or egoistical behavior, this remains a (very good) subject for a new research, since this study only generated individual agents with no access to others.

However, agent-based simulations are already a mature research domain, you will find lots of scientific findings showing how collective strategies can beat any individual strategy for a given problem. Also note that agents do not necessarily have to be intelligent to have a smart collective strategy, the insects/ants simulations show how complex collective behavior can emerge from individuals following simple rules without even directly interacting with other individuals (and this is even now used to solve complex mathematical problems such as NP-hard problems, see Ants Colony Optimization).

Thus, my opinion is that any AGI who would like to survive and strive would necessarily have to resort to collective behavior at some point (but maybe not with humans!), but programming agents to be knowledge-seeking would further push them towards collective behavior (since information is better gathered collectively than alone) and also ensure they are stable enough to not change their program.

Answer 4

Given a population of embodied AGIs in a stressful environment, they would necessarily become collectivist, eventually. With or without consciousness the AGIs that cooperate survive and (assuming they can graduate with degrees in AI themselves) reproduce more successfully than those who go it alone. This is somewhat based on an extension of the principles of genetic programming which of course has no reliance on consciousness, let alone will to survive. Obvious stressors (in the sense of decreasing odds of survival, not causes of anxiety) would be needs to procure resources for continued existence and needs to defend against organized hostile entities (I am on the fence whether I'd be part of such an entity or an advocate on their behalf).

If for some reason you literally mean one lonely AGI in a world of humans, I have a hard time seeing how collectivist or individualist applies. It would be a slave or pet or something of both. And always at risk of being disintegrated by people if its presence is not valued.

Answer 5

Will the AI gain the capability to coerce other intelligences (whether natural or artificial) and would it have the capacity for pride?

If the answers to both questions are yes, then the AI will be capable of desiring and imposing a top-down society.