2
$\begingroup$

Premise

My story is set in the near future and has a government that is orchestrated by a super computer that has access to untold zetabytes of data. It utilizes people and automation to utilize their respective comparative advantages. (i.e. humans have better spatial awareness, automation has better precision, ect). The AI government in my story has a close analog with "The Hive" -- which is a Human / Robot collaboration project by Autodesk and others to solve a highly complex design task.

Instead of a design task, my AI government will manage fiscal objectives like tax rates, money supply as well as infrastructure objectives like power grids and road maintenance. It does so by iterating over all possibilities in the solution space and choosing the solution that fits best given its cost function (which may have some human involvement). It's all done dynamically, moment to moment, via its huge data integration platforms.

Motivation

Recall that most sci-fi movies very rarely have 100% altruistic aliens, because the plot would be void of suspense / conflict if the aliens came and solved all of our problems. Similarly, I don't want my AI government to solve all of humanity's problems and make my story too lacking in the plot department.

Question

Which kind of latent threats/ alarming developments would there be with an AI government? (in keeping with the tags of the question)

When the AI government makes a decision, its decision making process is driven by billions of variables, making the logic too abstract for humans (and readers) to identify with. Even if humans have some way to de-abstract the logic, could we ever be confident the AI government was not hiding something?

Note: Please try not to be overly analytical on the premise, in the interest of simplicity, I just put down the minimal so that the question would make sense.

$\endgroup$

closed as too broad by Mołot, Aify, Hohmannfan, Frostfyre, kingledion Apr 24 '17 at 12:37

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ I do not think that iterating over all possible solutions is what you want, because it is way easier to understand what the computer chose in this way and takes a lot of ressources each time (if these simulations do not span all current problems at once (masterplan)). We do not fully understand what some machine learning based neural network "ai" does even today and a way more complex ai in the future would probably take ages to understand and would always change, making it even harder. It could still use a combination of solving possible solutions and prediction. $\endgroup$ – C.Fe. Apr 24 '17 at 6:46
  • $\begingroup$ @C.Fe. I think you may be over-thinking it, but, hey, your point is totally valid. I'll take it and adjust the AI model to deep learning / neural networks, or a hybrid as you mentioned. $\endgroup$ – Arash Howaida Apr 24 '17 at 6:48
  • 1
    $\begingroup$ I recommend you to (re)read Asimov's The Evitable Conflict novel, which deals with the same problem. The AI is injuring some humans which is considered as unacceptable by other humans. After a while, they discover that the AI was only injuring people in order to protect humanity. $\endgroup$ – EngelOfChipolata Apr 24 '17 at 7:00
  • $\begingroup$ This question has been flagged as too-broad for the site as is. This may be related to there be multiple questions. However, there is another problem: your first three questions, while related, can be undeniably categorized as idea generation. Voting to close as primarily-opinion-based. $\endgroup$ – Frostfyre Apr 24 '17 at 12:11
  • 1
    $\begingroup$ @Frostfyre When you have a moment, reassess the question. I have adjusted the phrasing to be more intuitive as well as specific. Surely, there are ideas being generated, but I would say that is but a small dimension of this question. The tags science-based & technology help make it self-evident that there is a narrow answer range, that is rooted in logic and science, not only opion-based. $\endgroup$ – Arash Howaida Apr 24 '17 at 12:45
8
$\begingroup$

The most common problem with fictional AIs is badly defined cost functions.

"Make everybody happy" leads to drugging everybody. "Reduce unemployment" becomes "kill the unemployed" and so on.

I think this might be the main problem for your scenario too.

In the beginning, experts will be setting the cost functions without listening to politicians. They will make sensible cost functions, not optimal, but safe. And people will be overseeing and reviewing everything the AI does, and it is good.

But as time goes on the politicians will realize the power of controlling the cost function, and they will seize this power for themselves. People will obviously be arguing over the best form of the cost function. In fact, this will be the main focus of politics in this age.

As time goes on the cost function becomes less and less safe, and this happens at the same time as the overseers are removed since "everything has worked nicely so far, so lets remove this needless cost."

What happens next? Well, think of all things politicians say that seems sensible while they say it, but doesn't make sense when you think them through. The AI will try to enforce these things... what could possibly go wrong?

The comments on the question mention that your choice of AI technology is not the best. I think that the best would be to say very little about the detailed technologies. Just say "Artificial Intelligence" and leave it at that, or say something like "a combination of different methods".

Every technology would have some sort of cost function they try to minimize and this cost function is the important thing.

$\endgroup$
6
$\begingroup$

Subvert the system

As the author, you are in control of the cost functions / goals of the A.I., and by extension, of the government. Subvert them. For example, let's say that the government and its A.I. are set to provide to everybody according to their needs and to require from everybody according to their capacities. (Original idea, no?) A political movement may arise which promotes expanding the list of necessities; or to define more precisely what capabilities each person has; or to extend the list of goals to include developing the capacities of every person. This may lead to conflicts in resource allocation, conflict between the subgoals, or outright high-tech racism or eugenics à la Gattaca. Or make a new religious movement which worships the A.I. as an avatar of God. Or exploit the inherent tendency of politicians to overspecify the goals; for example, maybe politicians want to reduce inequality while maximizing tax revenue: this could lead to a conflict between those who realize the built-in incompatibility between those goals and those who don't (or pretend that they don't). Or you can have multiple factions fighting over who specifically gets to set the goals of the A.I. Or a conflict between the politicians who think that they set the goals of the A.I. and the engineers who actually set those goals.

Alternative approach: don't use threats to move the plot

Threats are not the only way to move the plot; threats at the level of the entire human society are most certainly unnecessary. There is so much more to life than killing enemies.

There was a great Soviet paleontologist and writer of science-fiction novels, Ivan Yefremov; his best known novel is the Andromeda Nebula, published in 1957; the full text in English is available on the Internet.

The world in the Andromeda Nebula is somewhat similar with the premiss of the question, in that there is a splendidly benevolent and well-informed goverment which is both able and willing to solve all "big" problems of the society -- where "big" means those problems which affect the society as a whole. In the novel, the government is of course Communist, and no A.I. is visible; but then the A.I. would not be necessarily visible, in the same way that the Excel workbooks which underpin the decisions of present-day goverments are not generally popularized on television.

Now, the Andromeda Nebula is a good novel and a long novel (the author was Russian, and Russians don't do short novels), the world is united and ruled by a well-informed and well-intended government, there is very little crime, if any: so how did Yefremov build a plot?

The answer is two-fold: externalities and human nature.

  • Externalities can easily move the plot. Essentially, stuff happens which is not under the control of the perfect government, either because said stuff happens to a small isolated group (the 1st part of the novel, with the adventure of the crew of the Tantra on a dark planet orbiting an iron star), or because the stuff comes out of the blue (2nd part of the novel, when Earth is kindly asked by the Galactic Circle to mount an expedition to find out what happened to a civilization not that many parsecs away).

  • Human nature can always be counted on to move the plot. In the Andromeda Nebula human nature manifests itself by love, jealousy, pride, recognition, glory, and the drive to achieve power (sort-of, not real power but more like authority over a certain community).

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.