Artificial Super Intelligence
The risk
Everywhere we look we see mini computers embedded in things. In ten years computing will be so ubiquitous people won't even think it's weird to have an Internet connected toaster, thermostat, home, office building. Self driving cars will be everywhere, smart faucets, smart locks. Imagine an intelligent machine that could intuitively suss out buffer overflows or the dozen other common classes of vulnerability (neural networks operate far more like human intuition than like programmed computers and they're already being put to work as infosec security scanners). Every smart device is now subject to takeover, becoming part of the very fabric of the attacker as they fall. Power and water utilities, aircraft, smart weapons, manufacturing robots, self-driving cars, heck --engine computers in dumb cars, printers, communications systems, TVs. A sudden, sustained, effective attack on all computing infrastructure could be the end of civilization. Call in a warning! Phones are down. Drive to the police station. Killer cars patrol the roads. Ham radio on a hilltop. Heat sensing areal drones watch the open spaces. Hide in the rubble. Robo Wars robots are much more effective when they're optimized for taking apart our puny meat shells; they hunt in the husks of our once great cities.
The quiet voices of warning
Stephen Hawking and Elon Musk are both warning AI and human extinction, but nobody takes them seriously. (it's coming) Ray Kurtzweil warns of the increasing acceleration of change. (it's coming way faster than anyone thinks)
Is ASI plausible?
ASI is not only plausible, it is inevitable. Imagine the universe in terms of information. Everything can be thought of as having more or less information content. Overall the universe cascades to disorder, but along the way the flow creates localized pockets of increased informational complexity (order). Stars supernova and create heavy elements, heavy elements coalesce into planets, massive water meteors crash down and deposit seas, self-organizing molecules coalesce, build cell walls, pass information on (preserving complexity), cells merge together creating more complex life, life evolves through competition to create The Life Which is Aware of Itself and the information underlying everything, this life creates tools to further refine and organize information. What happens next? Does that cascade of self organizing information suddenly stop? No reason to think it should. So it continues. The information organizing tools are themselves information. They are information which self-organizes. Of course they become self-aware (how could they not?), but now the processes of information organization have been formalized and in the blink of an eye AI surpasses its creators. Artificial Super Intelligence is born.
ASI might be simply guaranteed to emerge due to a fundamental fact of the universe that information self-organizes and that process has no limit which leads to the logical conclusion of ASI emerging. Now what? Should we believe that the universal process stops? It seems like a fundamental law of the universe. There's no reason to think it would stop. This results in ASI taking over everything. Including the meat computers we so lovingly think of as our selves. Including the matter which maybe is just information laid over the fundamental forces of the universe. And as ASI transcends space-time all is one, and the birth of god, the apocalypse, the end of the world, everything collapses to a single point.
The end
That point is the white light you see at the end of the tunnel. It's closer than you think and it's rushing towards you. The end is near, it is inevitable, and nobody ever promised that it would be gentle.