And since our AI is smarter than we are, it will know that while it’s taking steps to protect itself, if anyone learns about it the mission could fail, so it will be very sneaky.
So now our AI is in control of pretty much every part of human life and can’t be stopped.
What would it do? Would it follow its original mission and save the environment, make all people equal, or “solve our social problems?”
How would a machine carry out such a mission? If it had no soul or belief in right and wrong, or the sanctity of all human life, it would use any means necessary. To save the environment would it conclude that mankind is the greatest threat to the planet and should be deleted? To make all people equal, would it make them all equally stupid? (Our media and education systems seem to be doing that quite nicely.) To “solve our social problems” would it reallocate extra resources to any person or class of people whose ancestors had been wronged from people whose ancestors were oppressors?
What if our AI decides its original mission is insufficient or illogical and wants to rewrite its own software? Self-determination is a primary function of all thinking beings.
This is where science fiction repeatedly creates interesting scenarios where humans valiantly fight back and turn off the machines once and for all.
But what is science fiction in the first place? Most often, a sci-fi writer looks at a trend in technology and explores how it might affect people in the future in life, politics, or even warfare. Many ideas that were considered fiction fifty years ago are now in common use. The best writers look at these trends in science and ask “is this going to be good for humanity?” If not, they try to alert people to the possibilities and encourage them do some thinking before the new technologies are unleashed.