Abstract: The evolution of the biological brain was constrained by environmental and metabolic demands. But AI has opened up a vast design space, offering novel ways to explore the space at a rate much faster than biological evolution. This exciting new enterprise of mind design is nothing less than a type of intelligent design. But we humans, not God, are the designers.
As exciting as mind design may seem, I explain that if we are not careful we may experience one or more perverse realizations of AI technology -- situations in which AI fails to make life easier but instead leads to our own suffering or demise, or to the exploitation of other conscious beings. The perverse realizations generally fall into one of the following types: (i), overlooked situations involving the creation of conscious machines and (ii), scenarios that concern the use of radical brain enhancements.
I then concentrate on the first type of perverse realization, investigating whether sophisticated AI would be conscious. I urge that we should be skeptical that consciousness will be an inevitable outgrowth of highly sophisticated AI, and that we must develop tests for machine consciousness. Drawing from my earlier research on the computational nature of thought, my tests for machine consciousness (with Princeton's Edwin Turner), and my AI project with NASA, I outline a path forward.