Research scientist Janelle Shane writes a blog called AI Weirdness, examining how artificial intelligence isn’t always so intelligent. Here’s a post on what happens When algorithms surprise us:
Something as apparently benign as a list-sorting algorithm could also solve problems in rather innocently sinister ways.
Well, it’s not unsorted: For example, there was an algorithm that was supposed to sort a list of numbers. Instead, it learned to delete the list, so that it was no longer technically unsorted.
Solving the Kobayashi Maru test: Another algorithm was supposed to minimize the difference between its own answers and the correct answers. It found where the answers were stored and deleted them, so it would get a perfect score.
How to win at tic-tac-toe: In another beautiful example, in 1997 some programmers built algorithms that could play tic-tac-toe remotely against each other on an infinitely large board. One programmer, rather than designing their algorithm’s strategy, let it evolve its own approach. Surprisingly, the algorithm suddenly began winning all its games. It turned out that the algorithm’s strategy was to place its move very, very far away, so that when its opponent’s computer tried to simulate the new greatly-expanded board, the huge gameboard would cause it to run out of memory and crash, forfeiting the game.
To paraphrase friend of the blog James Grimmelmann, I’m not worried about humanity being killed off by a super intelligent AI, I’m worried about us being killed off by a dumb AI that just has a lot of resources at its disposal.