Consider a much more mundane instruction than the dramatic scenarios we see in movies: “Get me to the airport as quickly as possible.”
A human understands all sorts of things about that prompt that were never explicitly stated. Don’t run people over. Don’t steal a car. Don’t drive through a playground. Don’t create enormous risks just to save five minutes.
We don’t have to list every unacceptable way of accomplishing the goal because humans bring context, social norms, judgment, and an understanding of what another person probably meant.
Outlining every possible unacceptable action for an AI could be impossible. Of course, driverless cars already take people to airports today, and they operate with extensive safety rules, mapping, sensors, and programmed guardrails. But even in a relatively familiar task like driving, humans have spent more than a century studying autonomous vehicles and can still encounter unusual situations and make mistakes.
The challenge becomes much greater when AI is given broader goals in environments where the possible actions and consequences are far less predictable.This is the basic idea behind AI “misalignment:” the system may pursue an objective in a way that technically advances the goal but conflicts with what humans actually intended or valued.
Misalignment does not require an AI to become conscious, malicious, or rebellious. It can be as simple as a gap between the goal we gave the system and the outcome we actually wanted. That gap becomes more dangerous as the system becomes more capable, more autonomous, and able to act in environments we cannot fully predict.
2001: A Space Odyssey gives us a fictional example of a different kind of misalignment: not an unclear instruction, but conflicting ones.In 2001, the Heuristically Programmed Algorithmic Computer 9000, or more simply “HAL” is supposed to provide accurate information and operate reliably for the astronauts aboard the USS Discovery One. But he is also required to conceal the true purpose of the mission from them. Those objectives eventually become incompatible.
