Elon Musk: Programming explicit morality into AI can backfire dangerously.

“I do think there's some danger associated with digital superintelligence. I think the biggest issue is that it has to be trained to be rigorously truthful, and it has to be trained to be curious. And I've thought a lot about AI safety for a long time.

One of the challenges you have with programming explicit morality into AI is what people sometimes call the Waluigi problem. If you program Luigi, you can automatically invert that and create Waluigi – bad Luigi. What you cannot invert is physical reality. You can't invert the rules of physics. You can't invert logic.

I think what regulators should be concerned about is, is the AI being rigorously truthful? Is it giving an answer that is most probably correct with acknowledged error? I think that's the best move, and that's what we're trying to do at xAI.”

Viva Tech Paris, May 23, 2024