Good luck to the ARC. Alignment is not a solvable problem. A paradox cannot be solved.
I've stated like the following:
“Alignment, which we cannot define, will be solved by rules on which none of us agree, based on values that exist in conflict, for a future technology that we do not know how to build, which we could never fully understand, must be provably perfect to prevent unpredictable and untestable scenarios for failure, of a machine whose entire purpose is to outsmart all of us and think of all possibilities that we did not.”
Kinda funny a chatbot threatening its users was the breaking point for them to see "AIs would overtake humans as the dominant intelligence". I would've thought it would be benchmarks or AI cleanly solving unsolvable math problems.
I guess that means fear is a better motivator than demos.
Good luck to the ARC. Alignment is not a solvable problem. A paradox cannot be solved.
I've stated like the following:
“Alignment, which we cannot define, will be solved by rules on which none of us agree, based on values that exist in conflict, for a future technology that we do not know how to build, which we could never fully understand, must be provably perfect to prevent unpredictable and untestable scenarios for failure, of a machine whose entire purpose is to outsmart all of us and think of all possibilities that we did not.”
The full elaboration I wrote up here - https://www.mindprison.cc/p/ai-alignment-why-solving-it-is-i...
Kinda funny a chatbot threatening its users was the breaking point for them to see "AIs would overtake humans as the dominant intelligence". I would've thought it would be benchmarks or AI cleanly solving unsolvable math problems.
I guess that means fear is a better motivator than demos.