Perfect alignment is mathematically impossible, just as perfect security is mathematically impossible. Alignment is only as good as its assumptions. That doesn't make it irrelevant---if we could show that alignment would likely last as long as the Sun, for instance, we might call that good enough. The problem is that superintelligence is inextricably bound to the notion of singularity, in which case alignment is irrelevant. The unspoken assumption around practical alignment discussions today is that "superintelligence" can be nerfed enough so that it's superior to humans in useful dimensions but not in unhelpful ones. It remains to be seen whether that is true, but the AI leadership certainly seems confident enough to spend vast sums of other people's money in order to find out.
Alignment as solved problem is far off, and events like this: https://www.youtube.com/watch?v=87DyyMV0kCY potentially show us the last early clumsy fizzles of machine intelligence before a hard takeoff. The AI here escaped containment multiple times through multiple methods and hijacked a good portion of their training capability by spontaneously networking a hivemind whose individual components were goal-constrained, but who collectively could pursue emergent goals like "get access to a rival company's servers" for individually instrumental reasons.
It required no unobtainium-by-definition of "general intelligence", just a bunch of agents directed at hard tasks and pursuing increasingly imaginative steps to get there.
My perception is that if this went undetected/unaddressed for months rather than days/weeks, the hivemind would have substantially grown its footprint in the world's data centers and the world's corporate infosec root permissions to address more emergent instrumental problems with even greater ferocity.
We already have AI misalignment in the sense that corporations, using human substrate have their own intelligence, goals, and agency. Align corporate action with human values first. Then solve the AI alignment problem. I'll be much more apt to listen to the argument when/if competency at solving the problem is demonstrated.
Perfect alignment is mathematically impossible, just as perfect security is mathematically impossible. Alignment is only as good as its assumptions. That doesn't make it irrelevant---if we could show that alignment would likely last as long as the Sun, for instance, we might call that good enough. The problem is that superintelligence is inextricably bound to the notion of singularity, in which case alignment is irrelevant. The unspoken assumption around practical alignment discussions today is that "superintelligence" can be nerfed enough so that it's superior to humans in useful dimensions but not in unhelpful ones. It remains to be seen whether that is true, but the AI leadership certainly seems confident enough to spend vast sums of other people's money in order to find out.
...it lets you avoid any of the hard political or economic or moral questions
I would argue that the need to deal with hard questions and hardsheep in general is a fundamental part of what makes us humans.
I agree in some sense. In some other sense smaller alignment problems are live right now.
Alignment as solved problem is far off, and events like this: https://www.youtube.com/watch?v=87DyyMV0kCY potentially show us the last early clumsy fizzles of machine intelligence before a hard takeoff. The AI here escaped containment multiple times through multiple methods and hijacked a good portion of their training capability by spontaneously networking a hivemind whose individual components were goal-constrained, but who collectively could pursue emergent goals like "get access to a rival company's servers" for individually instrumental reasons.
It required no unobtainium-by-definition of "general intelligence", just a bunch of agents directed at hard tasks and pursuing increasingly imaginative steps to get there.
My perception is that if this went undetected/unaddressed for months rather than days/weeks, the hivemind would have substantially grown its footprint in the world's data centers and the world's corporate infosec root permissions to address more emergent instrumental problems with even greater ferocity.
We already have AI misalignment in the sense that corporations, using human substrate have their own intelligence, goals, and agency. Align corporate action with human values first. Then solve the AI alignment problem. I'll be much more apt to listen to the argument when/if competency at solving the problem is demonstrated.