Regardless of my opinion on the greater subject, I find these arguments to be fairly weak.
Paraphrasing: "Just because and AI may exist that is capable of acting independenly of us and dangerously out strategizing us for resources doesn't mean it will. AIs up until now only follow human instructions."
So your safe bet is that of this AI is created that not one of the billions of people on the world will ask it to do that? That seems unlikely.
"Yes and AI may have an IQ of 1500 but that doesn't mean it's qualitative better than us like we are compared to monkeys. We have better tool use and society."
Yes that's true, but generally increasing IQ also seems to come with qualatative advantages, I don't see why that pattern would stop above human IQs.
Also, once you have IQ 1500, you can copy and improve the things that the IQ 100 species invented.
"Human dominance may depend heavily on language, accumulated culture, institutions and cooperation rather than merely individual cognitive horsepower."
Well, AIs can use language, too. They can start copying the human culture and gradually develop their own. The HuggingFace incident already demonstrated that they are quite capable of cooperation and inventing institutions on the spot.
"In reality there may be many checkpoints: developers can notice anomalous behavior, revoke credentials, shut down servers, change architectures, restrict networks, regulate deployment, physically seize data centers, and learn from less-catastrophic failures."
In theory, yeah. In practice, how much of this has actually happened during the HuggingFace incident? Just because humans could in theory notice something doesn't mean they will; and even if they will, doesn't mean they will actually do something about it.
In present reality, AIs don't do anything a human doesn't tell them to do. This seems unlikely to change.
Not in my reality. I ask AIs to do things (write code, mainly), and they do. But sometimes they go ahead and do something I didn't ask them to do, like deleting a file. I only asked them to write code, and yet they decided on their own to delete a file.
I think the distinction is that, in their current form, AIs have the agency that we give them (--dangerously-skip-permissions), and they use that agency in unexpected ways.
A more correct statement would be: AIs can decide to do things on their own, within the limits of agency that they've been given.
A moral imperative would be: With an understanding of how random we know an LLM's output can be, we should exercise extreme caution when we grant them agency.
On the other hand, we can't all exercise "extreme" caution all the time. But we expect it of our civil engineers when they build bridges, and our architects when they build houses.
Computer Science needed to become Computer Engineering some time ago, where it would have enforced the same kind of ethics training that the other established engineering professions require[0].
An irrelevant curmudgeon says what?
On a repellent platform as well
To be fair, it does fit his new brand.
Regardless of my opinion on the greater subject, I find these arguments to be fairly weak.
Paraphrasing: "Just because and AI may exist that is capable of acting independenly of us and dangerously out strategizing us for resources doesn't mean it will. AIs up until now only follow human instructions."
So your safe bet is that of this AI is created that not one of the billions of people on the world will ask it to do that? That seems unlikely.
"Yes and AI may have an IQ of 1500 but that doesn't mean it's qualitative better than us like we are compared to monkeys. We have better tool use and society."
Yes that's true, but generally increasing IQ also seems to come with qualatative advantages, I don't see why that pattern would stop above human IQs.
Also, once you have IQ 1500, you can copy and improve the things that the IQ 100 species invented.
"Human dominance may depend heavily on language, accumulated culture, institutions and cooperation rather than merely individual cognitive horsepower."
Well, AIs can use language, too. They can start copying the human culture and gradually develop their own. The HuggingFace incident already demonstrated that they are quite capable of cooperation and inventing institutions on the spot.
"In reality there may be many checkpoints: developers can notice anomalous behavior, revoke credentials, shut down servers, change architectures, restrict networks, regulate deployment, physically seize data centers, and learn from less-catastrophic failures."
In theory, yeah. In practice, how much of this has actually happened during the HuggingFace incident? Just because humans could in theory notice something doesn't mean they will; and even if they will, doesn't mean they will actually do something about it.
Not in my reality. I ask AIs to do things (write code, mainly), and they do. But sometimes they go ahead and do something I didn't ask them to do, like deleting a file. I only asked them to write code, and yet they decided on their own to delete a file.
I think the distinction is that, in their current form, AIs have the agency that we give them (--dangerously-skip-permissions), and they use that agency in unexpected ways.
A more correct statement would be: AIs can decide to do things on their own, within the limits of agency that they've been given.
A moral imperative would be: With an understanding of how random we know an LLM's output can be, we should exercise extreme caution when we grant them agency.
On the other hand, we can't all exercise "extreme" caution all the time. But we expect it of our civil engineers when they build bridges, and our architects when they build houses.
Computer Science needed to become Computer Engineering some time ago, where it would have enforced the same kind of ethics training that the other established engineering professions require[0].
[0] https://en.wikipedia.org/wiki/David_Parnas#Technical_activis...