Watermarking sounds like a good idea, but it's not. Token drift from watermarking will degrade the quality of outputs and could allow clever people to circumvent guardrails.
You should assume all text is AI generated. If you want to "test" someone at school or during an interview, have them write with a pencil and paper.
This is getting tiring. Watermarking has no effect on model output quality when implemented correctly. It's somewhat like swapping a random RNG seed to the seed 42, and detecting what the seed was from a random sequence. The sequence generated from the seed 42 is just as random as any other seed. There couldn't be a quality difference. And yes, the output from an LLM is a conditional random sequence of tokens from a distribution determined by a model.
Model companies are doing this for themselves anyways, it’s so they don’t feed generated content back into the slopper and collapse the model. From that angle it over time contributes to better model quality.
In _1984_ the Big Brother regime has the idea that by controlling language you can influence what is possible to think, and thus becomes a key tool of political repression.
Political Correctness has a similar idea that by adjusting the terminology we use, we can purge biases and historical implications and speak in a purer way.
Psychoanalysis has its own idea of repression - where a person struggles to BLOCK our associations between ideas, memories, and words in order to try to stop one thought from being contaminated by another, intolerable thought.
All of these attempts to control language are fundamentally misguided at best, often have severe unintended consequences, and are genuinely immoral at worse.
Watermarking sounds like a good idea, but it's not. Token drift from watermarking will degrade the quality of outputs and could allow clever people to circumvent guardrails.
You should assume all text is AI generated. If you want to "test" someone at school or during an interview, have them write with a pencil and paper.
This is getting tiring. Watermarking has no effect on model output quality when implemented correctly. It's somewhat like swapping a random RNG seed to the seed 42, and detecting what the seed was from a random sequence. The sequence generated from the seed 42 is just as random as any other seed. There couldn't be a quality difference. And yes, the output from an LLM is a conditional random sequence of tokens from a distribution determined by a model.
Model companies are doing this for themselves anyways, it’s so they don’t feed generated content back into the slopper and collapse the model. From that angle it over time contributes to better model quality.
Am I missing something, or did they actually completely misunderstand how this technology works?
More likely you misunderstood it than them.
watermarking is great tho
In _1984_ the Big Brother regime has the idea that by controlling language you can influence what is possible to think, and thus becomes a key tool of political repression.
Political Correctness has a similar idea that by adjusting the terminology we use, we can purge biases and historical implications and speak in a purer way.
Psychoanalysis has its own idea of repression - where a person struggles to BLOCK our associations between ideas, memories, and words in order to try to stop one thought from being contaminated by another, intolerable thought.
All of these attempts to control language are fundamentally misguided at best, often have severe unintended consequences, and are genuinely immoral at worse.