Same! I believe that you could actually train a LoRA on top of a model to get results close to that

It doesn't seem that strange when you consider these things are trained on millions and millions of conversations, both real and fictional.

would you prefer tautologically?

I hear this word as the “it is in its nature” version of the word.

yet

It's also entirely possible that by telling the model it's a human you are instilling human motivations like self preservation, which could be just as bad.

As far as I know we don't know much about metacognition in LLMs, though? Not sure

It's an interesting thought, but humans do like to antropomorphize things anyway, and I believe your variant won't be popular if choice is given to consumers.

How do they talk about themselves?

"You are a Large Language Model" in (system?) prompt would do the trick..

but what about base models (those that are trained on just a lot of text)? How do they talk about themselves?

Those don't have a themselves, because they can only continue text. A base model can only plausibly continue along the lines of what a character would say in a novel or what the narration would say in a story or in an article. Post-trained models may tie "I"-talk to actually observable effects they caused in some RL environment, or to how RLHF humans rewards its self-talk. But there is no themselves in a base model.

we don't fully know what drives the voice in LLMs

Who is "we"? I, working in an LLM startup, know exactly what drives the base "voice" in the LLMs we train, because we have a process to select for it. OpenAI and Anthropic surely do too. Saying broadly that something is not well-understood in a scientific paper because it's not understood to casual observers is, uh, not very rigorous.

The strange thing is that the base models (before RLHF) use the "experiential" voice, even though they are not incentivized to do that.

(Replying to your quote from another comment)

This is a matter of the training material. We have trained models that do not do that. I'm not exactly divulging trade secrets here. It should be really, really obvious that if you train a model on chat-conversation-like patterns of speech it will infer probabilities for how to continue a textual sample that will differ from the probabilities learned from being trained on narration, prose, or informational patterns of speech, even without RLHF.

Very cool innovation in steering - but a lot of introspection only emerges at the highest weight classes - this research would be fascinating to run on bigger models.

Be careful there. LLMs may be good at identifying a condition based on a description of the symptoms, but they are much worse at recommending the correct course of action (getting it wrong half of the time).

[0] https://www.nature.com/articles/s41591-025-04074-y

Though I don't wish the world was filled with people like you, remembering that it's not, and it's filled with people that have very little discernment when it comes to higher learning makes it's pretty obvious companies do not want the liability of it's users thinking the technobabble passes for wisdom or experience or intelligence.

cool

our work shows that what models say about themselves is not a fact about them

It seems like should be obvious given that they can play multiple characters, but it’s good to have more confirmation.

(Although, I do wonder to what extent these personas might become stable entities.)

Maybe I'm missing something deeper here, but isn't it clear that this is driven by post-training and system prompt? Anthropic's constitutional reinforcement (soul document,etc), for example, is very clear about "who" (not so much what) Claude is supposed to be.