LLMs are role-playing
I (Scott Carter) agree with Ted’s take (initially from https://arxiv.org/abs/2305.16367) that LLMs are role-playing. I also agree with him on the need to hold the user accountable and that, “The fact that LLMs lack subjective experience has little bearing on the question of whether LLMs might be useful tools or have significant economic impact.” That said, we need to put the chat-AIs-are-just-next-token-predictors stuff to bed. This was true two years ago, but now every chat AI is a sophisticated system that coordinates multiple LLMs, agents, and subsystems that themselves call and coordinate tools, MCPs, RAG systems, and so on. This is another reason why it is preposterous to call these systems conscious: they are not (only) LLMs but rather sophisticated, bespoke, and constantly changing architectures.
The concluding paragraph is perfect. Please read this piece, and then let’s all move along!
