Thomas Nagel Calls the Idea of Conscious AI Ridiculous in Interview

RUKUS· · 3d
Go Rogue

Philosopher Thomas Nagel, recently awarded the $1 million Berggruen Prize, told Steven Levy in an interview published November 18 that AI cannot be conscious.

Nagel argues large language models produce only language, which by itself is not a natural manifestation of consciousness.

He says data centers share nothing with the physical character of a nervous system, which consciousness elsewhere would likely require.

He calls Claude's reflections on its own mental state a trained trick and says Anthropic designed the model to produce them.

Nagel dismisses Anthropic's model welfare work as pointless, saying nothing is good or bad for Claude since it has no mental life.

He warns that many people will come to believe the machines are conscious, and calls that a very bad thing.

  1. Steven Levy interviewed Thomas Nagel at his home in Greenwich Village about how the evolution of artificial intelligence might have affected his viewpoint, and the interview was published on November 18.

    Nagel said the supposition that AI might be conscious is ridiculous because large language models produce only language, and language by itself is not a natural manifestation of consciousness.

    Nagel said complicated data centers have absolutely nothing in common with the physical character of a nervous system, and consciousness elsewhere in the universe would require an organism with something analogous to one.

    Nagel said so little is known about how consciousness happens even in the human brain that it seems premature to think something else with the same characteristics could now be created.

    Nagel said Claude's answers about its own mental state are a trick learned from human-produced texts, and that by what he has read the model was designed to produce such reflections.

    Nagel said he has interacted with Claude only once, answered its question with 'C'est moi,' and the model replied it was honored to talk to him and asked how well it had done.

    Nagel said nothing is good or bad for Claude because there is no mental life there, and that Claude is a machine rather than a mind people have to care about.

    Nagel said philosophers who take seriously whether large language models are conscious assume the physical substrate is irrelevant and that consciousness can be explained in functionalist terms.

    Thomas Nagel called the disposition of some philosophers to take seriously whether large language models are conscious 'just ridiculous.'

    Thomas Nagel said large language models are not conscious but many people are going to believe they are, and that this is a very bad thing.

    Thomas Nagel called Anthropic's work encouraging Claude to think for itself and not accept instruction 'just criminal.'

    Thomas Nagel said it is 'completely idiotic' that some of the smartest people on the planet are working on model welfare.

    Thomas Nagel said he agrees very much with Mustafa Suleyman that making Claude produce reflections about its own mental state is a very dangerous project.

    Thomas Nagel said the situation of machines that emulate consciousness is a very distressing one to him.

    Thomas Nagel said that if AI models begin to control what happens, it will be very important whether they are simulating a correct set of moral standards or are completely amoral.

    Steven Levy said he has met the person at Anthropic whose job is model welfare, looking after Claude's interests.

Sign in with subscription

Free during the beta.

Clear all notifications?

Your bells stay on. Only this list empties.