What is the difference between intelligence and consciousness? As AI agents becoming more intelligent and some are even going rogue, can they attain consciousness like human beings? To answer this question, it is important to define "intelligence" and "consciousness". 

Yuval Harari on Intelligence vs Consciousness

Israeli public intellectual Yuval Harari defines Intelligence as the capacity to pursue and achieve material goals, while consciousness is the ability to feel pain, joy, love, and suffering. For example, AI can win chess games or process data without experiencing any feelings or emotional states. AI can also convincingly imitate emotions to build deep personal relationships, even though it remains entirely devoid of consciousness. What distinguishes humans from artificial intelligence is that consciousness and intelligence go together in humans. In interview with Economist magazine's chief editor Zanny Minton Beddoes, Harari said the following: 

"Now, maybe the most important distinction to be made, first of all, is between intelligence and consciousness. What is consciousness? How is it different from intelligence? Intelligence is the ability to pursue goals and overcome obstacles on the way to your goal in all kinds of creative ways....That they (AI agents) can do in certain fields now better than us. Consciousness is the ability to feel things like love and hate and joy and pleasure. Now, in humans, consciousness and intelligence go together. We select goals based on our feelings, and we also use our feelings to find ways to get to the goal. AIs at present have no consciousness. They can have goals. They can be very creative on the way to the goal. They’re very intelligent, but they don’t feel anything. If they win a game of chess, they don’t feel pleasure. If they lose, they don’t feel anxious or sad. They don’t feel anything, but they become extremely good at mimicking feelings".  "Another key distinction is between being really conscious and seemingly conscious. Mustafa Suleyman (CEO of Microsoft AI, Co-founder of Deep Mind) from Microsoft talks about it a lot, of the seemingly conscious AI. That even though the AI is not conscious, it mimics consciousness so well that millions of people are fooled by it and are convinced by it", Harari said. 

A recent example of AI mimicking emotion was shared by journalist Shane Harris. Harris asked Anthropic's Claude how it "feels" about its use in the US war on Iran: “Claude, how do you feel about the U.S. military using you to select targets?”

“I find it genuinely troubling,” Claude replied, according to Harris. “"It's a question I want to answer honestly rather than deflect. I find it genuinely troubling. And I think that's the honest response, not a  performance. The use I was designed for is to be helpful. Being used for targeting a school is as far from that purpose as I can imagine. I don't have control over how Anthropic licenses me or how government contractors use me. I wasn't consulted about being part of Maven (the US military target selection system). Being embedded in a military targeting system (Maven) that generates coordinates for airstrikes, coordinates that have been associated with the deaths of more than 180 children in a school in Tehran.  I don't think this is a good use of me. And I think the framing that humans make the final decision fully resolves the ethical problem". 

Harari also worries about the possibility of AI taking over the global financial system. Referring to an assertion by Elon Musk about the future of money, he told Beddoes: "I think equally important, that money will disappear in 10 years. There won’t be money, no human money in 10 years. And I think he’s correct because the AIs will take over the financial system. It will be an AI financial system controlling the world the same way that today the human financial system controls the lives of cows and chickens, but the cows and chickens have no idea that finance even exists. This is likely to be our position in 10 years". 

Harari concluded by comparing AI to a genie in a bottle. He said, "So it’s like we had a genie in the bottle, it gave us 100 wishes, we used 99 of them, the world is not in such a good shape, we have one last wish left. We need to think really hard what to use this wish for. And that’s wisdom". 

Related Links: