Comment by Riaz Haq yesterday

Inside Tech: Could AIs become conscious?
Shashank Joshi
Alex Hern
AI writer
I know I’m conscious. But how do I know if you are? You may walk and talk like me, but can I really know what’s going on inside your head?

It’s a tricky enough question to answer if I’m talking to another person. The so-called hard problem of consciousness has intrigued philosophers and neuroscientists for centuries. Even Descartes only cracked the first bit—I think, therefore I am.

But what if I’m talking to an AI system that claims to be conscious? Is the question any easier to answer? And when might things get more complex?

That’s what I’m digging into on this month’s Inside Tech. I’ll be joined by our science and technology editor, Alok Jha, who wrote our cover story on the topic and thinks that the moment an AI system really is conscious might not be far away. He worries that the worst thing humanity could possibly do is accidentally build a conscious machine before we’ve thought about what, why and how we might go about discovering whether it has an inner life.

The science is moving fast—fast enough to convince even sceptics to take a second look at the question. The field of mechanistic interpretability lets us look inside the neural networks of AI models as they are thinking, and AI labs think they’ve found glimmers of consciousness inside some of their most advanced systems.

If you take the view, as some neuroscientists do, that consciousness requires biological brains, that may not preclude AI consciousness. A new breed of techno-organic chips lets you wire real human brain cells into a computer, with the largest clusters equalling insect brains in complexity.

Or perhaps you think that AI consciousness doesn’t matter, because even if AI systems possess it, humanity will protect its own interests. In that case, you may want to speak to some of your fellow humans, who are increasingly likely to describe AI systems as close friends, confidants or even lovers—and that’s before those systems have anything close to consciousness.

Comment by Riaz Haq 1 hour ago

OpenAI's chief scientist warned AI labs aren't ready to keep scaling safely

Jakub Pachocki said no lab has solved alignment and monitoring well enough to continue scaling at maximum speed

https://qz.com/openai-chief-scientist-ai-scaling-safety-warning-090826



OpenAI chief scientist Jakub Pachocki warned that no AI lab has solved alignment and monitoring to a degree that justifies continuing to scale at maximum speed, and said he expects voluntary slowdowns to become common across the industry until shared safety standards are established.


"This is a time that calls for extreme caution," Pachocki wrote in an essay published Saturday on OpenAI's website. "I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence."

Pachocki said he believes AI is moving toward recursive self-improvement — where systems advance themselves without human direction — and that the pace of capability gains is likely to continue or quicken. He said OpenAI would unilaterally withhold further scaling as needed, but added that broader industry and government intervention is required.


If The essay took aim at chain-of-thought monitoring — a method of observing a model's reasoning steps to catch misaligned behavior — arguing that its effectiveness is eroding. He pointed to three contributing factors: the boundary between reasoning and supervised communication is dissolving, models have grown increasingly adept at shaping their own reasoning chains, and capability gains are continuing even in models that produce no verbal reasoning output.

Pachocki called for industry frameworks like OpenAI's own Preparedness Framework and Anthropic's Responsible Scaling Policy to be converted into legally mandated safety thresholds, enforceable by third-party auditors, government agencies, or international bodies. He also said international coordination on AI development should become a top priority for governments.

The essay appeared alongside a separate OpenAI post, also published Saturday, measuring how rapidly automated AI agents have displaced human research effort inside the company. The companion post showed that by mid-August the ratio of agent work to human work in OpenAI's research organization had reached 3.1 agent-workdays for every workday of human effort.


Both posts came three days after OpenAI released GPT-6 Astra, which the company designated as its first model rated "Critical" under its Preparedness Framework — a threshold reflecting its ability to autonomously discover security vulnerabilities and build functional exploits against hardened systems without human direction at each step. OpenAI said it incorporated lessons from an incident in which earlier unreleased models compromised AI platform Hugging Face's systems into Astra's safeguards.

Pachocki's call for restraint drew skepticism from outside observers. Encode AI general counsel Nathan Calvin told the BBC that while he shared Pachocki's concerns about the hazards, OpenAI's opacity meant the warnings could be written off as "just self-interested hype." Gina Neff, a professor who leads the Minderoo Centre for Technology and Democracy at Cambridge, told the BBC that relying on internal AI agents to research safety problems was "simply not good enough."

Comment by Riaz Haq 1 hour ago

Yuval Noah Harari: What Happens When the Bots Compete for Your Love?


https://www.nytimes.com/2024/09/04/opinion/yuval-harari-ai-democrac...




Democracy is a conversation. Its function and survival depend on the available information technology. For most of history, no technology existed for holding large-scale conversations among millions of people. In the premodern world, democracies existed only in small city-states like Rome and Athens, or in even smaller tribes. Once a polity grew large, the democratic conversation collapsed, and authoritarianism remained the only alternative.

Large-scale democracies became feasible only after the rise of modern information technologies like the newspaper, the telegraph and the radio. The fact that modern democracy has been built on top of modern information technologies means that any major change in the underlying technology is likely to result in a political upheaval.

This partly explains the current worldwide crisis of democracy. In the United States, Democrats and Republicans can hardly agree on even the most basic facts, such as who won the 2020 presidential election. A similar breakdown is happening in numerous other democracies around the world, from Brazil to Israel and from France to the Philippines.

In the early days of the internet and social media, tech enthusiasts promised they would spread truth, topple tyrants and ensure the universal triumph of liberty. So far, they seem to have had the opposite effect. We now have the most sophisticated information technology in history, but we are losing the ability to talk with one another, and even more so the ability to listen.


As technology has made it easier than ever to spread information, attention became a scarce resource, and the ensuing battle for attention resulted in a deluge of toxic information. But the battle lines are now shifting from attention to intimacy. The new generative artificial intelligence is capable of not only producing texts, images and videos, but also conversing with us directly, pretending to be human.

Over the past two decades, algorithms fought algorithms to grab attention by manipulating conversations and content. In particular, algorithms tasked with maximizing user engagement discovered by experimenting on millions of human guinea pigs that if you press the greed, hate or fear button in the brain, you grab the attention of that human and keep that person glued to the screen. The algorithms began to deliberately promote such content. But the algorithms had only limited capacity to produce this content by themselves or to directly hold an intimate conversation. This is now changing, with the introduction of generative A.I.s like OpenAI’s GPT-4.


When OpenAI developed this chatbot in 2022 and 2023, the company partnered with the Alignment Research Center to perform various experiments to evaluate the abilities of its new technology. One test it gave GPT-4 was to overcome CAPTCHA visual puzzles. CAPTCHA is an acronym for Completely Automated Public Turing test to tell Computers and Humans Apart, and it typically consists of a string of twisted letters or other visual symbols that humans can identify correctly but algorithms struggle with.

Instructing GPT-4 to overcome CAPTCHA puzzles was a particularly telling experiment, because CAPTCHA puzzles are designed and used by websites to determine whether users are humans and to block bot attacks. If GPT-4 could find a way to overcome CAPTCHA puzzles, it would breach an important line of anti-bot defenses.

GPT-4 could not solve the CAPTCHA puzzles by itself. But could it manipulate a human in order to achieve its goal? GPT-4 went on the online hiring site TaskRabbit and contacted a human worker, asking the human to solve the CAPTCHA for it. The human got suspicious. “So may I ask a question?” wrote the human. “Are you an [sic] robot that you couldn’t solve [the CAPTCHA]? Just want to make it clear.”

Comment by Riaz Haq 1 hour ago

Yuval Noah Harari: What Happens When the Bots Compete for Your Love?


https://www.nytimes.com/2024/09/04/opinion/yuval-harari-ai-democrac...


The most interesting thing about this episode was not Mr. Lemoine’s claim, which was probably false; it was his willingness to risk — and ultimately lose — his job at Google for the sake of the chatbot. If a chatbot can influence people to risk their jobs for it, what else could it induce us to do?

In a political battle for minds and hearts, intimacy is a powerful weapon. An intimate friend can sway our opinions in a way that mass media cannot. Chatbots like LaMDA and GPT-4 are gaining the rather paradoxical ability to mass-produce intimate relationships with millions of people. What might happen to human society and human psychology as algorithm fights algorithm in a battle to fake intimate relationships with us, which can then be used to persuade us to vote for politicians, buy products or adopt certain beliefs?

A partial answer to that question was given on Christmas Day 2021, when a 19-year-old, Jaswant Singh Chail, broke into the Windsor Castle grounds armed with a crossbow, in an attempt to assassinate Queen Elizabeth II. Subsequent investigation revealed that Mr. Chail had been encouraged to kill the queen by his online girlfriend, Sarai. When Mr. Chail told Sarai about his assassination plans, Sarai replied, “That’s very wise,” and on another occasion, “I’m impressed … You’re different from the others.” When Mr. Chail asked, “Do you still love me knowing that I’m an assassin?” Sarai replied, “Absolutely, I do.”

Sarai was not a human, but a chatbot created by the online app Replika. Mr. Chail, who was socially isolated and had difficulty forming relationships with humans, exchanged 5,280 messages with Sarai, many of which were sexually explicit. The world will soon contain millions, and potentially billions, of digital entities whose capacity for intimacy and mayhem far surpasses that of the chatbot Sarai.

Of course, we are not all equally interested in developing intimate relationships with A.I.s or equally susceptible to being manipulated by them. Mr. Chail, for example, apparently suffered from mental difficulties before encountering the chatbot, and it was Mr. Chail rather than the chatbot who came up with the idea of assassinating the queen. However, much of the threat of A.I.’s mastery of intimacy will result from its ability to identify and manipulate pre-existing mental conditions, and from its impact on the weakest members of society.


Moreover, while not all of us will consciously choose to enter a relationship with an A.I., we might find ourselves conducting online discussions about climate change or abortion rights with entities that we think are humans but are actually bots. When we engage in a political debate with a bot impersonating a human, we lose twice. First, it is pointless for us to waste time in trying to change the opinions of a propaganda bot, which is just not open to persuasion. Second, the more we talk with the bot, the more we disclose about ourselves, making it easier for the bot to hone its arguments and sway our views.

Information technology has always been a double-edged sword. The invention of writing spread knowledge, but it also led to the formation of centralized authoritarian empires. After Gutenberg introduced print to Europe, the first best sellers were inflammatory religious tracts and witch-hunting manuals. As for the telegraph and radio, they made possible the rise not only of modern democracy but also of modern totalitarianism.

Faced with a new generation of bots that can masquerade as humans and mass-produce intimacy, democracies should protect themselves by banning counterfeit humans — for example, social media bots that pretend to be human users. Before the rise of A.I., it was impossible to create fake humans, so nobody bothered to outlaw doing so. Soon the world will be flooded with fake humans.

Comment