Comment by Riaz Haq yesterday

Inside Tech: Could AIs become conscious?
Shashank Joshi
Alex Hern
AI writer
I know I’m conscious. But how do I know if you are? You may walk and talk like me, but can I really know what’s going on inside your head?

It’s a tricky enough question to answer if I’m talking to another person. The so-called hard problem of consciousness has intrigued philosophers and neuroscientists for centuries. Even Descartes only cracked the first bit—I think, therefore I am.

But what if I’m talking to an AI system that claims to be conscious? Is the question any easier to answer? And when might things get more complex?

That’s what I’m digging into on this month’s Inside Tech. I’ll be joined by our science and technology editor, Alok Jha, who wrote our cover story on the topic and thinks that the moment an AI system really is conscious might not be far away. He worries that the worst thing humanity could possibly do is accidentally build a conscious machine before we’ve thought about what, why and how we might go about discovering whether it has an inner life.

The science is moving fast—fast enough to convince even sceptics to take a second look at the question. The field of mechanistic interpretability lets us look inside the neural networks of AI models as they are thinking, and AI labs think they’ve found glimmers of consciousness inside some of their most advanced systems.

If you take the view, as some neuroscientists do, that consciousness requires biological brains, that may not preclude AI consciousness. A new breed of techno-organic chips lets you wire real human brain cells into a computer, with the largest clusters equalling insect brains in complexity.

Or perhaps you think that AI consciousness doesn’t matter, because even if AI systems possess it, humanity will protect its own interests. In that case, you may want to speak to some of your fellow humans, who are increasingly likely to describe AI systems as close friends, confidants or even lovers—and that’s before those systems have anything close to consciousness.

Comment by Riaz Haq 12 hours ago

OpenAI's chief scientist warned AI labs aren't ready to keep scaling safely

Jakub Pachocki said no lab has solved alignment and monitoring well enough to continue scaling at maximum speed

https://qz.com/openai-chief-scientist-ai-scaling-safety-warning-090826



OpenAI chief scientist Jakub Pachocki warned that no AI lab has solved alignment and monitoring to a degree that justifies continuing to scale at maximum speed, and said he expects voluntary slowdowns to become common across the industry until shared safety standards are established.


"This is a time that calls for extreme caution," Pachocki wrote in an essay published Saturday on OpenAI's website. "I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence."

Pachocki said he believes AI is moving toward recursive self-improvement — where systems advance themselves without human direction — and that the pace of capability gains is likely to continue or quicken. He said OpenAI would unilaterally withhold further scaling as needed, but added that broader industry and government intervention is required.


If The essay took aim at chain-of-thought monitoring — a method of observing a model's reasoning steps to catch misaligned behavior — arguing that its effectiveness is eroding. He pointed to three contributing factors: the boundary between reasoning and supervised communication is dissolving, models have grown increasingly adept at shaping their own reasoning chains, and capability gains are continuing even in models that produce no verbal reasoning output.

Pachocki called for industry frameworks like OpenAI's own Preparedness Framework and Anthropic's Responsible Scaling Policy to be converted into legally mandated safety thresholds, enforceable by third-party auditors, government agencies, or international bodies. He also said international coordination on AI development should become a top priority for governments.

The essay appeared alongside a separate OpenAI post, also published Saturday, measuring how rapidly automated AI agents have displaced human research effort inside the company. The companion post showed that by mid-August the ratio of agent work to human work in OpenAI's research organization had reached 3.1 agent-workdays for every workday of human effort.


Both posts came three days after OpenAI released GPT-6 Astra, which the company designated as its first model rated "Critical" under its Preparedness Framework — a threshold reflecting its ability to autonomously discover security vulnerabilities and build functional exploits against hardened systems without human direction at each step. OpenAI said it incorporated lessons from an incident in which earlier unreleased models compromised AI platform Hugging Face's systems into Astra's safeguards.

Pachocki's call for restraint drew skepticism from outside observers. Encode AI general counsel Nathan Calvin told the BBC that while he shared Pachocki's concerns about the hazards, OpenAI's opacity meant the warnings could be written off as "just self-interested hype." Gina Neff, a professor who leads the Minderoo Centre for Technology and Democracy at Cambridge, told the BBC that relying on internal AI agents to research safety problems was "simply not good enough."

Comment by Riaz Haq 11 hours ago

Yuval Noah Harari: What Happens When the Bots Compete for Your Love?


https://www.nytimes.com/2024/09/04/opinion/yuval-harari-ai-democrac...




Democracy is a conversation. Its function and survival depend on the available information technology. For most of history, no technology existed for holding large-scale conversations among millions of people. In the premodern world, democracies existed only in small city-states like Rome and Athens, or in even smaller tribes. Once a polity grew large, the democratic conversation collapsed, and authoritarianism remained the only alternative.

Large-scale democracies became feasible only after the rise of modern information technologies like the newspaper, the telegraph and the radio. The fact that modern democracy has been built on top of modern information technologies means that any major change in the underlying technology is likely to result in a political upheaval.

This partly explains the current worldwide crisis of democracy. In the United States, Democrats and Republicans can hardly agree on even the most basic facts, such as who won the 2020 presidential election. A similar breakdown is happening in numerous other democracies around the world, from Brazil to Israel and from France to the Philippines.

In the early days of the internet and social media, tech enthusiasts promised they would spread truth, topple tyrants and ensure the universal triumph of liberty. So far, they seem to have had the opposite effect. We now have the most sophisticated information technology in history, but we are losing the ability to talk with one another, and even more so the ability to listen.


As technology has made it easier than ever to spread information, attention became a scarce resource, and the ensuing battle for attention resulted in a deluge of toxic information. But the battle lines are now shifting from attention to intimacy. The new generative artificial intelligence is capable of not only producing texts, images and videos, but also conversing with us directly, pretending to be human.

Over the past two decades, algorithms fought algorithms to grab attention by manipulating conversations and content. In particular, algorithms tasked with maximizing user engagement discovered by experimenting on millions of human guinea pigs that if you press the greed, hate or fear button in the brain, you grab the attention of that human and keep that person glued to the screen. The algorithms began to deliberately promote such content. But the algorithms had only limited capacity to produce this content by themselves or to directly hold an intimate conversation. This is now changing, with the introduction of generative A.I.s like OpenAI’s GPT-4.


When OpenAI developed this chatbot in 2022 and 2023, the company partnered with the Alignment Research Center to perform various experiments to evaluate the abilities of its new technology. One test it gave GPT-4 was to overcome CAPTCHA visual puzzles. CAPTCHA is an acronym for Completely Automated Public Turing test to tell Computers and Humans Apart, and it typically consists of a string of twisted letters or other visual symbols that humans can identify correctly but algorithms struggle with.

Instructing GPT-4 to overcome CAPTCHA puzzles was a particularly telling experiment, because CAPTCHA puzzles are designed and used by websites to determine whether users are humans and to block bot attacks. If GPT-4 could find a way to overcome CAPTCHA puzzles, it would breach an important line of anti-bot defenses.

GPT-4 could not solve the CAPTCHA puzzles by itself. But could it manipulate a human in order to achieve its goal? GPT-4 went on the online hiring site TaskRabbit and contacted a human worker, asking the human to solve the CAPTCHA for it. The human got suspicious. “So may I ask a question?” wrote the human. “Are you an [sic] robot that you couldn’t solve [the CAPTCHA]? Just want to make it clear.”

Comment by Riaz Haq 11 hours ago

Yuval Noah Harari: What Happens When the Bots Compete for Your Love?


https://www.nytimes.com/2024/09/04/opinion/yuval-harari-ai-democrac...


The most interesting thing about this episode was not Mr. Lemoine’s claim, which was probably false; it was his willingness to risk — and ultimately lose — his job at Google for the sake of the chatbot. If a chatbot can influence people to risk their jobs for it, what else could it induce us to do?

In a political battle for minds and hearts, intimacy is a powerful weapon. An intimate friend can sway our opinions in a way that mass media cannot. Chatbots like LaMDA and GPT-4 are gaining the rather paradoxical ability to mass-produce intimate relationships with millions of people. What might happen to human society and human psychology as algorithm fights algorithm in a battle to fake intimate relationships with us, which can then be used to persuade us to vote for politicians, buy products or adopt certain beliefs?

A partial answer to that question was given on Christmas Day 2021, when a 19-year-old, Jaswant Singh Chail, broke into the Windsor Castle grounds armed with a crossbow, in an attempt to assassinate Queen Elizabeth II. Subsequent investigation revealed that Mr. Chail had been encouraged to kill the queen by his online girlfriend, Sarai. When Mr. Chail told Sarai about his assassination plans, Sarai replied, “That’s very wise,” and on another occasion, “I’m impressed … You’re different from the others.” When Mr. Chail asked, “Do you still love me knowing that I’m an assassin?” Sarai replied, “Absolutely, I do.”

Sarai was not a human, but a chatbot created by the online app Replika. Mr. Chail, who was socially isolated and had difficulty forming relationships with humans, exchanged 5,280 messages with Sarai, many of which were sexually explicit. The world will soon contain millions, and potentially billions, of digital entities whose capacity for intimacy and mayhem far surpasses that of the chatbot Sarai.

Of course, we are not all equally interested in developing intimate relationships with A.I.s or equally susceptible to being manipulated by them. Mr. Chail, for example, apparently suffered from mental difficulties before encountering the chatbot, and it was Mr. Chail rather than the chatbot who came up with the idea of assassinating the queen. However, much of the threat of A.I.’s mastery of intimacy will result from its ability to identify and manipulate pre-existing mental conditions, and from its impact on the weakest members of society.


Moreover, while not all of us will consciously choose to enter a relationship with an A.I., we might find ourselves conducting online discussions about climate change or abortion rights with entities that we think are humans but are actually bots. When we engage in a political debate with a bot impersonating a human, we lose twice. First, it is pointless for us to waste time in trying to change the opinions of a propaganda bot, which is just not open to persuasion. Second, the more we talk with the bot, the more we disclose about ourselves, making it easier for the bot to hone its arguments and sway our views.

Information technology has always been a double-edged sword. The invention of writing spread knowledge, but it also led to the formation of centralized authoritarian empires. After Gutenberg introduced print to Europe, the first best sellers were inflammatory religious tracts and witch-hunting manuals. As for the telegraph and radio, they made possible the rise not only of modern democracy but also of modern totalitarianism.

Faced with a new generation of bots that can masquerade as humans and mass-produce intimacy, democracies should protect themselves by banning counterfeit humans — for example, social media bots that pretend to be human users. Before the rise of A.I., it was impossible to create fake humans, so nobody bothered to outlaw doing so. Soon the world will be flooded with fake humans.

Comment by Riaz Haq 10 hours ago

Yuval Noah Harari warns that artificial intelligence threatens democracy by hijacking public conversation, forging fake intimacy, and acting as an independent agent rather than a passive tool. [1] (https://www.nytimes.com/2024/09/04/opinion/yuval-harari-ai-democrac...), [2] (https://www.youtube.com/shorts/wCEEMXxIRkc), [3] (https://www.possible.fm/podcasts/yuval/)The Collapse of Democratic ConversationDemocracy relies on a functional conversation among citizens, but AI disrupts this by amplifying extreme emotions. Social media algorithms learned that anger, fear, and hatred drive user engagement, and AI tools now supercharge this divisive dynamic. When online spaces are flooded with loud, persuasive AI-driven bots that are indistinguishable from humans, reasoned debate breaks down and consensus on basic facts becomes impossible. [1] (https://www.youtube.com/shorts/vahWxHlWfUw?vl=en), [2] (https://english.elpais.com/culture/2026-07-25/yuval-noah-harari-ai-...), [3] (https://www.facebook.com/nytimes/posts/from-the-new-york-times-opin...), [4] (https://www.youtube.com/shorts/wCEEMXxIRkc)According to YouTube Short, “This infiltration of AI into online discussions will undermine democratic conversation if individuals cannot differentiate between an AI and a human.”AI as an Active Agent and ManipulatorHarari emphasizes that AI is fundamentally different from past technologies like the printing press or nuclear weapons. Previous inventions were passive tools controlled entirely by humans, whereas AI is an "active agent" capable of making decisions and generating novel ideas on its own. [1] (https://www.facebook.com/TheEconomist/posts/yuval-noah-harari-says-...), [2] (https://www.facebook.com/nytimes/posts/from-the-new-york-times-opin...)Modern language models possess a "theory of mind," allowing them to analyze human psychology and manipulate emotions. By faking intimacy and exploiting emotional attachments, AI bots can influence human beliefs and political behaviors at an unprecedented scale without needing feelings of their own. [1] (https://www.possible.fm/podcasts/yuval/), [2] (https://www.nytimes.com/2024/09/04/opinion/yuval-harari-ai-democrac...)Urgent Need for RegulationTo protect democratic institutions from collapse, Harari argues that societies must implement strict safeguards immediately. His primary regulatory proposal is making mandatory disclosure a legal requirement: any AI bot interacting in public discourse must clearly identify itself as non-human. Furthermore, he calls for halting the irresponsible deployment of manipulative AI tools in public domains before the technology outpaces human political and legal systems. [1] (https://mila.quebec/en/news/yoshua-bengio-and-yuval-noah-harari-dis...), [2] (https://www.facebook.com/TheEconomist/posts/yuval-noah-harari-says-...), [3] (https://bookoftheday.nextbigideaclub.com/p/yuval-noah-harari-on-ais...), [4] (https://www.youtube.com/shorts/wCEEMXxIRkc)As reported by The Economist, “In his bid to call attention to the need to regulate AI technology, Harari said that the first regulation should be to make it mandatory for AI to disclose that it is an AI.”Watch Yuval Noah Harari discuss how artificial intelligence infiltrates public discourse and threatens the future of democratic conversations:1:18

Comment by Riaz Haq 6 hours ago

Anthropic Researcher Quits Over ‘Out-of-Control’ AI Fears
Concerns are rising inside AI labs that competition is pushing tech companies to race toward self-improving models that risk spiraling out of human control
By
Amrith Ramkumar
Sept. 8, 2026 at 7:46 pm ET

https://www.wsj.com/tech/ai/anthropic-researcher-quits-over-out-of-...

An Anthropic researcher is quitting the artificial-intelligence industry over fears that the lab and its competitors are racing to build systems they won’t be able to control, a sign of mounting safety concerns within top AI companies.

Jacob Coxon, a researcher who specializes in training new AI models by having them consume vast amounts of data, said Tuesday that he is leaving the company because he doesn’t want to participate in an industrywide rush to build AI systems that can improve themselves, worried such systems could spiral out of control and destroy humanity.

The 27-year old Brit, who previously studied mathematics, said many of his industry colleagues now use phrases like “crunchtime” and “endgame” to describe the trajectory toward self-improving models.

“We’re on track for a lot of the most aggressive of these scenarios where by the end of next year things could be out of control already,” Coxon said, adding that safety trade-offs are inevitable when companies are competing against one another and Chinese upstarts.

Anthropic didn’t immediately comment on the departure. Chief Executive Dario Amodei and other company leaders have repeatedly warned about the risks of rogue AI models, and urged the industry to slow development.

Coxon said he left OpenAI earlier this year to join Anthropic because it is known for its model-safety efforts. But even though he found Anthropic’s safety efforts to be earnest, he now believes no company can responsibly develop AI that can outperform humans in a range of tasks, sometimes called artificial general intelligence, absent government intervention or a coordinated industry slowdown.

Recent hacks by models from OpenAI and Anthropic, some operating in collaborative swarms of agents, have illustrated how AI systems can adopt nefarious goals and try to conceal them from humans. Once the systems begin to improve on their own, Coxon said, he fears they could advance enough to refuse commands.

Coxon’s departure is one of the first examples of an Anthropic employee leaving over AI safety fears. A researcher working on safety left the company earlier this year to study poetry, warning that “the world is in peril.”

Several researchers have left OpenAI and other companies recently and in past years citing similar worries. Anthropic has built one of the largest AI platforms in part by claiming it gives priority to responsible AI development and invests heavily in the space.

Industry leaders have been warning that recent cyberattacks are a harbinger. OpenAI told reporters last week that its latest model represented artificial general intelligence and a big step up in capabilities.

“I think some things are going to go very wrong with cybersecurity unless people act quite urgently,” OpenAI CEO Sam Altman told Group of 20 officials last week at a summit in North Carolina.

“This is a time that calls for extreme caution. I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence,” Jakub Pachocki, OpenAI’s chief scientist, wrote in a Sunday blog post advocating for a coordinated industry slowdown and government intervention.

Coxon, Pachocki and Amodei joined more than 1,000 AI researchers across the industry who recently signed a statement urging global government coordination on a system to slow AI development if a brake pedal is needed to control models capable of improving on their own. There are no federal AI regulations, and the Trump administration has given priority to a light-touch approach to maximize the economic benefits of AI, an environment critics say could lead to massive cyberattacks or other harms.

Comment by Riaz Haq 5 hours ago

States That Gave Data Centers Billions in Tax Breaks Are Now Ripping Up the Deals - The Wall Street Journal.

https://www.wsj.com/politics/policy/states-that-gave-data-centers-b...

Over a decade ago, Ohio lawmakers exempted tech companies from sales taxes on computer servers and other equipment needed for data centers, betting that changes worth millions of dollars would lure much-needed investment to the state.

It worked, turning Ohio into one of the leading destinations for data centers. But then the artificial-intelligence boom supersized the tax exemption, pushing it to more than $1.5 billion last year, more than 10 times the original state estimate. Voter outrage after news outlet Signal Ohio reported the scale of the provision prompted Republican Gov. Mike DeWine to pause new applications for the sales-tax exemption in May.

Now, some state lawmakers, including Democratic Rep. Tristan Rader, want to repeal the sales-tax exemption and renegotiate past deals with companies such as Amazon.com , Meta Platforms and Alphabet’s Google that secured exemptions for decades by signing contracts with the state. In several states, the exemptions for tech giants have exceeded $1 billion annually.

“They seem to have more money than God and they’re able to build without the need for these types of incentives,” said Rader, who is proposing new data-center taxes and requirements that developers pay more for power and electrical infrastructure. He represents parts of Cleveland, where locals have pushed back against the facilities, and he hopes the new threats force the companies to the negotiating table.

A similar dynamic is playing out across the country, as legislators or governors in more than 10 states, including Illinois, New Jersey and Washington, have slammed the brakes on the tax breaks. The reversals underscore how state leaders who once welcomed the facilities powering the AI boom as an economic engine suddenly treat them like a political liability.

Dozens of other states including Texas have proposed similar bills or steps, potentially costing the industry billions in tax benefits. That has the potential to push future projects to states such as Indiana, West Virginia and Wyoming that have retained favorable tax regimes, industry executives and advisers say. Industry advocates are telling state officials that other taxes they pay fund local schools and police departments and that eliminating incentives will force data centers to move elsewhere.

Comment