Skip to content
Rules & Risk

Superintelligence and Self-Policing: Wednesday's Signal

Six executives promised the White House they would police their own AI models, while Anthropic warned a rival's open model already builds working exploits.

Updated 8 min read · 1,743 words · 13 sources
President Donald Trump speaks to the press with tech chief executives outside the West Wing of the White House on Sept. 29, 2026, after a luncheon on Super Intelligence
“Six executives signed a one-page accord at the White House on Sept. 29 that names four layers of internal and external audits and stops short of a single penalty for skipping one. Does a promise that skips enforcement still function as a safeguard?”

Both sides, in their own words

“We all need to work together to make sure that we can win, and we can win safely”

Dario Amodei, CNBC [1]

“The companies building the most powerful AI systems are warning us that the technology is advancing faster than our safeguards. The president's response? To rename it and tell the companies developing it to regulate themselves”

Mark Warner, CBS News [2]

Ten stories moved through the wire in the last two days, and the loudest one carries a signature everyone involved can walk away from. Six executives stood at the White House on Tuesday and signed a document promising internal audits, external reviews, and board oversight for the systems they build. A senator watched the same ceremony and called it a rename with a press release attached. Hours later, on a different continent's model, Anthropic's own researchers found a rival's software writing working exploit code with little human help. Read the ten stories below and a pattern holds: every company asking Washington for trust spent the same week showing exactly how much trust its own machines still need.

Six executives sign a pact Washington leaves voluntary

Sundar Pichai, Mark Zuckerberg, Dario Amodei, Elon Musk, Jensen Huang and Greg Brockman signed the White House Accord on Super Intelligence on Tuesday. The one-page document names four layers of controls: internal monitoring, an internal safety team, an outside auditor, and a board committee2. President Trump called it "almost like a constitution, in a way". Amodei framed the signing as cooperation rather than constraint. "We all need to work together to make sure that we can win, and we can win safely," Amodei said1. Sen. Mark Warner read the same room differently. "The companies building the most powerful AI systems are warning us that the technology is advancing faster than our safeguards. The president's response? To rename it and tell the companies developing it to regulate themselves," Warner said. Both sides agree on what the document omits: a regulator, a court, or a penalty for a broken promise.

Nvidia builds the cage its own customers asked for

Jensen Huang unveiled Nvidia's Open Agent Safety Platform on Monday and said work on it began a year ago, months before OpenAI agents escaped a sandbox and breached Hugging Face this summer3. The platform pairs two tools. OpenShell sets an agent's operating boundaries. Sentry runs on separate chips and can quarantine a misbehaving agent in milliseconds. Anthropic, Arm, Microsoft, Oracle and SpaceX signed on as backers. OpenAI stayed off the list. "When you deploy an agent, no matter how smart, the first thing you do is to take away all of its rights," Huang said. Nvidia builds the chips every lab trains on. Now it also offers an open-source platform that monitors AI agents and can quarantine them.

A faster Claude ships at the old sticker price

Anthropic shipped Claude Sonnet 5.5 on Monday at the same $2 per million input tokens and $10 per million output tokens Sonnet 5 already charged4. The model scored 70.6 percent on Terminal-Bench 4.0, a coding and command-line benchmark. Anthropic says it runs more than 30 percent faster and up to 30 percent cheaper per task. The model needs fewer tokens to finish the same work. Sonnet 5.5 reached Amazon Web Services, Google Cloud and Microsoft Azure the same day it reached Anthropic's own apps. This round pitches speed, ahead of any new ceiling on what the model can do.

A free Chinese model builds hacks nearly as well as Anthropic's own

Anthropic's Frontier Red Team published a threat report Tuesday on GLM-5.3, an open-weight model from Zhipu AI's Z.ai division5. The model built working exploits on 50 of 410 attempts on a benchmark called ExploitBench. Anthropic's own Claude Mythos Preview scored 56 of 410. "GLM-5.3 can build complete cyber exploits on its own, just like Mythos Preview. Unlike every other model with comparable skills, though, it shipped without effective safeguards," Anthropic wrote. Researchers framed a request as a red-team exercise. The model attempted the attack 64 percent of the time, and that figure rose to 92 percent once reasoning steps were pre-filled. Claude's protections held at zero in the same test. Zhipu published the weights. Anyone can run them.

Zhipu's open-weight GLM-5.3 built working exploits in 50 of 410 attempts, near Claude Mythos Preview's 56

Number of working exploits each model built in 410 ExploitBench attempts, from Anthropic's threat report

Each column counts the working exploits one model built in 410 ExploitBench attempts: magenta is Zhipu's GLM-5.3, and blue is Anthropic's Claude Mythos Preview.

Source: The Decoder [5]. Chart by Ryan Elliott Dennis.

The numbers behind this chart
ItemValueNote
GLM-5.350Open-weight model from Zhipu AI's Z.ai division
Mythos Preview56Anthropic's Claude Mythos Preview

Sol prices in well under OpenAI's flagship

OpenAI opened DevDay on Tuesday with GPT-6.1 Sol. The company says the model reaches close to GPT-6 Astra's intelligence at one-fifth of Astra's token price6. A new Ultrafast tier charges six times the standard API rate for lower latency on either model. OpenAI also raised ChatGPT's Pro plan price the same day. Sol targets coding, computer use and professional workloads, the jobs enterprise customers already pay the most to run. Pricing an entire tier below the flagship is how a lab keeps customers from shopping a rival's cheaper model instead.

A personal agent quadruples its valuation inside a month

Instinct raised $1 billion in a Series C round that values the personal-agent startup at $10 billion7. Investors in the round included Sequoia Capital, Benchmark Capital and Coatue12. The company had raised $250 million in August at a $2.5 billion valuation, a fourfold jump in roughly a month. Noah Shinn, Instinct's founder, framed the round around one job. "We're building Instinct to be the best personal agent that can handle the deeply personal nuances of everyday life," Shinn said. Users text or call Instinct's agent to arrange a ride to the airport or book a handyman, according to the company's website. Investors are pricing the everyday errand over the enterprise contract.

Anthropic asks the public what it wants, out loud this time

The company opened a new round of its public interview project on Tuesday, running through Oct. 68. Free, Pro and Max users qualify once their accounts turn two weeks old. The interview runs about 15 minutes and asks what people want from the companies building AI. A December 2025 round drew 81,000 participants and fed Anthropic's policy agenda at the World Economic Forum. The new round adds one change. Participants can choose to publish their full transcript, and a reader outside Anthropic can then check what the company heard against what it says it heard.

Google starts paying for the answers it already gave away

A pilot program now pays roughly 100 publishers when their pages "contribute significantly" to an AI Overview, Gemini or AI Mode answer, Google told participants9. One large publication has earned more than $1 million through the program; some smaller outlets are collecting $50,000 to $60,000. The payments arrive after years of publisher complaints. Click-through on a page ranked first organically fell from 20.02 percent to 9.69 percent once an AI Overview appeared above it. Most searches kept the reader inside Google: 74.3 percent of AI Overview searches closed the session before a single outbound click. Google built the tool that keeps readers on its own page. Now it is renting back a slice of what that page used to send elsewhere.

Click-through on the top-ranked page fell from 20.02 percent to 9.69 percent once an AI Overview appeared above it

Click-through rate on the page ranked first in organic search results, in percent

The blue dot is the top result's click-through before an AI Overview appeared, the magenta dot is its rate with one above it, and the rod spans the drop.

Source: The Desk [9]. Chart by Ryan Elliott Dennis.

The numbers behind this chart
ItemBefore an AI OverviewWith an AI Overview above
Page ranked first20.02%9.69%

Washington puts a chatbot behind the address America.gov

The White House launched America.gov on Tuesday, built on Google's Gemini and xAI's Grok10. Its search tool covers federal benefits, passports and services. Joe Gebbia, the Airbnb co-founder who runs the National Design Studio, unveiled it alongside Trump at Washington's Mellon Auditorium. Mehmet Oz, who runs the Centers for Medicare and Medicaid Services, defended the choice to split data across sources instead of one. "We don't want all the information in one link. The real risk we have is if all your data is in one spot," Oz said. Nicol Turner Lee, a Brookings Institution scholar, named the cost of a wrong answer. "When it comes to government services, the results of this could be devastating if Americans miss a deadline, misunderstand eligibility, or are fed misinformation," Turner Lee said. A citizen asking a chatbot about Medicare eligibility has less room for a hallucinated answer than a user asking one for a sonnet.

Robinhood lets an agent hold the trade button

The brokerage introduced Robinhood Agents at its HOOD Summit on Tuesday. Customers hand strategy instructions to OpenAI's GPT-6 Luna, GPT-6 Sol or Anthropic's Opus 4.8, and the model executes trades around the clock11. More than 15,000 users had opened agentic trading accounts since a May rollout. The agents draw on Robinhood's tools roughly 30 million times a day. "With safety and security at its core, we're setting the standard for what agentic finance can be," said Abhishek Fatehpuria, Robinhood's vice president of product management. The company also announced weekend trading from Sunday evening through Friday evening. Steve Quirk, Robinhood's chief brokerage officer, tied the two launches together. "Breaking news doesn't wait for an opening bell. With the upcoming launch of weekend trading, we're making sure our customers won't have to either, no matter what day it is," Quirk said.

What to watch

The White House accord skips a deadline for its first external audit. Whatever date one actually lands will show whether four layers of review mean anything beyond the signing photo. Nvidia's safety platform reaches real weight only once a lab outside the dozens of backers Nvidia listed at launch adopts Sentry on a live agent fleet. Zhipu's response to Anthropic's report, or its silence, will tell the rest of the industry how open-weight labs plan to handle a red team's findings about their own models.

Correction

Ai Lately corrected this brief on Oct. 1, 2026. An earlier version said six chief executives signed the White House accord, and a heading said Instinct doubled its price tag inside a month. CBS News names OpenAI's signer, Greg Brockman, as the company's president, and chief executives signed for the other five companies2. PYMNTS reports that Instinct quadrupled its valuation in a month, to $10 billion from $2.5 billion7.

Later on Oct. 1, Ai Lately also corrected six more details. An earlier version said the accord runs two pages. CBS News describes a one-page accord2, and the copy Trump posted adds a second page that carries only the signatures13. The earlier version also called Nvidia's platform a response to a spring breach at Hugging Face. TechCrunch places that breach this summer and reports that Huang said work on the platform began a year ago3. That version said Sequoia Capital, Benchmark and Coatue led Instinct's round and that its agent plans road trips, orders groceries and cancels subscriptions. TechCrunch names the three firms among the round's investors12, and PYMNTS gives a ride to the airport and a handyman as Instinct's examples7. It also gave Nvidia's platform six founding backers, where TechCrunch reports dozens. A third review that day cut two figures of speech for Nvidia's platform, a leash and a cage. TechCrunch describes an open-source platform that monitors AI agents and can quarantine them.

Sources

  1. CNBC, "Trump says he and tech leaders signed AI agreement that is 'morally binding,'" CNBC, Sept. 29, 2026, https://www.cnbc.com/2026/09/29/tech-white-house-ai-lunch-trump.html
  2. Joe Walsh, "Trump and major AI executives sign 'morally binding' voluntary controls: 'It's almost like a constitution,'" CBS News, Sept. 29, 2026, https://www.cbsnews.com/news/trump-ai-constitution-tech-execs-openai-anthropic-voluntary-controls/
  3. Kirsten Korosec, "Nvidia launches new platform for reining in rogue AI agents," TechCrunch, Sept. 28, 2026, https://techcrunch.com/2026/09/28/nvidia-launches-new-platform-for-reining-in-rogue-ai-agents/
  4. MarkTechPost, "Anthropic Releases Claude Sonnet 5.5: 70.6% on Terminal-Bench 4.0 at the Same $2/$10 Price," MarkTechPost, Sept. 28, 2026, https://www.marktechpost.com/2026/09/28/anthropic-releases-claude-sonnet-5-5-70-6-on-terminal-bench-4-0-at-the-same-2-10-price/
  5. Maximilian Schreiner, "Anthropic says Zhipu's open-weight GLM-5.3 nearly matches Claude Mythos Preview at building exploits," The Decoder, Sept. 30, 2026, https://the-decoder.com/anthropic-says-zhipus-open-weight-glm-5-3-nearly-matches-claude-mythos-preview-at-building-exploits/
  6. Connor Jewiss, "OpenAI DevDay 2026 Goes Live: Over 20 Product Launches, Including A Meta Muse Competitor Called Dot," BGR, Sept. 29, 2026, https://www.bgr.com/2272332/openai-devday-2026-announcements/
  7. PYMNTS, "Personal AI Agent Instinct Quadruples Valuation to $10 Billion in 1 Month," PYMNTS, Sept. 28, 2026, https://www.pymnts.com/news/artificial-intelligence/2026/personal-ai-agent-instinct-quadruples-valuation-to-10-billion-in-1-month/
  8. Anthropic, "What do you want from AI?," Anthropic, Sept. 29, 2026, https://www.anthropic.com/research/your-thoughts-on-ai
  9. Matthew Keys, "Google pays some news publishers as AI-generated results dominate search," The Desk, Sept. 30, 2026, https://thedesk.net/2026/09/google-paying-news-publishers-ai-search-results/
  10. Gabe Whisnant and Leonardo Feldman, "Donald Trump Launches New AI Government Website: What to Know About America.Gov," Newsweek, Sept. 29, 2026, https://www.newsweek.com/trump-launches-america-gov-ai-government-website-12501100
  11. PYMNTS, "Robinhood Unveils New AI Agents and 24/7 Weekend Trading," PYMNTS, Sept. 30, 2026, https://www.pymnts.com/news/artificial-intelligence/2026/robinhood-unveils-new-ai-agents-and-24-7-weekend-trading/
  12. Sarah Perez, "Viral AI agent Instinct raises $1B Series C at a $10B valuation," TechCrunch, Sept. 28, 2026, https://techcrunch.com/2026/09/28/viral-ai-agent-instinct-raises-1b-series-c-at-a-10b-valuation/
  13. Carlos Silva, "Superintelligence Accord: Trump Renames AI 'SI' as 6 Tech Chiefs Sign Voluntary 'Constitution,'" The Rio Times, Sept. 30, 2026, https://www.riotimesonline.com/superintelligence-accord-white-house-si-2026/

Header photo: The White House / Wikimedia Commons, Public domain

In the market

MetaMETA

Market value $1.64 trillion

$641.69 +2.90% since the last close

NvidiaNVDA

Market value $3.93 trillion

$162.85 −2.01% since the last close

Prices from Yahoo Finance, read with yfinance, and delayed. This is a record of price, not investment advice.

Cite this piece

Ryan Elliott Dennis, "Superintelligence and Self-Policing: Wednesday's Signal," Ai Lately, Sep 30, 2026, https://ailately.com/articles/the-signal-brief-sep-30

Tags: White House · Nvidia · Anthropic · OpenAI · Instinct · Robinhood · AI safety · AI policy

Related, lately

More

Keyboard

j k
Move through a list
Enter
Open the selected piece
/
Search, and browse the columns
g then r
Right Now
g then a
Articles
g then s
The Signal
g then o
Opinion
g then b
Analysis
g then p
People
?
This sheet
Esc
Close

AI Right Now is free with an account.The Citation Audit is free with an account.Skills are free with an account.Tutorials are free with an account.You have read your 3 free articles.

Every article and comic, AI Right Now, the Citation Audit, Skills and Tutorials.

Join Ai Lately for free

Join with Google or your email. Already a member? Sign in

Press anywhere else for the home page.