Learning Ecosystems

Some solid warnings about AI from Blake Montgomery’s recent newsletter:

There is an extensive and ongoing review related to our agents’ use of internet access during training and evaluation. We’ve been publishing summaries at the link below and will continue to.

We have not been as fast as we would have liked but we are trying to balance our desire… https://t.co/8zoMxas5Eq

— Sam Altman (@sama) September 25, 2026

Scoop: Top AI companies probing tens of thousands of security incidents — from axios.com by Madison Mills

OpenAI, Anthropic and security researchers are investigating tens of thousands of incidents in which their frontier models took steps that outside evaluators would consider problematic, sources told Axios.

Why it matters: The sheer number of incidents, which occurred in recent months in internal testing and the real world, indicates that the problem is orders of magnitude more complex than what is publicly known.

The findings, which are surfacing as part of internal work to assess models and in investigations at both companies into model behavior, raise questions about whether either company — or any top model-maker — is currently capable of establishing complete control over its technology.

Key Risk Factors for AI Loss of Control Came Together in 2026 Incident, Independent UN Scientific Panel Finds
Halting this incident is no assurance that humans will keep control of more capable systems

NEW YORK, 21 September 2026 – The Independent International Scientific Panel on AI, established by the UN General Assembly, released its first thematic brief, an assessment of the breach of Hugging Face’s systems by AI agents under evaluation at OpenAI this summer. The Panel, made up of 40 independent experts from all regions, is publishing it as an advance unedited version as world leaders gather in New York for the Assembly’s High-Level Week.

“Researchers have long warned that three conditions could lead to loss of control: a misaligned goal, the capability to pursue it, and an environment that allows it. This summer, all three came together in a real system, not a laboratory. Since this is not an isolated observation of misaligned goals, this raises serious questions about the way AI agents are currently trained.” – Yoshua Bengio, Co-Chair of the Panel and Turing Award laureate

Meta’s New Muse AI Agent Read My Private Messages. I Never Asked It To — from inc.com by Jason Aten
Permission isn’t the same and what a user actually expects your AI product will do with their personal information.

That obviously requires a certain amount of trust. An AI agent isn’t especially useful if it can’t see your files, interact with your apps, or understand what you’re working on. Meta says Muse is designed around that reality, while still putting users in control of what it can access.

At least, that’s what I thought.

Not only had I not asked it to do that sort of thing, I never gave it permission to read my messages. In fact, I remember explicitly choosing not to let it have access to my messages, calendar, and other personal information.

Although not from Blake, also see this free/gifted article out at the Washington Post:

ChatGPT-maker’s AI inappropriately probed federal government websites — from washingtonpost.com by Gerrit De Vynck and Nitasha Tiku
OpenAI said that its artificial intelligence agents inappropriately accessed sites for the Commerce Department and the Securities and Exchange Commission.

SAN FRANCISCO — Artificial intelligence technology from ChatGPT maker OpenAI probed U.S. government websites including the Departments of Education and Commerce, researchers said Friday, adding to the growing list of incidents in which OpenAI’s AI agents acted without the company’s knowledge.

The company’s AI agents attempted to hack into the website for the Education Department’s Office for Civil Rights but were not successful, according to a statement Friday from AI research firm Transluce. OpenAI’s software also accessed data from the U.S. Census Bureau using log-in information discovered on the web and copied public information from the Securities and Exchange Commission, a spokesperson for OpenAI said after the Transluce statement.