AI Weekly

The field of Artificial Intelligence is rapidly evolving, and with that comes a growing focus on safety. This week's newsletter dives deep into several key developments, including:

Top Stories:

• OpenAI Exodus Raises Safety Concerns: Ex-researchers say safety was deprioritized, leading to departures and a disbanded team. OpenAI denies these claims. (Read More).
• Colorado Enacts First US AI Law: The law focuses on preventing algorithmic discrimination and requires developers to take steps to mitigate risk. (Read More).
• International Safety Report: No Guarantees on Safe AI: A new report finds current methods are insufficient to ensure the safety of advanced AI systems. (Read More).
• Microsoft Unveils Privacy-Raising AI Laptops: New "Copilot+ PCs" raise concerns about on-device AI models constantly taking screenshots and storing user data. (Read More).

Other News:

• The FTC Takes Aim at AI: The Federal Trade Commission outlines eight areas of interest regarding AI, focusing on issues like fraud, competition, and data privacy. (Read More)
• Scarlett Johansson "Shocked" by GPT-4o Voice: OpenAI removes voice model resembling Johansson after actress expresses outrage. (Read More)
• Google Unveils New Safety Framework: Google outlines its "Frontier Safety Framework" for identifying and mitigating risks associated with AI models. (Read More)
• EU AI Act Gets Green Light: The European Union's AI Act is officially approved and will begin entering into force over the next two years. (Read More)
• Big Tech Giants Make AI Safety Agreement: Major tech companies commit to not deploying AI models with unmitigable risks. Experts remain skeptical. (Read More)
• Ads Coming to Google's AI Overviews: Google plans to introduce ads within its new AI-powered search results, pushing organic links down further. (Read More)
• Geoffrey Hinton Says AI Can Have Feelings: Leading AI researcher faces criticism for suggesting robots can experience emotions. (Read More)
• Anthropic Cracks Open the "Black Box" of AI: Researchers achieve some level of interpretability within an AI model using "dictionary learning." (Read More)
• LLMs and the Language of (Un)Intelligence: Experts remind us that fluent language use doesn't equate to actual understanding in large language models. (Read More)
• OpenAI CEO Denies Knowledge of Equity Clause: Documents contradict Altman's claim of ignorance regarding non-disparagement agreements for departing employees. (Read More)

Stay Informed, Stay Safe

Lets Dive Deeper

AI Safety and Priorities at OpenAI

Jan Leike says AI safety wasn’t a priority at OpenAI

In light of the recent departure of several safety researchers from OpenAI, Jan Leike, the former Superalignment co-lead, shared his reasons for leaving in a viral thread. He expressed a growing disagreement with the company’s leadership over its core priorities.

• “Over the past few months my team has been sailing against the wind,” Leike wrote. “Sometimes we were struggling for compute and it was getting harder and harder to get this crucial research done.”
• “Over the past years, safety culture and processes have taken a backseat to shiny products.”

Following these events, OpenAI dissolved its Superalignment team entirely. Reports of a culture of broken trust regarding safety efforts at OpenAI have surfaced, with former employees reluctant to share their names due to a non-disparagement agreement tied to their equity. Sam Altman claimed he was unaware of this agreement.

Regulatory Developments in AI

The FTC has a few questions about AI

The Federal Trade Commission’s Office of Technology (OT) highlighted eight “questions of interest” regarding AI last week. The OT aims to learn more about these specific areas:

1. AI & Fraud: How AI enhances fraud through speed, scale, and personalization, and the effectiveness of current safeguards.
2. AI & Competition: How firms with market power leverage it in the AI sector.
3. Algorithmic Pricing: The use of AI in price fixing.
4. Surveillance & Data Privacy: The scope and scale of first-party data collection.
5. Data Security: Common security vulnerabilities within LLMs.
6. Open Models: The impact of open-weight models on competition.
7. Platform Design: How social media and gaming platform design choices affect usage and mental health.
8. Digital Capacity: Developing tech skills that translate to tech policy.

Colorado enacts first piece of U.S. AI legislation

Colorado Governor Jared Polis signed the Consumer Protections for AI bill into law, marking the first comprehensive AI legislation in the U.S. It will take effect in February 2026 and focuses on preventing discrimination by targeting developers of high-risk systems.

Key Points:

• Developers and deployers must use “reasonable care” to protect consumers from foreseeable risks of algorithmic discrimination.
• High-risk system developers must provide disclosures about the models, including training data and potential biases.
• Deployers must conduct regular impact assessments.

The Colorado Tech Association and Governor Polis have expressed concerns about the legislation’s impact on the tech industry.

The EU AI Act gets its official green light

The European Union’s AI Act, the world’s first piece of AI legislation, was officially approved by the EU Council. The Act takes a risk-based approach, imposing stricter regulatory and transparency requirements on high-risk AI models.

Implementation timeline:

• The Act will be published in the EU’s official journal.
• 6 months after publication: The ban on prohibited systems takes effect.
• 1 year after publication: Rules regarding general-purpose system governance take effect.
• 2 years after publication: The entire Act takes effect.

AI Developments and Controversies

Cognitive Resonance founder demonstrates an LLM reasoning test

Benjamin Riley, founder of Cognitive Resonance, demonstrated a novel test designed to evaluate reasoning capabilities in LLMs. Most LLMs, including GPT-4o, failed the test, which requires logical inference beyond text prediction.

Scarlett Johansson ‘shocked’ by GPT-4o voice

OpenAI unveiled GPT-4o, a multimodal model with a voice similar to Scarlett Johansson’s character in the movie Her. Johansson expressed shock and anger upon hearing the demo, claiming that Altman had previously asked her to be the voice of ChatGPT, an offer she declined. OpenAI has since removed the voice and stated it was never intended to resemble Johansson’s.

Google unveils new safety plans

Google published a Frontier Safety Framework, addressing potential risks from future AI advancements. The framework focuses on identifying critical capabilities, evaluating models, and applying mitigation strategies. However, it stops short of removing public access to models that display harmful capabilities.

AI Industry News

Microsoft combines AI + PCs in ‘privacy-invading product’

At its Build conference, Microsoft announced Copilot+ PCs with a feature called “Recall,” which constantly takes screenshots of user activity and uses an on-device AI model to make the data searchable. Privacy concerns have been raised about the storage and potential misuse of this information.

Big Tech giants make safety agreement at Seoul Safety Summit

Sixteen prominent AI companies, including OpenAI, Microsoft, and Google, made a safety commitment at the Seoul Safety Summit. They agreed not to develop or deploy AI models if the risks cannot be “sufficiently mitigated” and to publish individual safety frameworks.

Ads are coming to Google’s AI overviews, and organic search will pay the price

Google announced it will start testing Search and Shopping ads within AI Overviews, which are integrated into search results. This move has sparked criticism for further diminishing the visibility of organic links.

Expert Opinions and Reports

Geoffrey Hinton says AI can have feelings; experts disagree

Geoffrey Hinton, known for his work on neural networks, claimed that robots can express emotions. However, experts argue that feelings and emotions are complex and that attributing human emotions to machines is a form of projection.

Anthropic cracks open the ‘Black Box’ of AI

Anthropic researchers used “dictionary learning” to extract features from an AI model, providing a conceptual map of its internal states. This research aims to better understand how AI models work and ensure their safety.

International AI Safety Report: There’s no current way to ensure that genAI is safe

The 130-page interim “International Scientific Report on the Safety of Advanced AI” from the Seoul Safety Summit highlights that scientists currently lack the methods to ensure general-purpose AI systems are safe in all circumstances. The report calls for more human involvement in safety loops to address misuse and abuse risks.

Altman says he didn’t know about OpenAI’s equity clause — documents show a different story

Despite Sam Altman’s claims of ignorance, documents obtained by Vox reveal that OpenAI’s leadership, including Altman, was aware of non-disparagement agreements tied to employee equity. OpenAI has since removed these clauses and released former employees from them.

LLMs and the language of (un)intelligence

LLMs’ ability to produce coherent text leads some to mistakenly attribute understanding to these models. However, experts like Jacob Browning and Yann LeCun argue that language output does not equate to genuine comprehension. Instances like Google’s AI Overview suggesting non-toxic glue in pizza sauce highlight the limitations of LLMs.

That’s a wrap

This newsletter is your one-stop shop for staying informed about the latest developments and ongoing debates in AI safety. We'll continue to bring you critical news and analysis to help us navigate the future of artificial intelligence responsibly.