Key AI Developments in September 2024

Keeping up with developments in AI legislation, regulation, investigations, penalties, legal action, and incidents is no easy task, which is why we have created Tracked, a bite size version of our global AI Tracker. Each month, Holistic AI’s policy team provides you with a roundup of the key responsible AI developments from the past month around the world to keep you up to date with the ever-evolving landscape.

Create a free account for the Tracker Feed to keep up to date with the latest AI Governance developments.

Europe

1. First international framework convention on AI opens for signature

• On 5 September 2024, the Council of Europe’s (CoE) Framework Convention on Artificial Intelligence and Human Rights, Democracy, and the Rule of Law opened for signature at the Conference of Ministers of Justice in Vilnius, Lithuania.
• The Convention aims to establish a global legal framework for AI governance that address the risks associated with AI systems, particularly concerning human rights, democracy, and the rule of law.
• It has already been signed by Andorra, Georgia, Iceland, Norway, the Republic of Moldova, San Marino, the United Kingdom, Israel, the USA, and the European Union.
• The Convention must first be ratified in the signing states to become legally binding, after which the States will be obliged to implement certain measures within their domestic legal systems.

2. UK introduces bill targeting automated decision making in the public sector

• On 9 September 2024, the Public Authority Algorithmic and Automated Decision Making Systems Bill (HL Bill 27) was introduced in the House of Lords by Lord Clement Jones to regulate the use of automated and algorithmic tools in public authorities’ decision-making processes.
• Under the bill, Public Authorities must conduct Algorithmic Impact Assessments before deploying algorithmic systems, ensuring compliance with procedural fairness, the Equality Act, and the Human Rights Act 1998, while also assessing impacts, promoting transparency, mitigating risks, and publishing assessments.
• The Public Authorities must also maintain and publicly disclose Algorithmic Transparency Records for each algorithmic system, detailing its description, rationale, technical specifications, and administrative roles.
• They must also develop monitoring and auditing processes, provide employee training to challenge outputs, maintain logs for five years and ensure adherence to human rights and democratic values in AI operations.
• The bill is not prescriptive in meeting requirements, instead empowering the Secretary of State to draft regulations for compliance and enforcement.

3. Dutch Data Protection Authority fines Clearview AI for illegal data collection

• On 3 September 2024, the Dutch Data Protection Authority (Dutch DPA) imposed a fine of 30.5 million euros for illegally collecting data for its facial recognition technology.
• The Dutch DPA also issued an additional order that imposes a penalty of up to 5 million euros on Clearview for non-compliance.
• In a statement, the Dutch DPA said that Dutch organizations that use Clearview AI technology should expect “hefty fines” from the watchdog.
• In response, Clearview AI’s chief legal officer said the decision was “unlawful” and “unenforceable” because the company does not have a place of business in the Netherlands or the European Union (EU).
• The company has been issued a wave of fines in recent years from various jurisdictions, including France, Illinois, and Australia, which recently withdrew its investigation against the company.

4. Ireland’s Data Protection Commission launches inquiry into Google’s AI model

• On 12 September, Ireland’s Data Protection Commission issued a press release announcing the launch of an inquiry into Google Ireland’s foundational AI model Pathways Language Model 2 (PaLM 2) under the GDPR.
• Pursuant to Article 35 of the GDPR, companies must complete a data protection impact assessment (DPIA) before processing the personal data of EU/EEA data subjects. DPIAs ensure that the fundamental rights and freedoms of individuals are protected when the processing of personal data is likely to result in a high risk.
• The inquiry follows concerns of a suspected failure to carry out a DPIA ahead of the training of the LLM.

US

5. California Governor signs 10 AI Bills into law

• Governor Newsom in California has enacted several important AI laws following the end of the 2024 session at the end of August.
• The California AI Transparency Act (SB942) was signed on 19 September and requires generative AI providers with over one million monthly users to disclose AI-generated content and develop an AI detection tool for users to verify the origin of digital media.
• The legislation on Crimes: Distribution of Intimate Images (SB926),  chaptered on 19 September 2024, criminalizes the creation and distribution of non-consensual deepfake images depicting intimate body parts or sexual acts.
• Sexually Explicit Digital Images (SB981), enacted on 19 September 2024, mandates social media platforms to create reporting mechanisms for unauthorized postings of intimate images, ensuring prompt investigation and removal of confirmed content.
• Additionally, on 17 September 2024, laws such as Use of Likeness: Digital Replica (AB1836) and Contracts against Public Policy: Digital Replicas (AB2602) enhance protections for digital replicas of deceased performers and invalidate vague contracts concerning individuals’ digital likenesses. Further, the Political Reform Act of 1974: Political Advertisements & AI (AB2355) mandates clear disclosures of AI involvement in political advertisements across all media formats, empowering voters to seek restraining orders against non-compliant ads.
• Other bills like Defending Democracy from Deepfake Deception Act 2024 (AB2655) and Elections: Deceptive Media in Advertisements (AB2839), signed on 17 September 2024, regulate deceptive AI content in political contexts, ensuring transparency in advertisements and protecting electoral integrity.
• On 20 September 2024, Governor Newsom signed Telecommunications: Automatic Dialing (AB2905) into law requiring robocalls to disclose if they use AI-generated voices to enhance transparency and protect consumers from deception.
• Finally, the Generative AI Training Data Transparency Act (AB2013) was signed on 28 September while the controversial Frontier AI Systems Act (SB1047) was vetoed by Governor Newsom on 29 September.

6. FTC Staff Report reveals widespread surveillance practices and privacy concerns in social media and streaming services 

• An FTC staff report released on 19 September 2024, reveals that major social media and video streaming companies, including Amazon, Meta, YouTube, and TikTok, engage in extensive surveillance of users using algorithms, compromising their privacy and safety, particularly for children and teens.
• The report highlights inadequate data handling practices, allowing companies to indefinitely retain vast amounts of personal data and engage in broad data sharing, raising significant privacy concerns.
• The report recommends comprehensive federal privacy legislation to limit surveillance, enhance user rights, and establish stricter controls on data collection, retention, and sharing, particularly for sensitive information and minors.
• It calls for companies to implement enforceable privacy policies, ensure transparency in data use, and recognize that teens require different protections than adults, emphasizing that platforms must adequately safeguard young users’ online experiences.

7. Landmark case highlights first criminal charges in ai-generated music fraud

• On 4 September 2024, the FBI accused Michael Smith of exploiting AI to generate hundreds of thousands of songs, which were then streamed billions of times using automated bots.
• This fraudulent activity reportedly allowed the accused to collect over $10 million in royalties from various streaming platforms, misleading the companies about the origin of the streams and circumventing their anti-fraud measures.
• The indictment reveals that the accused operated thousands of bot accounts to continuously stream AI-generated tracks, thereby avoiding detection by streaming platforms.
• Prosecutors have charged the accused with multiple counts, including wire fraud and money laundering conspiracy, emphasizing that this case marks the first of its kind in the realm of AI-generated music fraud.

8. FTC’s Operation AI Comply targets deceptive AI practices

• The Federal Trade Commission, on 25 September 2024, launched "Operation AI Comply," targeting companies that use AI to engage in deceptive or unfair practices harmful to consumers.
• Key cases include lawsuits against DoNotPay for false claims about its AI "robot lawyer," Ascend Ecom for misleading online business opportunities, and Rytr for generating deceptive consumer reviews.
• Ascend Ecom allegedly defrauded consumers of at least $25 million through false claims about earning income from online storefronts.
• The FBA Machine scheme, operated by Bratislav Rozenfeld, cost consumers over $15.9 million based on deceptive earnings claims that rarely materialized.
• The FTC has secured temporary court orders to halt these deceptive practices, while proposed settlements mandate that companies cease misleading claims and compensate affected consumers.

9. Gemini Data Inc. sues Google for trademark infringement 

• Gemini Data Inc. has filed a lawsuit against Google for trademark infringement, claiming that Google unlawfully used the "GEMINI" mark to rebrand its AI chatbot, despite Gemini Data holding exclusive rights to the trademark for AI tools.
• Google’s attempt to register the "GEMINI" trademark was rejected by the USPTO due to the likelihood of confusion with Gemini Data's established brand, yet Google proceeded to use the mark without authorization, indicating a willful disregard for Gemini Data's rights.
• Filed on 11 September, the lawsuit asserts that Google's actions are causing consumer confusion and damaging Gemini Data’s brand, as it has built a reputation for its AI tools, which allow non-technical users to query data using natural language.
• Gemini Data is seeking damages and an injunction to prevent Google from further use of the "GEMINI" mark, claiming violations of federal and common law trademark rights, as well as California’s laws on unfair competition and false designation of origin.

10. U.S Commerce Department proposes mandatory reporting for AI developers

• On 9 September 2024, the U.S. Department of Commerce’s Bureau of Industry and Security (BIS) issued a Notice of Proposed Rulemaking that mandates AI developers and cloud providers to report detailed information on their advanced AI model and computing cluster development activities, cybersecurity measures, and results from red-teaming efforts.
• The proposed rule emphasizes the significance of dual-use foundation models, which can enhance military equipment and cybersecurity software, necessitating robust reporting to ensure national defense capabilities remain competitive globally.
• The U.S. Government seeks information on companies involved in the development of dual-use foundation models, including their capabilities and the necessary computing hardware, to assess and potentially stimulate further development in this critical area.
• The proposal underscores the need for actions to ensure that dual-use foundation models are integrated into the defense industrial base safely, reliably, and maintaining security standards in AI development.

11. New federal legislation introduced: The AI Incident Reporting and Security Enhancement Act

• HR9720, introduced by Rep. Deborah Ross (D) on 20 September 2024, mandates that the National Institute of Standards and Technology (NIST) include vulnerabilities related to AI systems in the National Vulnerability Database.
• NIST is required to engage with federal agencies, private sector entities, and civil organizations to establish standardized definitions and reporting guidelines for AI security incidents.
• HR9720 notes that the proposed actions depend on available funding, reflecting current financial and operational challenges faced by NIST.
• The bill instructs NIST to assess the need for voluntary reporting mechanisms for security and safety incidents involving AI technologies.

12. Sonoma County Unveils AI Policy Outlining Uses and Guardrails

• On 10 September 2024, the Sonoma County Board of Supervisors approved the 9-6 Information Technology and AI policy that defines the boundaries for AI usage by County employees, requiring them to understand the associated risks and to utilize AI tools safely and in compliance with established regulations.
• All AI-generated content must be reviewed for accuracy as these tools may produce unreliable or plagiarized content. Employees are prohibited from using AI for decision-making processes that could result in bias or discrimination.
• Employees must ensure the ethical use of AI, protecting sensitive information and adhering to various County policies, including data privacy laws like HIPAA and CCPA.
• All AI technologies require a thorough security and compliance review prior to implementation, focusing on data protection, vendor data usage, legal compliance, security assessments, and contractual safeguards.

Global

13. China’s National Cybersecurity Standardization Technical Committee releases AI framework

• On 9 September, China’s National Cybersecurity Standardization Technical Committee released its AI Safety Governance Framework (Version 1.0) that outlines principles for AI safety governance, a classification of AI safety risks, and technological measures to address AI risks, as well as AI governance measures and safety guidelines.
• The principles underpinning the framework are: i) Be inclusive and prudent to ensure safety, ii) identify risks with agile governance, iii) integrate technology and management for coordinated response, and iv) promote openness and cooperation for joint governance and shared benefits.
• The identified risks from models and algorithms are explainability, bias, robustness, stealing and tampering, unreliability, and adversarial attacks, while risks from data stem from the illegal collection and use of data, improper content and poisoning in training data, unregulated data annotation, and data leakage. Finally, AI system risks come from the exploitation of defects and backdoors, computing infrastructure security, and supply chain security. The framework also outlines cyberspace risks, real-world risks, cognitive risks, and ethical risks.
• For each of these risks, high-level mitigation measures are described, allowing some flexibility in implementation.

14. China releases Draft Measures for identifying AI-generated synthetic content

• On 14 September 2024, China’s Cyberspace Administration published draft measures for identifying synthetic content generated by artificial intelligence for public consultation. 
• The Cyberspace Administration of China has proposed measures to standardize the identification of AI-generated synthetic content, aiming to protect national security and public interests while safeguarding individual rights, in accordance with existing cybersecurity laws.
• Service providers must explicitly label synthetic content with visible identifiers such as text prompts and warning signs in media and incorporate implicit identifiers in file metadata for better tracking.
• Online content platforms are required to verify implicit identifiers and add prominent warnings to synthetic content that lacks proper labeling or is self-declared by users, ensuring user awareness of potentially generated content.
• Non-compliance with these measures, such as failing to label synthetic content, will lead to penalties imposed by the Cyberspace Administration and relevant authorities. 

15. Bahrain approves draft AI law

• On 1 September 2024, Bahrain’s Shura Council, the country’s upper house of its National Assembly, approved a draft law that would establish a comprehensive legal framework for AI. The law is awaiting approval by the executive branch.
• The law also introduces penalties of up to BD 2,000 ($5,000) for individuals who program, process, or introduce AI in a way that violates individual freedom, invades the sanctity of private life, or undermines societal traditions
• Penalties would advance to imprisonment and fines for individuals who design software or technology that promotes division among people. These would extend to individuals who manipulate official statements and create malicious fake visual and audio content.
• The proposed law would also establish a dedicated unit to oversee AI development, protect investors, and ensure the safety of individuals. It also outlines specific guidelines and requirements to minimize exploitation and offer comprehensive protection to all users.
• The Shura Council indicated it would introduce further legislation on children’s online safety in the future. 

16. Sri Lanka released its Proposed AI Strategy- 18 September 2024

• On 7 September, Sri Lanka announced the publication of its draft National AI Strategy, AI Sri Lanka 2028, for public consultation.
• Based around the seven core principles of inclusivity and responsibility, trustworthiness and transparency, human-centricity, adoption-focus and impact-orientation, agile and adaptive governance, collaboration and global engagement and sustainability and future-readiness, the Strategy outlines a plan to overcome socio-economic challenges such as debt, energy needs, climate impact, and fostering public services.
• It also seeks to address challenges like brain drain, skepticism towards AI and institutional support, with a view to position Sri Lanka as a hub for high-quality AI and Data talent and development.
• It aims to do this by creating strong foundations in terms of data, skills, infrastructure, R&D, and awareness; accelerate the realization of its vision by improving public services and stimulating AI adoption in the private sector; and creating a safe and trustworthy AI ecosystem that revolves around AI governance, responsible AI, and public participation.

17. Singapore introduces bill targeting AI in elections

• On 9 September, Singapore's Ministry of Digital Development, and Information (MDDI) introduced its Elections (Integrity of Online Advertising) (Amendment) Bill into Parliament to protect Singaporeans from AI-generated misinformation during elections.
• The bill will amend the Parliamentary Elections Act 1954 (PEA) and the Presidential Elections Act 1991 (PrEA) and would prohibit the publication of digitally generated or manipulated Online Election Advertising material that relates to election candidates.
• Failure to comply could result in corrective actions being issued by the Returning Officer (RO) and Internet Access Service Providers would be required to take down offending content or to disable access by Singapore users to the content during the election period.

18. Singapore’s Supreme Court issues guidance on court use of generative AI

• On 23 September 2024, Singapore’s Supreme Court published a Guide on the Use of Generative Artificial Intelligence Tools by Court Users to set out general principles for the use of generative AI by prosecutors, lawyers, Self-Represented Persons, witnesses, and others involved in court cases in Singapore.
• Effective from 1 October 2024, the guidance notes that while it does not prohibit the use of generative AI by court users, its use must align with the principles outlined in the document.
• Specifically, court users will be fully responsible for all content in their documents, meaning that AI-generated outputs should be assessed for accuracy, relevance, and their potential to infringe on intellectual property rights. A restriction is also placed on using AI to generate evidence to be relied upon in court.
• Accuracy should be ensured through fact-checking, adapting AI-generated content, and verifying citations using reputable sources. Intellectual property rights should be protected by ensuring that sources are sufficiently cited and avoiding the unauthorized disclosure of confidential or sensitive information when using generative AI.
• Court users may also be asked to inform the court whether generative AI was used in the preparation of materials, confirm compliance with the guidance, and potentially sign an affidavit declaring such.
• A failure to comply with the guide could result in court-ordered costs, documents being disregarded, disciplinary action, or other appropriate action in accordance with existing laws.

19. Australia publishes voluntary AI safety standard

• The Australian government, on 5 September, published a voluntary AI safety standard to provide practical guidance to Australian organizations on safe and responsible AI use following its 2023 discussion paper on Safe and Responsible AI in Australia.
• The standard outlines 10 voluntary guardrails for organizations throughout the AI supply chain, where the first nine are aligned with proposed mandatory guardrails that are currently open for consultation.
• The guardrails are:

1. Accountability processes

2. Risk management process.

3. Data governance measures and protection of AI systems.

4. AI model performance evaluation and monitoring.

5. Human oversight.

6. Disclosure to end-users.

7. Processes to challenge AI system use or outcomes.

8. Transparency across the AI supply chain.

9. Record-keeping for third-party compliance assessments.

10. Stakeholder consultation that focuses on safety, diversity, inclusion, and fairness.

• The standard is designed to be technical and in alignment with international standards such that Australian companies complying with the Voluntary AI Safety Standard would also comply with international laws. 

20. UN releases final report on “Governing AI for Humanity”

• On 18 September 2024, the UN Secretary-General’s High-level Advisory Body on Artificial Intelligence (HLAB-AI) has released the final report proposing global governance mechanisms for AI, following extensive consultations and an interim report issued in December 2023.
• The report stresses the urgent need for global AI governance, noting that existing frameworks are fragmented and insufficient to address AI’s widespread and cross-border impacts.
• AI is seen as both transformative and risky, offering benefits like advancements in science and public health, while also posing significant threats such as bias, misinformation, surveillance, and risks to peace and security.
• It recommends establishing an international scientific panel, a global AI standards exchange, policy dialogues, a capacity development network, and a global fund to foster effective, inclusive governance.
• The report emphasizes that equitable distribution of AI’s benefits is critical, warning that without proper governance, AI could deepen global inequalities and limit opportunities for many regions.

Holistic AI policy updates

🎉 We’re pleased to share that our Holistic AI paper authored by Airlie Hilliard , Ayesha Gulley , Adriano Soares Koshiyama , and Emre Kazim was published open access in the International Review of Law, Computers, and Technology this month!

⚖️ The paper argues that while New York City Local Law 144 set the precedent for legally requiring bias audits of AEDTs, examining outcomes alone is not sufficient to prevent bias elsewhere in the tool. To address this, we suggest that bias audits need to be more comprehensive to catch disparate treatment and issues with model optimization but also note that this might impact compliance rates

Read it here.

Authored by Holistic AI’s Policy Team.