From OpenAI · · 3m
OpenAI Disrupts AI-Enabled False Front Operations
OpenAI bans two influence operations using AI to create fake journalist personas and spread conflict-related content.
In brief
OpenAI bans two influence operations using AI to create fake journalist personas and spread conflict-related content. OpenAI disrupted Russian and Iranian operations that used AI to generate false fronts for spreading geopolitical messaging. Originally reported by OpenAI.
OpenAI Disrupts AI-Enabled False Front Operations
OpenAI banned two influence operations, one from Russia and one from Iran, that used AI to create fake journalist personas and spread geopolitical messaging.
Iranian Operation: Seven Journalist Personas
Iranian actors created seven fake journalist personas to pitch long-form articles on US-Iran conflict to small and medium online outlets globally.
Russian Operation: Latin America Think Tank
Russian actors co-opted individuals in Latin America to run a fake 'think tank' called the Social Research Center (SRC) to spread disinformation.
Operation 'Dark Clark' Details
Russian operators used ChatGPT to draft internal reports, create fake emails, and generate audio scripts targeting Latin American countries.
Fake Emails in Peru and Ecuador
Operators created fake emails purporting to come from Peruvian and Ecuadorian institutions to trick schools into holding events promoting Ukrainian nationalism.
TikTok Fakes Spreading
TikTok posts with fabricated audio attributed to Ukraine’s consul in Ecuador and fake letters about Panama’s financial sector were shared by operation-generated accounts.
Social Media Comments Generated
Operators generated batches of social media comments in English and Persian on US-Iran conflict topics, often within minutes of each other.
Iranian Operation's Impact
The Iranian operation's articles reached Category 4 on the IO Breakout Scale, while the Russian operation reached Category 5, the highest category OpenAI has disrupted.
Operation 'Bogus Bylines'
Iranian actors used seven fake journalist personas to pitch long-form articles on US-Iran conflict to online outlets globally.
Fake Social Media Accounts
Iranian accounts showed transparency settings indicating locations in Iran, despite claiming to be Western journalists.
Social Media Engagement
The Iranian operation's social media comments generated low engagement, with replies making up a minority of total comments on posts.
Internal Reporting
Both operations used ChatGPT to draft internal reports detailing their activities, including fake stories and disinformation campaigns.
Impact on Media Outlets
The Iranian operation's articles were published by outlets with substantial followings, including one with nearly 2 million Facebook followers.
Operation 'Dark Clark' Impact
The Russian operation's fake stories triggered fact checks and official denials in Peru and Ecuador, leading to public comment from politicians.
False Front Operations
Both operations used AI to create false fronts for spreading geopolitical messaging, with the Iranian operation reaching Category 4 and the Russian operation Category 5 on the IO Breakout Scale.
What Happens Next
OpenAI aims to make further research and disruption easier while making continuing the operations harder through responsible disclosure.
The one thing
OpenAI disrupted Russian and Iranian operations that used AI to generate false fronts for spreading geopolitical messaging.
