Generative AI has a plethora of well-documented misuses, from making up educational papers to copying artists. And now, it seems to be cropping up in state affect operations.
One latest marketing campaign was “very likely” helped by business AI voice era merchandise, together with tech publicly launched by the recent startup ElevenLabs, in line with a latest report from Massachusetts-based menace intelligence firm Recorded Future.
The report describes a Russian-tied marketing campaign designed to undermine Europe’s help for Ukraine, dubbed “Operation Undercut,” that prominently used AI-generated voiceovers on pretend or deceptive “news” movies.
The movies, which focused European audiences, attacked Ukrainian politicians as corrupt or questioned the usefulness of army assist to Ukraine, amongst different themes. For instance, one video touted that “even jammers can’t save American Abrams tanks,” referring to gadgets utilized by US tanks to deflect incoming missiles – reinforcing the purpose that sending high-tech armor to Ukraine is pointless.
The report states that the video creators “very likely” used voice-generated AI, together with ElevenLabs tech, to make their content material seem extra professional. To confirm this, Recorded Future’s researchers submitted the clips to ElevenLabs’ personal AI Speech Classifier, which offers the power for anybody to “detect whether an audio clip was created using ElevenLabs,” and received a match.
ElevenLabs didn’t reply to requests for remark. Although Recorded Future famous the doubtless use of a number of business AI voice era instruments, it didn’t title any others apart from ElevenLabs.
The usefulness of AI voice era was inadvertently showcased by the affect marketing campaign’s personal orchestrators, who – somewhat sloppily – launched some movies with actual human voiceovers that had “a discernible Russian accent.” In distinction, the AI-generated voiceovers spoke in a number of European languages like English, French, German, and Polish, with no foreign-soundings accents.
According to Recorded Future, AI additionally allowed for the deceptive clips to be rapidly launched in a number of languages spoken in Europe like English, German, French, Polish, and Turkish (by the way, all languages supported by ElevenLabs.)
Recorded Future attributed the exercise to the Social Design Agency, a Russia-based group that the U.S. authorities sanctioned this March for operating “ a network of over 60 websites that impersonated genuine news organizations in Europe, then used bogus social media accounts to amplify the misleading content of the spoofed websites.” All this was executed “on behalf of the Government of the Russian Federation,” the U.S. State Department mentioned on the time.
The total impression of the marketing campaign on public opinion in Europe was minimal, Recorded Future concluded.
This isn’t the primary time ElevenLabs’ merchandise have been singled out for alleged misuse. The firm’s tech was behind a robocall impersonating President Joe Biden that urged voters to not exit and vote throughout a major election in January 2024, a voice fraud detection firm concluded, in line with Bloomberg. In response, ElevenLabs mentioned it launched new security options like routinely blocking voices of politicians.
ElevenLabs bans “unauthorized, harmful, or deceptive impersonation” and says it makes use of varied instruments to implement this, resembling each automated and human moderation.
ElevenLabs has skilled explosive development since its founding in 2022. It lately grew ARR to $80 million from $25 million lower than a yr earlier, and should quickly be valued at $three billion, TechCrunch beforehand reported. Its buyers embody Andreessen Horowitz and former Github CEO Nat Friedman.
