Meta and Apple Under Fire for Hosting AI Deepfake Porn Ads and Apps
Both **Meta Platforms** and **Apple** are facing scrutiny for their platforms hosting advertisements and applications promoting AI-generated nonconsensual intimate imagery, including deepfakes of female politicians. Despite explicit policies against such content, these platforms have repeatedly failed to prevent the proliferation of tools that facilitate the creation of sexualized deepfakes, raising serious concerns for IT security professionals and privacy advocates.
Recent investigations have revealed that **Meta Platforms** ran ads for an AI porn-generation tool named **Kromix**, which appeared to encourage users to create deepfaked videos resembling female US politicians. This incident marks another in a series of failures by **Meta** to effectively enforce its policies against sexual material and nonconsensual intimate imagery on its advertising platforms.
### The Kromix Scandal
The **Kromix** tool, marketed as an "AI image styler," was promoted through video advertisements viewed by WIRED, boasting "no restrictions" and claiming "all the characters are real people." One particularly disturbing ad depicted a woman closely resembling a prominent female US politician, winking before transitioning to a pornographic video featuring the same face. The **Kromix** app itself, previously available on **Apple's App Store**, offered paid users the ability to upload photos for use in scenarios including "bedroom rape" and "Disney love," and featured AI-generated pornographic videos of women in Spider-Man costumes.
### Apple's Ongoing Challenges
**Apple** has also struggled to keep such illicit tools off its **App Store**, despite policies prohibiting nonconsensual intimate imagery and sexually explicit content. San Francisco's city attorney previously issued a cease-and-desist letter to **Apple** and **Google**, demanding the removal of 13 specific apps and an end to "aiding and abetting" the sale of explicit deepfake images. Following inquiries from WIRED, **Apple** confirmed the removal of **Kromix** from the **App Store**, citing the addition of prohibited content and features after initial review.
### Targeted Advertising and Policy Failures
According to **Meta's** ad library, the **Kromix** ads were exclusively targeted at male users with the tagline "Unleash Next-gen AI Magic." These ads were only removed after WIRED's direct inquiries. **Katie Paul**, director of the **Tech Transparency Project**, which uncovered the ads, highlighted a pattern: "All of the ads that we've seen so far broadly on the nudify issue are exclusively sexualizing and nudifying and putting women into AI-generated porn." This raises critical questions about **Meta's** ability to manage such content, especially when it targets female politicians.
### The Broader Threat to Politicians
Politicians and elected officials, predominantly women, have been persistent targets of nonconsensual deepfake images and videos. Incidents in Sweden, England, and Northern Ireland have shown how tools like **Grok** have been used to create millions of sexualized images. **Benjamin Shultz**, lead researcher at the **American Sunlight Project**, noted that "formerly benign mediaβofficial portraits, video from a press conferenceβhave become raw material for low- and no-code tools that anyone with an internet connection can use."
### Automated Review Systems Under Scrutiny
**Meta's** public documentation states that its ad review system relies primarily on automated tools to check ads against policies before they run. However, the discovery of the **Kromix** ads, which ran from an account created shortly before the campaign with no followers, suggests significant weaknesses in these automated defenses. **Meta's** policies explicitly forbid ads containing sexual imagery, nudity, and the promotion of "nudify" or "AI kissing" apps. Despite this, thousands of such ads have been found on **Meta's** platforms, with the company itself reporting the removal of 344,000 ads promoting these services between November and January.
**Cindy Southworth**, head of womenβs safety at **Meta**, stated, "Sexual exploitation, including promoting nudify apps, is horrific, and we work aggressively to fight it." She acknowledged the adversarial nature of the space and the continuous strengthening of defenses, including new AI technology for detection and legal action against developers. However, the persistence of these ads, often using cover photos of benign scenes to evade detection, underscores the ongoing challenge.
### Recurring Issues and AI Moderation
This incident follows a previous report by WIRED where **Tech Transparency Project** researchers found over 50 ads on **Meta's** platforms containing AI child sexual abuse material. These ads, some of which ran even after **Meta** claimed to have introduced new AI detection technology, highlighted the severe limitations of their moderation systems. **Katie Paul** criticized **Meta's** increasing reliance on AI for content moderation while simultaneously reducing human moderators, stating, "Ads are Metaβs primary product, and there appears to be zero quality control."
Both **Meta** and **Apple** are under pressure to demonstrate more effective enforcement of their policies and to safeguard users against the rapidly evolving threat of AI-generated nonconsensual intimate imagery.