Google Play cracks down on AI apps after circulation of apps for making deepfake nudes

Comment
Google on Thursday is issuing new guidance for developers building AI apps distributed through Google Play, in hopes of cutting down on inappropriate and otherwise prohibited content. The company says apps offering AI features will have to prevent the generation of restricted content — which includes sexual content, violence and more — and will need to offer a way for users to flag offensive content they find. In addition, Google says developers need to “rigorously test” their AI tools and models, to ensure they respect user safety and privacy.
It’s also cracking down on apps where the marketing materials promote inappropriate use cases, like apps that undress people or create nonconsensual nude images. If ad copy says the app is capable of doing this sort of thing, it may be banned from Google Play, whether or not the app is actually capable of doing it.
The guidelines follow a growing scourge of AI undressing apps that have been marketing themselves across social media in recent months. An April report by 404 Media, for example, found that Instagram was hosting ads for apps that claimed to use AI to generate deepfake nudes. One app marketed itself using a picture of Kim Kardashian and the slogan, “Undress any girl for free.” Apple and Google pulled the apps from their respective app stores, but the problem is still widespread.
Schools across the U.S. are reporting problems with students passing around AI deepfake nudes of other students (and sometimes teachers) for bullying and harassment, alongside other sorts of inappropriate AI content. Last month, a racist AI deepfake of a school principal led to an arrest in Baltimore. Worse still, the problem is even affecting students in middle schools, in some cases.
Google says that its policies will help to keep out apps from Google Play that feature AI-generated content that can be inappropriate or harmful to users. It points to its existing AI-Generated Content Policy as a place to check its requirements for app approval on Google Play. The company says that AI apps cannot allow the generation of any restricted content and must also give users a way to flag offensive and inappropriate content, as well as monitor and prioritize that feedback. The latter is particularly important in apps where users’ interactions “shape the content and experience,” Google says, like apps where popular models get ranked higher or more prominently, perhaps.
Developers also can’t advertise that their app breaks any of Google Play’s rules, per Google’s App Promotion requirements. If it advertises an inappropriate use case, the app could be booted off the app store.
In addition, developers are responsible for safeguarding their apps against prompts that could manipulate their AI features to create harmful and offensive content. Google says developers can use its closed testing feature to share early versions of their apps with users to get feedback. The company strongly suggests that developers not only test before launching but document those tests, too, as Google could ask to review it in the future.
The company is also publishing other resources and best practices, like its People + AI Guidebook, which aims to support developers building AI apps.
Every weekday and Sunday, you can get the best of TechCrunch’s coverage.
Startups are the core of TechCrunch, so get our best coverage delivered weekly.
The latest Fintech news and analysis, delivered every Tuesday.
TechCrunch Mobility is your destination for transportation news and insight.
By submitting your email, you agree to our Terms and Privacy Notice.
Featured Article
Two senior police officials in Bangladesh are accused of collecting and selling citizens’ personal information to criminals on Telegram.
Carta, a once-high-flying Silicon Valley startup that loudly backed away from one of its businesses earlier this year, is working on a secondary sale that would value the company at…
Boeing’s Starliner spacecraft has successfully delivered two astronauts to the International Space Station, a key milestone in the aerospace giant’s quest to certify the capsule for regular crewed missions.  Starliner…
Rivian needs to sell its new revamped vehicles at a profit in order to sustain itself long enough to get to the cheaper mass market R2 SUV on the road.
Featured Article
Apple is hoping to make WWDC 2024 memorable as it finally spells out its generative AI plans.
In a research note, HSBC estimates that the Indian edtech giant Byju’s, once valued at $22 billion, is now worth nothing.
As WWDC 2024 nears, all sorts of rumors and leaks have emerged about what iOS 18 and its AI-powered apps and features have in store.
Apple’s annual list of what it considers the best and most innovative software available on its platform is turning its attention to the little guy.
Meta launched its Meta Verified program today along with other features, such as the ability to call large businesses and custom messages.
Last year, during the Q3 2023 earnings call, Mark Zuckerberg talked about leveraging AI to have business accounts respond to customers for purchase and support queries. Today, Meta announced AI-powered…
TikTok is testing streaks that are similar to Snapchat’s in order to boost engagement, including how long people stay on the app.
Welcome back to TechCrunch Mobility — your central hub for news and insights on the future of transportation. Sign up here for free — just click TechCrunch Mobility! Your usual…
New York-based Revel has made a lot of pivots since initially launching in 2018 as a dockless e-moped sharing service. The BlackRock-backed startup briefly stepped into the e-bike subscription business.…
Google says apps offering AI features will have to prevent the generation of restricted content.
The British retailers association also takes aim at Amazon’s “Buy Box,” claiming that Amazon manipulated which retailers were selected for the coveted placement.
Featured Article
Rivian has changed 600 parts on its R1S SUV and R1T pickup truck in a bid to drive down manufacturing costs, while improving performance of its flagship vehicles.  The end goal, which will play out over the coming year, is an existential one. Rivian lost about $38,784 on every vehicle…
Twitch has come up with a solution for the ongoing copyright issues that DJs encounter on the platform. The company announced Thursday a new program that enables DJs to stream…
Google said today it is partnering with RapidSOS, a platform for emergency first responders, to enable users to contact 911 through RCS (Rich Messaging Service).
Long before product-led growth became a buzzword, Atlassian offered free tiers for virtually all of its productivity and developer tools. Today, that mostly means free access for up to 10…
Featured Article
Artists have finally had enough with Meta’s predatory AI policies, but Meta’s loss is Cara’s gain. An artist-run, anti-AI social platform, Cara has grown from 40,000 to 650,000 users within the last week, catapulting it to the top of the App Store charts. Instagram is a necessity for many artists,…
Google has developed a new AI tool to help marine biologists better understand coral reef ecosystems and their health, which can aid in conversation efforts. The tool, SurfPerch, created with…
Only a few years ago, one of the hottest topics in enterprise software was ‘robotic process automation’ (RPA). It doesn’t feel like those services, which tried to automate a lot…
SpaceX achieved a key milestone in its Starship flight test campaign: returning the booster and the upper stage back to Earth.
There’s a lot of buzz about generative AI and what impact it might have on businesses. But look beyond the hype and high-profile deals like the one between OpenAI and…
Carlo Kobe and Scott Smith believed so strongly in the need for a debit card product designed specifically for Gen Zers that they dropped out of Harvard and Cornell at…
A new app called MyGlimpact is intended not only to help people understand their environmental footprint, but why they shouldn’t feel guilty about it.
Prolific Machines believes it has a way of transitioning away from molecules to something better: light.
It’s been 20 years since Shira Yevin, the lead singer of punk band Shiragirl drove a pink RV into the Vans Warped Tour grounds, the now-defunct punk rock festival notorious…
While the transport industry does use legacy software, many of these platforms are from an earlier era. Qargo hopes its newer technologies can help it leapfrog the competition.
When you look at how generative AI is being implemented across developer tools, the focus for the most part has been on generating code, as with GitHub Copilot. Greptile, an…
Powered by WordPress VIP

source
Sponsor:News technical sponsor
Sponsor:News AI sponsor
Sponsor: AI sponsor
Sponsor: AI sponsor

Leave a Comment

Vélemény, hozzászólás?

Az e-mail címet nem tesszük közzé. A kötelező mezőket * karakterrel jelöltük