The real power of Apple Intelligence will show up in third-party apps

Comment
Apple Intelligence, the iPhone maker’s new set of AI capabilities arriving in iOS 18, is laying the groundwork for a new way to use apps.
Today, the dated App Store model is under constant regulatory attack. Meanwhile, users can accomplish a lot of tasks with fairly simple questions to an AI assistant like ChatGPT. Proponents believe AI could become the preferred way we’ll search for answers, be productive at work, and experiment with creativity.
Where does that leave the world of apps, and the growing services revenue (more than $6 billion last quarter) they generate for Apple?
The answer cuts to the core of Apple’s AI strategy.
Apple Intelligence itself only offers a small set of capabilities out-of-the-box, like writing helpers, summarization tools, generative art, and other baseline offerings.
But earlier this year at its Worldwide Developers Conference (WWDC) in June, Apple presented new features that will allow developers’ apps to connect more deeply with both Siri and Apple Intelligence.
Improvements to the smart assistant will allow Siri to invoke any item from an app’s menu without additional work on a developer’s part. That means users could ask Siri to “show me my presenter notes” in a slide deck, for instance, and Siri would know what to do. Siri will also be able to access any text displayed on the page, allowing users to reference and act on what’s on their screen.
So, if you were looking at your reminder to wish a family member a “happy birthday,” you could say something like “FaceTime him” and Siri would know what action to take.
That’s already an upgrade from the basic functionality today’s Siri offers, but it doesn’t end there. Apple is also providing developers with tools to use Apple Intelligence in their own apps. At WWDC, the company indicated that Apple Intelligence would first be made available to certain categories of apps, including Books, Browsers, Cameras, Document readers, File management, Journals, Mail, Photos, Presentations, Spreadsheets, Whiteboards, and Word processors. Over time, Apple is likely to open up these capabilities to all developers across the App Store.
The AI functionality will be built on top of the App Intents framework, which is being expanded with new intents for developers. The eventual goal is to allow users to interact with Siri not just to open their apps, but also to use them.
That means a user wouldn’t have to dig around in an app’s menus to find the feature they needed to perform a task. They could just ask Siri.
Users could also make these requests while speaking naturally — conversationally — and could reference things that related to their personal context.
So, for instance, you could ask a photo-editing app like Darkroom to “apply a cinematic present to the photo I took of Ian yesterday.” Today’s version of Siri would balk at this sort of request, but the AI-powered Siri would instead know to leverage the app’s Apply Filter intent, as well as which photo you’re asking to use it on.
Siri will be able to take action even if you stumbled over your words or referenced an earlier part of the conversation in your instructions, Apple has said.
You could also take action across apps. For example, after editing your photo, you could ask Siri to move it into another app, like Notes, without having to tap on anything.
In addition, the iPhone’s search feature, Spotlight, will be able to search data from apps by incorporating app entities into its index. This refers to Apple Intelligence’s understanding of things like photos, messages, files, calendar events, and more.
This subtler use case for AI, of course, requires developer adoption. Apple has over the years alienated some of its larger developers and even some of its indies with its revenue-sharing rules, which generally allow the company to keep 30% of revenues for products and services sold through any app. But developers could be drawn back in as Siri takes apps that were previously hidden in a back-of-the-phone App Library and makes them easily accessible through voice commands.
Instead of boring onboarding screens to train users on how to navigate and use their app, developers could instead focus on making sure Siri understands how their app works, and how users might ask for the things they want to do in it. That way, users could engage with the app via Siri either by speaking or typing in commands, similar to how they today engage with an AI chatbot like ChatGPT.
Third-party developers will gain other benefits from Apple’s new AI architecture, too.
With its OpenAI partnership, Siri will be able to hand off queries to ChatGPT when it doesn’t have the answer. With its visual search feature on the iPhone 16 lineup, Apple will also allow users to access OpenAI’s chatbot or Google Search just by tapping on the new Camera Control button on the side, turning what they’re seeing through the camera’s viewfinder into an actionable query.
These developments won’t feel as immediately revolutionary as the introduction of something like ChatGPT did because the rate of developer adoption will likely vary.
Moreover, these future promises seem like they’re still a ways out. In the latest iOS 18 betas, the functionality feels incomplete. As often as I was surprised by what the new Siri can do, I was just as often confused by those things it can’t. That includes within Apple’s own apps. For instance, you can ask Siri in the Photos app to send a photo you’re viewing to someone, but you can’t ask it to do something more complex, like turn the photo into a sticker. Until Siri stops hitting these kinds of roadblocks, the functionality may end up feeling frustrating to use.


Every weekday and Sunday, you can get the best of TechCrunch’s coverage.
Startups are the core of TechCrunch, so get our best coverage delivered weekly.
The latest Fintech news and analysis, delivered every Tuesday.
TechCrunch Mobility is your destination for transportation news and insight.
By submitting your email, you agree to our Terms and Privacy Notice.
Paymob has evolved into an omnichannel gateway offering over 50 methods for offline and online payments to more than 350,000+ merchants.
Adtech startup InMobi has raised $100 million in debt financing as the profitable Indian firm looks to “significantly deepen” its artificial intelligence initiatives and fund potential AI acquisitions ahead of…
Google co-founder and ex-Alphabet president Sergey Brin said he’s back working at Google “pretty much every day” because he hasn’t seen anything as exciting as the recent progress in AI…
The $49 cases work with MagSafe and the iPhone’s new Camera Control button.
A group of Democratic senators is urging the FTC and Justice Department to investigate whether AI tools that summarize and regurgitate online content like news and recipes may amount to…
Sydney-based Neurode has come up with a novel way to offer some relief to people with ADHD: a wearable headband that can track and treat ADHD symptoms. And you only…
Bumble is increasing its investments in AI and branching out with new features to stay relevant amid a generational shift in dating behavior among younger users. During Goldman Sachs’ annual technology…
Apple Intelligence, the iPhone maker’s new set of AI capabilities arriving in iOS 18, is laying the groundwork for a new way to use apps. Today, the dated App Store…
SpaceX has launched its most public and aggressive offensive against regulators to date, with a blog post published Tuesday urging more expeditious launch licensing — lest the country lose its…
Is the grass greener on the other side? We’re not sure, but the sky is most certainly bluer. It’s been almost two years since Elon Musk purchased Twitter, now X,…
Apple has not discussed whether the feature’s removal was a hardware change on the Series 10 or if it was simply disabled via the software, like with the Series 9.
Drybaby is the newest dating app designed for singles who are sober or interested in reducing their alcohol consumption.  The app arrives as 41% of Americans are cutting back on…
Gemini is Google’s long-promised, next-gen generative AI model family.
The London transport authority removes a claim that said there was no evidence that customer data was compromised during a recent hack.
Hearing Test will be built into iOS 18, though Apple is currently awaiting FDA approval for both it and the newly announced hearing aid feature before launch.
Featured Article
Apple’s lineup of announcements echoed many of the anticipated hardware reveals, including the new iPhone 16, AirPods 4, the Apple Watch Series 10 and more.
One would have expected that “Apple’s first phone made from the ground up for Apple Intelligence” would justify being so.
Welcome to TechCrunch Fintech! It’s finally cooling down here in my home base of Austin, Texas, and that makes me happy. In this week’s newsletter, we’re looking at a big…
The Mate XT Ultimate Design costs roughly $2,800 and is only available in China for now.
Neat helps other companies sell insurance products to their own customers. In insurance lingo, it focuses on affinity insurance contracts linked to another service or product.
EV warranty startup Amber is expanding nationwide, bringing on new vehicles and offering a new remote diagnostic scan — all signs that the one-year-old company is putting its recent $3.18…
A SpaceX Falcon 9 lifted off in the early hours of Tuesday morning carrying a crew that will attempt the first commercial spacewalk and travel higher than any crewed mission…
Amazon has announced a new pilot package delivery service in Frankfurt, Germany, that meshes electric road and rail transport. The e-commerce giant will use vans from its delivery depot to…
Apple’s relationship with Google as its search partner is taking a new turn with Apple’s introduction of visual search, or “Visual Intelligence,” as the iPhone maker dubbed it Monday during…
Spotter, a startup that underwrites creators and offers AI tools, raised $7.4 million, according to a Form D filing viewed by TechCrunch. A representative for Spotter confirmed the filing’s legitimacy,…
Cities spend hundreds of millions each year on paratransit services, or public rides for disabled residents, yet those services remain limited and unpredictable. Wheelchair users often face late pickups, hindering…
Lucid Motors is just a few months away from finally launching its electric SUV, and the company now says the upcoming vehicle — dubbed Gravity — will have Tesla’s North…
MariaDB‘s short-lived tenure as a public company is all but over, as the struggling database business is now fully under the auspices of K1 Investment Management. MariaDB also announced a…
In addition, JFrog is launching a runtime security solution, as well as an integration with Nvidia’s NIM microservices.
In the world of pharmaceuticals, companies aim to operate under GMP guidelines — a set of production and manufacturing measures to assure standards for medicinal products. But too often, the…
Powered by WordPress VIP

source
Sponsor:News technical sponsor
Sponsor:News AI sponsor
Sponsor: AI sponsor
Sponsor: AI sponsor