Siri was the first widely available AI bot but they forgot to open it to developers. The ecosystem would have grown a million fold in all these years
Never underestimate the power of a bored programmer
Siri turn on the lights
Siri code a Doom clone in Swift
Siri design a landing page for a Consulting Agency
Siri summarize the latest news about AI
Siri find a solution to the Riemann hypothesis
Now that's an agent I'd use. Can we start with Siri for the terminal?
It’d be short sighted to not have a wrapper around which AI to use, even without implementation for them. Like, having an abstract parent class with a Siri implementation of it would be baseline good software engineering.
Yeah; plus we know they're designing with potential future local-only Vs. hybrid Vs. cloud-only in mind depending on hardware/market changes. So having it be this flexible buys them a ton of future potential.
People have speculated that with FPGAs or even an ASIC you could take some smaller models we have today, turn the chip into essentially that model-as-hardware, and get some really compelling performance/power-usage characteristics.
At that point running a GOOD quantized Q8 model on a phone is not beyond the realm.
I’m not even sure how they would implement these features without them being interoperable. All the AI systems use pretty similar if not identical API standards.
That’s actually a good solution. Create a platform where app actions can be invoked with a permission model by any AI model.
Maybe apple can even have something similar to MCPs for tool invocations of apps and publish it so all models can actually be trained to use that framework
I actually think this is what they ended up doing - just regular conversations with tool calling into your various apps, behind a Siri AI API. not HTTP based mcps (for now) but rather local ones.
Mark Gurman seems to get extraordinarily accurate and detailed leaks. I wonder how Apple got this leaky. Pretty much everything about the Duo was known months in advance.
Well, either corporate security got incompetent or they intentionally let select leaks exist to "test the market response". Established companies tend to get more risk averse overtime.
This is a very clever platform play. If you make Siri / your mac the central access point for all AI related workflows and make models interchangeable, then Apple has a huge weapon in the AI race.
I don't disagree, but I do feel the need to point out this was not Apple's plan A. Luckily for them they have so many tools and options at their disposal, that they are able to massively fail at developing their own models and still end up in a good position.
Apple “genius” has been to accept their plan A failed (the original Apple Intelligence), and instead of doubling down, let the others explore the space, then offer a good interface (I say that, I haven’t seen if it’s actually good).
So many other actors in the industry would have just doubled down and follow the race to the bottom
I imagine that Google assistant only makes money for Google because they’re an advertising company. Alexa loses money for Amazon and it’s becoming more gated to Prime subscription features as a result. Amazon thought everyone would be buying stuff with their voice and that future never panned out.
It turns out that voice assistants aren’t all that important. Customers aren’t willing to pay for them and they’re more of a nice to have convenience feature.
They didn’t invest in Siri and became the most valuable company in history. I don’t think Siri being better would have mattered much to be honest. Not joining the financial black hole that is AI training seems to have benefited them well so far. But yes, it’s definitely a case of failing upwards.
Adding quotes around “genius” as it seems you misunderstood what I meant
Meh, AI is deep in it's diversification stage -- there are soooooooo many players in the game right now and basically one maybe two will survive once the dust settles. All these frontier labs are fighting tooth and nail to be the one left standing. Apple is making a measured approach saying "hey, we're probably not going to be the Google of AI, but the smartest thing for us to do is to put infrastructure in place so that once there is a contraction (which there will be, think the search engine landscape in the 2000s) we're relatively ahead for relatively little effort/cost."
I don't know why you're getting downvoted. You're spot on.
Almost everyone developing their own AI is doomed to fail. I would not be surprised that when the dust settles, Microsoft CoPilot and Google Gemini will be left standing while OpenAI and Anthropic fade away. They will get what they need and then throw them away.
"Embrace, extend, extinguish" is the same playbook for decades, and you can only do that when you make money. Yes, making money still matters and post-capitalism is a big fat lie.
I think that's the end goal, at least looking at where their hardware is going. Or flexible routing with easy tasks running locally and only the more complex ones hitting the cloud
Just playing the devil advocate: what happens when the answers from a not-Apple LLM are garbage? Who the users are going to blame, Apple or OpenAI, Anthropic, etc..?
I'm not 100% on Apple side on this, I would love to chose whatever I want, even local models, but I get why trey trying to control 100% of the user experience. If they could trust any provider with customers data, they would let them in, since they're not competitors. They would even let Apple not spend a ton of money on developing LLMs....
Are people blaming Apple when Google search in Safari gives garbage results? Some probably are, but that really shouldn’t be a relevant discussion, in particular when the user explicitly chose which AI backend to use.
Well, we can generalize this a bit to model routing in general. Actually, just AI in general - even when directly interacting with GPT we haven't figured out good protocols yet.
This appears to be the current feature prototype intended for the EU and potentially China. Apple has chosen not to release the new Siri AI in the EU because our Digital Markets Act requires them to allow interoperability. I.e. that users be allowed to choose another AI provider.
Ostensibly they'll activate this in the EU when it's ready. Given the current functionality, it looks ready (or close to being ready). My guess is it is ready and Apple is doing their usual negotiation tactics with the EU to see who blinks first.
My secret hope is that Tim Cook was the main weight behind cutting off half the new features from Apple software in the EU an attempt to weaponize their users against legislators. Even features that have had analogues where nobody would've ever asked for interoperability (i.e. Handoff/Continuity exists in the EU, but iPhone mirroring doesn't)
If this consistently has happened at every place you've worked in your career, I think you're luckier than you realize. Independent of the implementation, it's also still interesting as a sign of what might be supported for users down the line; I don't agree that this is how they'd write the code if they never expected to allow another provider, and other comments in the thread have already provided some insight into what the context might be (e.g. EU market).
You can download an app that does that. You don’t have to use Siri, and it can be turned off entirely, or decoupled from AI entirely, and the wake word can be disabled entirely. A lot of options are available.
Make a Shortcut that will open your local voice assistant app of choice when you press the action button.
Siri was the first widely available AI bot but they forgot to open it to developers. The ecosystem would have grown a million fold in all these years
Never underestimate the power of a bored programmer
Now that's an agent I'd use. Can we start with Siri for the terminal?$ Siri 'List the files in this folder'
It’d be short sighted to not have a wrapper around which AI to use, even without implementation for them. Like, having an abstract parent class with a Siri implementation of it would be baseline good software engineering.
Yeah; plus we know they're designing with potential future local-only Vs. hybrid Vs. cloud-only in mind depending on hardware/market changes. So having it be this flexible buys them a ton of future potential.
People have speculated that with FPGAs or even an ASIC you could take some smaller models we have today, turn the chip into essentially that model-as-hardware, and get some really compelling performance/power-usage characteristics.
At that point running a GOOD quantized Q8 model on a phone is not beyond the realm.
Maybe we would get LLMs on (pluggable) cartridges.
May happen with Android, not going to happen on $AAPL devices... Heck, even the battery is non-swappable!
I’m not even sure how they would implement these features without them being interoperable. All the AI systems use pretty similar if not identical API standards.
And given how Apple switched out OpenAI for Gemini (to power Siri), they experienced firsthand the importance of having said wrapper.
That’s actually a good solution. Create a platform where app actions can be invoked with a permission model by any AI model.
Maybe apple can even have something similar to MCPs for tool invocations of apps and publish it so all models can actually be trained to use that framework
I actually think this is what they ended up doing - just regular conversations with tool calling into your various apps, behind a Siri AI API. not HTTP based mcps (for now) but rather local ones.
Now Claude will be listening to your audio 24/7 via the AOP logs:
And serve you ads accordingly (or sell the log data to other ad networks)Glad I’ve never turned on Siri
I would pay a good chunk of money if Apple released a local AI hub to coordinate all AI usage locally (including photo indexing).
That is rumored to exist.
https://www.macrumors.com/guide/apple-command-center/
Mark Gurman seems to get extraordinarily accurate and detailed leaks. I wonder how Apple got this leaky. Pretty much everything about the Duo was known months in advance.
Well, either corporate security got incompetent or they intentionally let select leaks exist to "test the market response". Established companies tend to get more risk averse overtime.
Photo indexing is already local. What specifically are you looking for?
You should look at Osaurus.
This is a very clever platform play. If you make Siri / your mac the central access point for all AI related workflows and make models interchangeable, then Apple has a huge weapon in the AI race.
I don't disagree, but I do feel the need to point out this was not Apple's plan A. Luckily for them they have so many tools and options at their disposal, that they are able to massively fail at developing their own models and still end up in a good position.
Apple “genius” has been to accept their plan A failed (the original Apple Intelligence), and instead of doubling down, let the others explore the space, then offer a good interface (I say that, I haven’t seen if it’s actually good).
So many other actors in the industry would have just doubled down and follow the race to the bottom
Genius would have been for Apple to get Siri in a better state after so many years.
This looks more like a participation medal, if at all.
I imagine that Google assistant only makes money for Google because they’re an advertising company. Alexa loses money for Amazon and it’s becoming more gated to Prime subscription features as a result. Amazon thought everyone would be buying stuff with their voice and that future never panned out.
It turns out that voice assistants aren’t all that important. Customers aren’t willing to pay for them and they’re more of a nice to have convenience feature.
They didn’t invest in Siri and became the most valuable company in history. I don’t think Siri being better would have mattered much to be honest. Not joining the financial black hole that is AI training seems to have benefited them well so far. But yes, it’s definitely a case of failing upwards.
Adding quotes around “genius” as it seems you misunderstood what I meant
> They didn’t invest in Siri...
What makes you think that? They insisted on Siri for so long and even made a big deal of Apple Intelligence in WWDC.
Then Apple threw the towel and waved the white flag with regards to Siri and replaced their efforts with third party AI.
Meh, AI is deep in it's diversification stage -- there are soooooooo many players in the game right now and basically one maybe two will survive once the dust settles. All these frontier labs are fighting tooth and nail to be the one left standing. Apple is making a measured approach saying "hey, we're probably not going to be the Google of AI, but the smartest thing for us to do is to put infrastructure in place so that once there is a contraction (which there will be, think the search engine landscape in the 2000s) we're relatively ahead for relatively little effort/cost."
I don't know why you're getting downvoted. You're spot on.
Almost everyone developing their own AI is doomed to fail. I would not be surprised that when the dust settles, Microsoft CoPilot and Google Gemini will be left standing while OpenAI and Anthropic fade away. They will get what they need and then throw them away.
"Embrace, extend, extinguish" is the same playbook for decades, and you can only do that when you make money. Yes, making money still matters and post-capitalism is a big fat lie.
Nice, thanks EU for forcing Apple. I guess it was possible all along.
It would be great if AI could be completely removed from iOS altogether, including any underlying frameworks.
Don't they need that for China for compliance with their law?
Possibly, but certainly the EU.
Is there any hint that Siri can be connected to arbitrary MCP providers?
That would be really cool, if you can route your local model for it.
I think that's the end goal, at least looking at where their hardware is going. Or flexible routing with easy tasks running locally and only the more complex ones hitting the cloud
Just playing the devil advocate: what happens when the answers from a not-Apple LLM are garbage? Who the users are going to blame, Apple or OpenAI, Anthropic, etc..? I'm not 100% on Apple side on this, I would love to chose whatever I want, even local models, but I get why trey trying to control 100% of the user experience. If they could trust any provider with customers data, they would let them in, since they're not competitors. They would even let Apple not spend a ton of money on developing LLMs....
Are people blaming Apple when Google search in Safari gives garbage results? Some probably are, but that really shouldn’t be a relevant discussion, in particular when the user explicitly chose which AI backend to use.
Well, we can generalize this a bit to model routing in general. Actually, just AI in general - even when directly interacting with GPT we haven't figured out good protocols yet.
If you are smart enough to change the default, you would know which LLM you picked and could point the finger at it.
Apple is in the game of providing an application platform. Maybe in the future they will provide their own LLM.
So will they have access to the AOP?
You can’t even search this because Google thinks you mean “app” even when you use quotes around “aop”.
It’s the thing that runs PhraseSpotter 24/7
Hmmm then why is it not available in EU?
This appears to be the current feature prototype intended for the EU and potentially China. Apple has chosen not to release the new Siri AI in the EU because our Digital Markets Act requires them to allow interoperability. I.e. that users be allowed to choose another AI provider.
Ostensibly they'll activate this in the EU when it's ready. Given the current functionality, it looks ready (or close to being ready). My guess is it is ready and Apple is doing their usual negotiation tactics with the EU to see who blinks first.
My secret hope is that Tim Cook was the main weight behind cutting off half the new features from Apple software in the EU an attempt to weaponize their users against legislators. Even features that have had analogues where nobody would've ever asked for interoperability (i.e. Handoff/Continuity exists in the EU, but iPhone mirroring doesn't)
This is not how politics works here.
There’s a difference between Apple being able to swap out the model in code and Apple letting end users choose which model to use.
I think it is because the EU is requiring broader access to the deeper layers that Apple doesn’t want to give.
EDIT: by broader I mean all apps the user installs on their phone.
Siri, and Alexa, would be significantly more useful if it could be connected to ChatGPT etc.
It happens now – when I ask certain questions Siri asks if I want ChatGPT to answer it.
Good news, Alexa+ (included with Prime) is already an Anthropic model, probably Sonnet.
News at 11: Developers write decoupled code.
If this consistently has happened at every place you've worked in your career, I think you're luckier than you realize. Independent of the implementation, it's also still interesting as a sign of what might be supported for users down the line; I don't agree that this is how they'd write the code if they never expected to allow another provider, and other comments in the thread have already provided some insight into what the context might be (e.g. EU market).
Neat. Now let me point it to an OpenAI-compatible API endpoint of my choosing, like a Mac Mini running Hermes on my LAN.
I do not want my personal data or the queries against it to leave my network boundaries if at all possible. This should be a default option.
You can download an app that does that. You don’t have to use Siri, and it can be turned off entirely, or decoupled from AI entirely, and the wake word can be disabled entirely. A lot of options are available.
Make a Shortcut that will open your local voice assistant app of choice when you press the action button.