Your Full Stack AI App With Zero API Costs Is Here. Apple Just Showed Us How.

You know that moment when you are building a feature and you think “I could add AI here, but then I need an API key, a billing account, a fallback for when the service is down, and now my app has a dependency I cannot control”? That mental tax is real, and a lot of developers quietly drop the idea and move on.

Apple just made that excuse disappear. At least on their platform.

At a recent Apple Developer Experience webinar on Foundation Models, they demonstrated a single app doing speech to text, image generation, image recognition, action item extraction, calendar checking via tool calling, summarisation, guardrails, prompt injection prevention, few shot prompting, structured output with generatable types, and context window management. All of it running on device. No API calls. No cloud dependency. No usage costs.

That list deserves a second read. Because if you were to build that today using cloud services, you would be stitching together Whisper for speech, a diffusion API for image generation, a vision service, an LLM provider for reasoning, a moderation layer for guardrails, and your own JSON wrangling to get structured outputs. Every one of those is a separate billing relationship, a separate point of failure, and a separate privacy consideration.

Apple collapsed that entire stack into the frameworks already shipping on the device in your pocket.

Why Apple Can Actually Pull This Off When Others Cannot Fully

This is where it gets interesting, and where the Android comparison is worth being honest about.

Google is doing serious on-device AI work. Gemini Nano runs on Pixel devices and is genuinely capable. Some open source models running via llama.cpp on Android flagship hardware are competitive on raw benchmarks. Android is not standing still here.

But there is a structural difference that matters enormously to developers shipping real apps.

Apple designs the Apple Neural Engine on their silicon, writes the OS, owns the developer frameworks, and controls app distribution. That full stack ownership means when they say “this runs locally on device”, they can guarantee it because they designed the chip it runs on. Every developer targets the same stack. Every supported user gets the same capability.

Android cannot make that same promise uniformly. Google ships Gemini Nano on Pixel, but that is only Pixel. Samsung does their own thing. Qualcomm and MediaTek devices have completely different NPU capabilities and driver stacks. An Android developer writing a feature that depends on on device AI is writing a feature that works reliably on some devices and degrades unpredictably on others. The ecosystem fragmentation that has always been Android’s challenge does not disappear when AI enters the picture, it gets more complicated.

Apple developers write to the framework and trust the hardware is there. That consistency is not a small thing. It is the difference between a feature you can confidently ship and support versus one you have to wrap in capability checks and fallbacks.

This is not a knock on Android as a platform. It is a structural reality of how the two ecosystems are built. Apple’s vertical integration, often cited as a limitation for openness, turns out to be a significant advantage when the thing you are trying to do requires tight coordination between silicon, OS, and developer APIs.

What the Stack Actually Looks Like

Apple is exposing this through a combination of mature frameworks rather than one monolithic API.

The Speech framework handles transcription. The Vision framework handles image understanding, text recognition, and object detection. The Foundation Models framework exposes the on device LLM for reasoning, summarisation, and generation. These frameworks have been growing for years and are now connected by a capable on device language model that can orchestrate between them.

The tool calling piece is what makes it genuinely agentic. The model can decide to check your calendar, invoke a vision task, or trigger a speech transcription as part of a reasoning chain. Here is roughly what that looks like in Swift:

let session = LanguageModelSession(tools: [CalendarTool(), VisionTool()])
let response = try await session.respond(to: userPrompt)

The model handles the orchestration. Your code just defines what tools are available and what they do. That pattern will feel familiar to anyone who has used function calling with OpenAI or tool use with Claude, except there is no network call happening.

Generatable Types Are the Quiet Killer Feature

If you have spent any time coaxing LLMs into returning valid JSON and then parsing it defensively because the model decided to wrap it in markdown or add an apology, generatable types will feel like a relief.

You define a Swift struct, the model generates directly into that type. No prompt engineering tricks to force JSON. No fragile parsing. The output is guaranteed to match your schema.

@Generable
struct ActionItems {
    let items: [String]
    let priority: Priority
    let suggestedDate: String
}

let result = try await session.respond(
    to: "Extract action items from this meeting note",
    generating: ActionItems.self
)

That is a meaningful reduction in the surface area where things go wrong.

The Context Window Reality Check

The on device foundation models have a 4096 token context window. Roughly 3000 words. For back and forth chat that burns fast, but chat is actually the wrong mental model for how these features are designed to work.

Every feature in that demo was a bounded, stateless task. Transcribe this audio. Recognise what is in this image. Extract action items from this text. Generate a calendar event. None of those need a long conversation history. They are smart function calls, not ongoing dialogues.

Where 4096 does become a real constraint is long document summarisation or agentic workflows with many tool call steps accumulating in context. The standard workarounds apply: chunking, sliding windows, summarising older context back in. Apple’s context window management APIs handle some of this automatically, but it is worth designing your features with this ceiling in mind from the start.

But Are the Models Actually Good Enough?

Honest answer: not for everything, but for more than you probably expect.

These are models in the 3 to 7 billion parameter range. They are not going to outthink frontier cloud models on complex multi step reasoning. Open ended creative generation and long context understanding are weaker. If you are building something that needs deep analytical reasoning or genuinely creative output, you will still want a cloud model.

But most app features are not asking for frontier intelligence. They are bounded, well scoped tasks. Does this image contain a receipt? Extract the line items. Convert this voice note into a structured reminder. Summarise this notification thread. For tasks like these, a well prompted smaller model with few shot examples performs well in practice.

The demo Apple showed was smart precisely because every single feature was a constrained, well defined task. That is not hiding the limitations. That is good product thinking about what on device AI is actually suited for.

What This Means for the Platform Longer Term

Apple is making a clear bet. As on device models get more capable with each silicon generation, the gap between what you can do locally versus what you need the cloud for keeps narrowing. Every A-series and M-series chip generation has brought meaningful improvements to the Neural Engine. The models running on an iPhone 17 will be noticeably better than what runs on an iPhone 15.

That trajectory matters. Google can improve their cloud models faster, but improving on device AI uniformly across hundreds of Android device variants from dozens of manufacturers is a coordination problem that does not get easier with time. Apple only has to improve their own silicon and their own frameworks.

Privacy by default. Zero marginal cost per user. No third party uptime dependency. Works offline. Consistent across every supported device. These are not just nice properties, they are the foundation of a different category of AI powered product you can build with confidence.

The full stack AI app without an API bill is not a future thing. Apple just showed it running. Whether this becomes a defining platform advantage depends on how capable these on device models become over the next few hardware generations. But the direction is clear, and the infrastructure is already in your users’ hands.

The question now is what you build with it.