Core ML vs Core AI: A Side-by-Side Comparison

Part of the ANE Knowledge Base. Built from the WWDC research in doc 08 (Core ML 2022–2025) and doc 09 (Core AI, WWDC26), the pinned repos under references/, and third-party reporting where noted (fetched 2026-07-12). Apple's official stance as of WWDC26: both frameworks coexist in iOS/macOS 27; no Core ML deprecation date has been announced.

1. TL;DR#

	Core ML (2017 → )	Core AI (WWDC26 → )
One-liner	General ML deployment framework; the OS decides how to run your converted model	Inference stack purpose-built for modern (transformer/generative) workloads with explicit developer control
Sweet spot	Classic ML & mid-size neural nets: classifiers, vision/audio models, pipelines already built on coremltools	LLMs, VLMs, diffusion, agentic features; anything KV-cached/autoregressive; models needing per-hardware tuning
Minimum OS	Very broad (iOS 11+; features gated by year — see doc 08)	iOS/macOS 27+, Xcode 27
Status	Maintained; zero WWDC26 sessions; still the only option for pre-27 deployment targets	The strategic direction; powers on-device Apple Intelligence

2. Full comparison#

Dimension	Core ML	Core AI
Model asset	.mlmodel / .mlpackage (ML Program, MIL ops), compiled on device to .mlmodelc	.aimodel (device-independent source asset)
Conversion entry	coremltools.convert() over torch.jit.trace output (ct 8 added early torch.export support)	coreai_torch.TorchConverter over torch.export programs natively
Dynamic shapes	Bolted on: enumerated/range shapes, or one trace per shape + multifunction merging (doc 06 §8)	Native: torch.export.Dim(min=, max=) flows into the .aimodel (dims shown as ? in Xcode)
High-level ops	Author decomposes by hand (the 2022–2024 era of this KB); fused SDPA arrived only in iOS 18 targets	Composite ops (SDPA, RMSNorm, RoPE, sliding-window attention, gather_mm…) preserved via decomp tables; compiler picks per-hardware lowering (coreai-torch/coreai_torch/composite_ops/)
Per-platform optimization	One graph; Core ML segments it across CPU/GPU/ANE at load	Explicit ios/ vs macos/ primitives and --platform export flag; iOS classes hand-shaped for ANE, macOS uses fused GPU ops (coreai-models/.../primitives/)
Compilation	On-device "specialization" at first load only; opaque; mitigations = async load + cache folklore (doc 05)	Same specialization plus AOT: xcrun coreai-build compile --platform iOS on the dev machine; explicit AIModel.specialize/AIModelCache APIs
State / KV cache	Stateless by design; states retrofitted as MLState (iOS 18, doc 08)	Stateful from day one: NDArray buffers passed as non-escapable MutableViews, updated in-place; mutable_slice_update primitive for cache writes
Swift API	MLModel + feature providers / MLMultiArray; MLTensor (2024) for glue math	AIModel → InferenceFunction.run(inputs:states:) with NDArray; zero-copy views, pre-allocation, async pipelining; memory-safety via non-escapable types
LLM ergonomics	Hand-rolled decode loop (MLTensor sampling, manual KV plumbing — the WhisperKit era, doc 06)	CoreAILanguageModel plugs into Foundation Models LanguageModelSession: streaming, @Generable structured output, tool calling — with your custom weights
Compression	ct.optimize.coreml / ct.optimize.torch (doc 07): palettization, pruning, INT4/8, per-grouped-channel (iOS 18 gates)	coreai-opt: same families plus FP4/FP8 and QAT, preset-driven (presets.w4().without("lm_head")), declarative YAML recipes in the model zoo
Custom ops	Custom MIL composite ops / custom layers (CPU/GPU code in the app)	TorchMetalKernel DSL: MSL written in Python, verified against a PyTorch reference, embedded inside the .aimodel
Multi-model packaging	Multifunction models (iOS 18); separate .mlmodelc per component (WhisperKit pattern)	Multi-entrypoint conversion: several exported functions → one .aimodel (SAM3 image_encode/text_encode/detect)
Observability	Xcode performance reports, Core ML + ANE Instruments, MLComputePlan (programmatic dispatch, doc 06 §8)	Core AI Instruments template (load/specialization/inference events), Core AI Debugger (graph ↔ original Python source, on-device tensor inspection, PSNR comparison mode), Xcode debug gauge
Numerical verification	DIY: PSNR harnesses like argmaxtools' (35 dB threshold)	Built-in: Python coreai.runtime parity checks + Debugger comparison mode (PSNR/MSE/MAE per layer)
Distribution	Bundle or on-demand resources	Guidance: keep models out of the bundle, deliver via Background Assets on opt-in (session 326)
Model zoo	None official (community: Hugging Face, WhisperKit)	coreai-models: whisper, qwen3, gemma3, mistral/mixtral, gpt_oss, sam3, stable-diffusion, flux2… + Swift runtime packages + agent skills

3. The four deep deltas#

3.1 Who owns the optimization#

The defining difference for this KB. In the Core ML era, the model author owned ANE optimization: Apple published principles (2022), and teams like Argmax hand-built BC1S attention, split softmax, cache masking (docs 02–06). In Core AI, the toolchain and platform primitives own it: you write composite_ops.SDPA, and the compiler lowers it per hardware — falling back to Apple's own hand-shaped ios/ primitives (which are the 2022 principles, verbatim — doc 09 §3) for ANE-bound components. The escape hatches (custom Metal kernels, re-authoring) remain for the frontier.

Measure of the shift: Whisper needed a from-scratch reimplementation + PyPI library in the Core ML era (whisperkittools, ~5k lines studied in doc 06); the Core AI recipe exports stock HF whisper-large-v3-turbo in one ~200-line script (references/coreai-models/models/whisper/export.py).

3.2 KV-cache lineage#

The full evolution, in one line each:

1. 2022–23 (Core ML, stateless): cache as model I/O + one-hot mask blend cache*(1−m)+current*m — WhisperKit (doc 06 §4).
2. 2024 (Core ML + MLState): cache as registered buffer, in-place update, 1.6× decode speedup on Mistral-7B (doc 08).
3. 2026 (Core AI): states as first-class MutableViews in the run call; graph-level mutable_slice_update writes the new token slice directly; iOS still updates on the last dim (dim 4) vs macOS dim 3 — the 64-byte last-axis constraint (doc 01 §2.2) still dictating layout.

3.3 Compilation moves left#

Core ML compiled entirely on-device at first load — a UX hazard managed with folklore (async loads, warning screens). Core AI splits compilation: the expensive graph work happens at build time (coreai-build, per-platform artifacts in the bundle), leaving only device-specific finishing on first run, with explicit cache APIs for the rest. First-run latency went from "hide it" to "engineer it."

3.4 Verification becomes a product#

The argmaxtools testing stack (doc 06 §8) — PSNR thresholds, per-op dispatch dumps, layer-sensitivity profiling — was a third party filling a tooling vacuum. Core AI ships all of it: Python parity checks, a Debugger that traces any tensor back to the Python line that produced it, per-layer PSNR/MSE/MAE comparison for compression decisions, and Instruments events for specialization. The KB's "verify in CI, not by eyeball" lesson is now the official workflow.

4. Decision guide (mid-2026)#

Apple's only official signals: Core AI got all the WWDC26 airtime; Core ML got none but was not deprecated. Third-party reporting (9to5Mac, InfoQ, byteiota) converges on the same reading. Practical guide:

Situation	Recommendation
Must support devices below iOS/macOS 27	Core ML — no choice; Core AI is 27+ only
Existing, working Core ML pipeline	Stay put — no deprecation announced; migrate opportunistically
Classic ML (classifiers, regressors, small vision/audio)	Core ML — mature, broad reach, nothing to gain from migrating
New transformer/generative feature, 27+ target	Core AI — composite ops, states, AOT, model zoo, Foundation Models bridge
Custom LLM behind a chat/structured-output UX	Core AI (CoreAILanguageModel + LanguageModelSession) — or Foundation Models' built-in model if it suffices
Research / fine-tuning / custom training loops on Mac	MLX (train) → export via coreai-torch (ship)
Squeezing a model Apple's compiler doesn't handle well	Either — this KB's principles (docs 01–07) are the manual toolkit; on Core AI, express them as custom primitives/kernels

5. Migration mapping (Core ML → Core AI)#

Core ML concept	Core AI equivalent
torch.jit.trace + ct.convert	torch.export.export (+ dynamic_shapes) + TorchConverter().to_coreai()
.mlpackage / .mlmodelc	.aimodel (+ AOT-compiled variants via coreai-build)
MLModel / feature providers / MLMultiArray	AIModel / InferenceFunction / NDArray
MLState	state MutableViews passed to run(inputs:states:)
MLTensor glue code	largely absorbed by coreai-models Swift packages / Foundation Models session API
Multifunction .mlpackage	multi-entrypoint .aimodel
Enumerated/flexible shapes	torch.export.Dim dynamic dims
ct.optimize.coreml / ct.optimize.torch	coreai-opt (Quantizer, KMeansPalettizer, presets, YAML recipes, QAT)
compute_units=CPU_AND_NE + MLComputePlan	--platform iOS export + per-platform primitives; Core AI Instruments (op-level ANE-dispatch reporting: still an open question, doc 09 §5.4)
Xcode performance tab	Xcode model viewer (.aimodel) + Core AI Instruments + Debugger
Custom MIL composite ops	TorchMetalKernel DSL (MSL embedded in the asset)
Async load + hope (first-run)	AOT compile + AIModel.specialize + AIModelCache + Background Assets

6. What doesn't change#

Every hardware truth in doc 01 is framework-independent, and Apple's own Core AI iOS primitives prove it (doc 09 §3): channels-first 4D layouts, the last-axis alignment constraint, per-head attention computation, static shapes for the ANE, FP16 numerics, and palettization as the bandwidth-bound remedy. Frameworks are how you express the work; the ANE is why the work looks the way it does. That's the reason docs 01–07 stay relevant regardless of which runtime wins.