Experimental local execution · Apple Silicon
Run compatible mixture-of-experts models whose complete weights would not normally fit in your device’s memory—without sending inference to a cloud API.
Local storage
Expert payloadsverified · checksummedUnified memory
Expert bankdevice-sized · reusableApple Silicon
Metal inferencerouted · localWhy MoE is different
A mixture-of-experts model may contain hundreds of experts, but each token activates only a small subset. Traditional inference still loads every expert into memory.
Overfit keeps shared layers and non-expert weights resident, while routed experts stay in a verified package on local storage. The complete parameter count becomes less tied to memory capacity; active parameters, storage speed, reuse, context, and bank size shape performance instead.
How Overfit works
Package
On Mac, Noema converts a compatible MoE GGUF into a .noema-paged package. Conversion does not retrain, prune, or replace the model’s weights—it changes how they are stored and loaded.
Size
At launch, Noema sizes the bank from available memory. A larger bank retains more experts and reduces storage reads; a smaller bank leaves room for context, compute buffers, and the rest of the app.
Route
Bank hits proceed immediately. Missing experts are read asynchronously from local storage, verified, and made visible to inference only after validation succeeds.
Execute
Overfit extends Noema’s llama.cpp and Metal runtime with paged MoE scheduling, fused routed reductions, and expert-major prompt processing.
Measured on your device
The Canary Test measures the exact package, Apple device, storage volume, available memory, native runtime, and app build. When any of them changes, the result expires.
Observed signals
Honest outcome
Faster follow-up conversations
Cold prompt processing can sweep broadly through the expert package. After a successful conversation, Noema can save a local prompt-state checkpoint and restore an unchanged system prompt and conversation prefix on a matching future launch.
Checkpoints are tied to the exact model and configuration. A cache failure never blocks the model; Noema falls back to a cold prompt.
Private by design
Relay may be recommended for an unsuitable model, but using it remains a separate choice.
Designed to fail safely
The honest tradeoff
A smaller model that fits completely in memory will usually start faster, generate faster, and use less energy. Overfit is for cases where access to a larger model’s capabilities matters more than minimum latency.
Current availability
The current runtime supports paged packages for Qwen 3 MoE, Qwen 3.5 MoE, Qwen 3.6 MoE, and Gemma 4 MoE. It is not a generic mechanism for dense models or arbitrary GGUF architectures.
Package creation is currently available on macOS. Conservative context limits and some auxiliary or multimodal workflows may continue to use the normal resident runtime.
Compatible models are published in the NoemaAI-labs/Noema-Overfit repository on Hugging Face.
Noema Overfit
Private, measurable, and under your control.
