Local runtime choices
| Runtime | Best fit | What to know |
|---|---|---|
| GGUF | Broad compatibility and detailed tuning | Flexible quantizations, context controls, projectors, and advanced runtime options. |
| MLX | Apple-silicon optimization | Efficient Apple-native execution with a more opinionated settings surface. |
| ExecuTorch | Packaged mobile deployments | Compatibility and available controls depend on the exported model bundle. |
| CML / Core ML | Apple neural-engine and packaged model workflows | Dependencies and supported capabilities are model-specific. |
| AFM | Apple’s built-in Foundation Model | Availability depends on device and OS; it is activated, not downloaded like a model file. |
Choose by fit, not parameter count alone
- RAM fit includes model weights, key-value cache, context length, runtime overhead, and vision or audio dependencies.
- A lighter quantization reduces memory use but can change quality and speed.
- Long context can consume more memory than expected even with a small model.
- Vision, tool calling, reasoning, and audio support are separate capabilities; inspect badges and model notes before downloading.
Load and run a model
- Open Explore and choose a compatible model and quantization.
- Review license, provenance, capability badges, dependencies, and estimated fit.
- Download the model and any required projector or companion files.
- Open Stored, review its runtime settings, and load it.
- Select it in Chat and monitor the context gauge and runtime receipt.
Performance and energy
Use a runtime preset instead of enabling system Low Power Mode as a model-tuning strategy. Battery Saver reduces the workload through Noema’s own settings; Balanced is a good default; Max Speed favors throughput when heat and power use are acceptable.
- Reduce context length before assuming the model is too large.
- Close memory-heavy apps and avoid loading multiple large local models unless the fit estimate allows it.
- Benchmark on the device where the model will actually run.