Run Noema models

Quick start

Pick the runtime you already use. Each command downloads the model from Hugging Face the first time it runs.

llama-server -hf NoemaAI-labs/Noema-1.5-4B-Fast-GGUF:Q4_K_M --jinja -c 40960 \
  --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0

llama-server serves an OpenAI-compatible API on port 8080 and vllm serve one on port 8000, so any OpenAI-compatible client can call the model. To answer without thinking, see Prompt format.

Choose a model and file

Fast suits iPhone and other tight budgets; Deep suits a Mac and hard problems. Each GGUF repository holds one file per quantization.

Noema 2B is deprecated; use Noema 1.5 2B instead. Its weights stay available.

Noema 1.5 4B Deep reasons at length. Give it a context of at least 40,960 tokens for general use and about 98,304 for competition mathematics and programming.

llama.cpp

Use a llama.cpp build with Qwen3.5 (qwen35) architecture support. -hf downloads the file from Hugging Face, and --jinja applies the chat template embedded in the GGUF. The sampling flags match each model card's recommended setting.

llama-server -hf NoemaAI-labs/Noema-1.5-4B-Deep-GGUF:Q5_K_M --jinja -c 98304 \
  --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0

For an interactive session in the terminal, run the same arguments with llama-cli. To turn thinking off on Noema 1.5 4B, add --chat-template-kwargs '{"enable_thinking": false}'.

Ollama

Ollama runs GGUF repositories from Hugging Face directly. Add the quantization after a colon; without one, Ollama picks Q4_K_M when the repository has it.

ollama run hf.co/NoemaAI-labs/Noema-1.5-4B-Deep-GGUF:Q5_K_M

Ollama chooses the chat template and default parameters itself. Set the recommended sampling inside the session, for example /set parameter temperature 0.6, /set parameter top_p 0.95 and /set parameter top_k 20 for Noema 1.5 4B. If replies show template tokens or do not stop, use llama.cpp, which applies the template embedded in the file.

LM Studio

LM Studio downloads GGUF repositories from Hugging Face, and MLX repositories on Apple silicon. Open a repository with its link, choose a file, download it and load the model.

# macOS. On Windows or Linux, open the link in a browser.
open "lmstudio://open_from_hf?model=NoemaAI-labs/Noema-1.5-4B-Fast-GGUF"

Swap in any repository from the model reference. Then set temperature, top-p and top-k in the model's settings to the values in Prompt format.

MLX

MLX runs on Apple silicon. Each Noema 1.5 model has BF16, 8-bit, 6-bit and 4-bit builds, converted with MLX-LM 0.31.3. Noema 2B has no MLX builds.

pip install "mlx-lm==0.31.3"

mlx_lm.generate --model NoemaAI-labs/Noema-1.5-4B-Deep-MLX-4bit \
  --prompt "How many positive divisors does 2024 have?" \
  --max-tokens 32768 --temp 0.6 --top-p 0.95 --top-k 20

To turn thinking off, add --chat-template-config '{"enable_thinking":false}'. For the other precisions, replace MLX-4bit with MLX-6bit, MLX-8bit or MLX-BF16.

Transformers and vLLM

Noema models need a Transformers release with Qwen3.5 support (qwen3_5_text). Each repository ships a generation config with its recommended sampling, so generate uses it by default.

• Noema 1.5 4B Deep: validated with Transformers 5.19.0.
• Noema 1.5 4B Fast: validated with Transformers 5.19.0.
• Noema 1.5 2B: validated with Transformers 5.12.1.
• Noema 2B (deprecated): validated with Transformers 5.12.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "NoemaAI-labs/Noema-1.5-4B-Deep"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": "How many positive divisors does 2024 have?"}]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    enable_thinking=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=32768)
text = tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)
print(text.rpartition("</think>")[2].strip())

With vLLM, the qwen3 reasoning parser returns the reasoning trace separately from the answer, and qwen3_coder parses tool calls. That is the configuration Noema 1.5 4B's tool calling was evaluated under.

Check a download

Noema 1.5 GGUF repositories include a SHA256SUMS file, and the Noema 1.5 Transformers and MLX repositories include WEIGHTS_SHA256SUMS. Compare a file's hash with the listed value before you rely on it. Noema 2B repositories have no checksum file.

hf download NoemaAI-labs/Noema-1.5-4B-Fast-GGUF \
  Noema-1.5-4B-Fast-Q4_K_M.gguf SHA256SUMS --local-dir noema
cd noema
shasum -a 256 Noema-1.5-4B-Fast-Q4_K_M.gguf
grep Q4_K_M SHA256SUMS

In the Noema app

Noema lists Noema 1.5 4B Deep and Fast first in Explore, with GGUF and MLX builds. See Running models locally to download one and choose a runtime preset.