---
title: "Run Noema models"
description: "Copyable commands to run Noema models in llama.cpp, Ollama, LM Studio, MLX, Transformers and vLLM, with the file each model card recommends."
version: "Noema 1.5 · Noema 2B"
applies_to: "llama.cpp · Ollama · LM Studio · MLX · Transformers · vLLM"
reviewed: "2026-10-23"
canonical: "https://noemaai.com/docs/run-noema-models"
---

# Run Noema models

Copyable commands for llama.cpp, Ollama, LM Studio, MLX and Transformers, with the file each model card recommends.

NoemaAI publishes every model on [Hugging Face](https://huggingface.co/NoemaAI-labs). Each release has a Transformers repository with BF16 weights and a GGUF repository, and Noema 1.5 models also have MLX repositories for Apple silicon. Noema 2B has no MLX builds.

The quick start runs Noema 1.5 4B Fast. Each runtime section below has the command for every model, and the [model reference](https://noemaai.com/docs/models) lists every repository.

## Quick start

Pick the runtime you already use. Each command downloads the model from Hugging Face the first time it runs.

**llama.cpp**

```bash
llama-server -hf NoemaAI-labs/Noema-1.5-4B-Fast-GGUF:Q4_K_M --jinja -c 40960 \
  --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0
```

**Ollama**

```bash
ollama run hf.co/NoemaAI-labs/Noema-1.5-4B-Fast-GGUF:Q4_K_M
```

**LM Studio**

```bash
# macOS. On Windows or Linux, open the link in a browser.
open "lmstudio://open_from_hf?model=NoemaAI-labs/Noema-1.5-4B-Fast-GGUF"
```

**MLX**

```bash
pip install "mlx-lm==0.31.3"

mlx_lm.generate --model NoemaAI-labs/Noema-1.5-4B-Fast-MLX-4bit \
  --prompt "How many positive divisors does 2024 have?" \
  --max-tokens 32768 --temp 0.6 --top-p 0.95 --top-k 20
```

**Transformers**

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "NoemaAI-labs/Noema-1.5-4B-Fast"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": "How many positive divisors does 2024 have?"}]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    enable_thinking=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=32768)
text = tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)
print(text.rpartition("</think>")[2].strip())
```

**vLLM**

```bash
vllm serve NoemaAI-labs/Noema-1.5-4B-Fast \
  --max-model-len 40960 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder
```

`llama-server` serves an OpenAI-compatible API on port 8080 and `vllm serve` one on port 8000, so any OpenAI-compatible client can call the model. To answer without thinking, see [Prompt format](https://noemaai.com/docs/prompt-format).

## Choose a model and file

Fast suits iPhone and other tight budgets; Deep suits a Mac and hard problems. Each GGUF repository holds one file per quantization.

| Model | GGUF repository | Files | Which file to use |
| --- | --- | --- | --- |
| Noema 1.5 4B Deep | [NoemaAI-labs/Noema-1.5-4B-Deep-GGUF](https://huggingface.co/NoemaAI-labs/Noema-1.5-4B-Deep-GGUF) | BF16, Q8_0, Q6_K, Q5_K_M, Q4_K_M | Q5_K_M is the smallest Deep build with no measurable loss against BF16; Q4_K_M scores measurably lower on non-thinking GPQA Diamond. |
| Noema 1.5 4B Fast | [NoemaAI-labs/Noema-1.5-4B-Fast-GGUF](https://huggingface.co/NoemaAI-labs/Noema-1.5-4B-Fast-GGUF) | BF16, Q8_0, Q6_K, Q5_K_M, Q4_K_M | Q4_K_M is the compact default for laptops and phones; no quantized Fast build is measurably worse than BF16. |
| Noema 1.5 2B | [NoemaAI-labs/Noema-1.5-2B-GGUF](https://huggingface.co/NoemaAI-labs/Noema-1.5-2B-GGUF) | BF16, Q8_0, Q6_K, Q5_K_M, Q4_K_M | Q4_K_M is the recommended compact default. |
| Noema 2B (deprecated) | [NoemaAI-labs/Noema-2B-GGUF](https://huggingface.co/NoemaAI-labs/Noema-2B-GGUF) | F16, Q8_0, Q6_K, Q4_K_M | Q4_K_M if you still run this model. |

Noema 2B is deprecated; use Noema 1.5 2B instead. Its weights stay available.

> Noema 1.5 4B Deep reasons at length. Give it a context of at least 40,960 tokens for general use and about 98,304 for competition mathematics and programming.

## llama.cpp

Use a llama.cpp build with Qwen3.5 (`qwen35`) architecture support. `-hf` downloads the file from Hugging Face, and `--jinja` applies the chat template embedded in the GGUF. The sampling flags match each model card's recommended setting.

**Noema 1.5 4B Deep**

```bash
llama-server -hf NoemaAI-labs/Noema-1.5-4B-Deep-GGUF:Q5_K_M --jinja -c 98304 \
  --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0
```

**Noema 1.5 4B Fast**

```bash
llama-server -hf NoemaAI-labs/Noema-1.5-4B-Fast-GGUF:Q4_K_M --jinja -c 40960 \
  --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0
```

**Noema 1.5 2B**

```bash
llama-server -hf NoemaAI-labs/Noema-1.5-2B-GGUF:Q4_K_M --jinja \
  --temp 0 --chat-template-kwargs '{"enable_thinking": false}'
```

**Noema 2B (deprecated)**

```bash
llama-server -hf NoemaAI-labs/Noema-2B-GGUF:Q4_K_M --jinja -c 4096 \
  --temp 0.7 --top-p 0.8 --top-k 20 --presence-penalty 1.5
```

For an interactive session in the terminal, run the same arguments with `llama-cli`. To turn thinking off on Noema 1.5 4B, add `--chat-template-kwargs '{"enable_thinking": false}'`.

## Ollama

Ollama runs GGUF repositories from Hugging Face directly. Add the quantization after a colon; without one, Ollama picks Q4_K_M when the repository has it.

**Noema 1.5 4B Deep**

```bash
ollama run hf.co/NoemaAI-labs/Noema-1.5-4B-Deep-GGUF:Q5_K_M
```

**Noema 1.5 4B Fast**

```bash
ollama run hf.co/NoemaAI-labs/Noema-1.5-4B-Fast-GGUF:Q4_K_M
```

**Noema 1.5 2B**

```bash
ollama run hf.co/NoemaAI-labs/Noema-1.5-2B-GGUF:Q4_K_M
```

**Noema 2B (deprecated)**

```bash
ollama run hf.co/NoemaAI-labs/Noema-2B-GGUF:Q4_K_M
```

Ollama chooses the chat template and default parameters itself. Set the recommended sampling inside the session, for example `/set parameter temperature 0.6`, `/set parameter top_p 0.95` and `/set parameter top_k 20` for Noema 1.5 4B. If replies show template tokens or do not stop, use llama.cpp, which applies the template embedded in the file.

## LM Studio

LM Studio downloads GGUF repositories from Hugging Face, and MLX repositories on Apple silicon. Open a repository with its link, choose a file, download it and load the model.

**GGUF**

```bash
# macOS. On Windows or Linux, open the link in a browser.
open "lmstudio://open_from_hf?model=NoemaAI-labs/Noema-1.5-4B-Fast-GGUF"
```

**MLX**

```bash
# macOS. On Windows or Linux, open the link in a browser.
open "lmstudio://open_from_hf?model=NoemaAI-labs/Noema-1.5-4B-Fast-MLX-4bit"
```

Swap in any repository from the [model reference](https://noemaai.com/docs/models). Then set temperature, top-p and top-k in the model's settings to the values in [Prompt format](https://noemaai.com/docs/prompt-format).

## MLX

MLX runs on Apple silicon. Each Noema 1.5 model has BF16, 8-bit, 6-bit and 4-bit builds, converted with MLX-LM 0.31.3. Noema 2B has no MLX builds.

**Noema 1.5 4B Deep**

```bash
pip install "mlx-lm==0.31.3"

mlx_lm.generate --model NoemaAI-labs/Noema-1.5-4B-Deep-MLX-4bit \
  --prompt "How many positive divisors does 2024 have?" \
  --max-tokens 32768 --temp 0.6 --top-p 0.95 --top-k 20
```

**Noema 1.5 4B Fast**

```bash
pip install "mlx-lm==0.31.3"

mlx_lm.generate --model NoemaAI-labs/Noema-1.5-4B-Fast-MLX-4bit \
  --prompt "How many positive divisors does 2024 have?" \
  --max-tokens 32768 --temp 0.6 --top-p 0.95 --top-k 20
```

**Noema 1.5 2B**

```bash
pip install "mlx-lm==0.31.3"

mlx_lm.generate --model NoemaAI-labs/Noema-1.5-2B-MLX-4bit \
  --prompt "Write a Python function that merges two sorted lists." \
  --max-tokens 1024 --temp 0 \
  --chat-template-config '{"enable_thinking":false}'
```

To turn thinking off, add `--chat-template-config '{"enable_thinking":false}'`. For the other precisions, replace `MLX-4bit` with `MLX-6bit`, `MLX-8bit` or `MLX-BF16`.

## Transformers and vLLM

Noema models need a Transformers release with Qwen3.5 support (`qwen3_5_text`). Each repository ships a generation config with its recommended sampling, so `generate` uses it by default.

- **Noema 1.5 4B Deep:** validated with Transformers 5.19.0.
- **Noema 1.5 4B Fast:** validated with Transformers 5.19.0.
- **Noema 1.5 2B:** validated with Transformers 5.12.1.
- **Noema 2B (deprecated):** validated with Transformers 5.12.

**Noema 1.5 4B Deep**

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "NoemaAI-labs/Noema-1.5-4B-Deep"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": "How many positive divisors does 2024 have?"}]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    enable_thinking=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=32768)
text = tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)
print(text.rpartition("</think>")[2].strip())
```

**Noema 1.5 4B Fast**

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "NoemaAI-labs/Noema-1.5-4B-Fast"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": "How many positive divisors does 2024 have?"}]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    enable_thinking=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=32768)
text = tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)
print(text.rpartition("</think>")[2].strip())
```

**Noema 1.5 2B**

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "NoemaAI-labs/Noema-1.5-2B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": "Write a Python function that merges two sorted lists."}]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    enable_thinking=False,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=1024)
text = tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)
print(text.strip())
```

**Noema 2B (deprecated)**

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "NoemaAI-labs/Noema-2B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": "A farmer has 17 red apples and 25 green apples. How many in total?"}]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    enable_thinking=False,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512)
text = tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)
print(text.strip())
```

**vLLM: Noema 1.5 4B Deep**

```bash
vllm serve NoemaAI-labs/Noema-1.5-4B-Deep \
  --max-model-len 98304 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder
```

**vLLM: Noema 1.5 4B Fast**

```bash
vllm serve NoemaAI-labs/Noema-1.5-4B-Fast \
  --max-model-len 40960 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder
```

With vLLM, the `qwen3` reasoning parser returns the reasoning trace separately from the answer, and `qwen3_coder` parses tool calls. That is the configuration Noema 1.5 4B's tool calling was evaluated under.

## Check a download

Noema 1.5 GGUF repositories include a `SHA256SUMS` file, and the Noema 1.5 Transformers and MLX repositories include `WEIGHTS_SHA256SUMS`. Compare a file's hash with the listed value before you rely on it. Noema 2B repositories have no checksum file.

```bash
hf download NoemaAI-labs/Noema-1.5-4B-Fast-GGUF \
  Noema-1.5-4B-Fast-Q4_K_M.gguf SHA256SUMS --local-dir noema
cd noema
shasum -a 256 Noema-1.5-4B-Fast-Q4_K_M.gguf
grep Q4_K_M SHA256SUMS
```

## In the Noema app

Noema lists Noema 1.5 4B Deep and Fast first in Explore, with GGUF and MLX builds. See [Running models locally](https://noemaai.com/docs/running-llms) to download one and choose a runtime preset.

## Related documentation
- [Noema models](https://noemaai.com/docs/models)
- [Prompt format and sampling](https://noemaai.com/docs/prompt-format)
- [Model licenses](https://noemaai.com/docs/model-licenses)
- [Local API quickstart](https://noemaai.com/docs/local-api-quickstart)