---
title: "How to run Qwen models locally on a Mac"
description: "Choose a Qwen model for your Mac's memory, then run it with Noema, Ollama, LM Studio, llama.cpp or MLX. Commands, file sizes and the settings that matter."
type: "Tutorial"
publisher: "NoemaAI"
published: "2026-11-18"
updated: "2026-11-18"
facts_checked: "2026-11-18"
canonical: "https://noemaai.com/guides/run-qwen-locally-on-mac"
---

# How to run Qwen models locally on a Mac

Choose a Qwen model for your Mac's memory, then run it with Noema, Ollama, LM Studio, llama.cpp or MLX. Commands, file sizes and the settings that matter.

## Short answer

Pick a Qwen model that fits your Mac's memory, then run it with whichever tool you prefer. On a 16 GB Mac, **Qwen3.5-9B** is the usual choice; with 8 GB use **Qwen3.5-2B** or **4B**; with 24 GB or more you can run **Qwen3.8-27B**. All of these are Apache 2.0.

The quickest routes are `ollama run qwen3.5:9b` in a terminal, or an app with a model browser: search for Qwen in **Noema**'s Explore tab or **LM Studio**'s Discover tab, download, and chat. **llama.cpp** and **MLX** give you more control from the command line. Every option runs entirely on your Mac once the model is downloaded.

## Which Qwen models you can run

Qwen is Alibaba's family of open-weight models. The ones people run locally in late 2026 are:

| Family | Sizes | Released | Good for |
| --- | --- | --- | --- |
| Qwen3.8 | 27B | August 2026 | The newest general model; text and images; 262K-token context |
| Qwen3.6 | 27B, 35B-A3B | April 2026 | General use; the 35B-A3B is a mixture-of-experts model with about 3B active per token |
| Qwen3.5 | 0.8B, 2B, 4B, 9B, 27B, 35B-A3B, 122B-A10B | February and March 2026 | The widest range of sizes, from phones to workstations; text and images |
| Qwen3 | 0.6B, 1.7B, 4B, 8B, 14B, 32B, 30B-A3B | April 2025 | Older, text only, very widely supported |
| Qwen3-Coder | 30B-A3B | July 2025 | Coding and agent tools |

_Open-weight Qwen families commonly run locally, from their Hugging Face model cards on November 18, 2026._

Every model in that table is released under Apache 2.0. A few of the largest Qwen3.8 releases use other licences, so read the model card before using one commercially. “A3B” means a mixture-of-experts model: 35B-A3B stores 35 billion parameters but uses about 3 billion for each token, so it needs the memory of a large model and runs closer to the speed of a small one.

## Pick a size for your Mac's memory

A model has to fit in memory alongside macOS and your other apps. As a rule of thumb, a model needs about its file size plus 1 GB at a short context, and more as the context grows. Apple's MLX team measured this for Qwen3 at 4-bit with a 4,096-token prompt: the 8B used 5.61 GB, the 14B 9.16 GB and the 30B-A3B 17.31 GB, each a little more than the size of its file.

| Model | GGUF Q4_K_M | MLX 4-bit | Mac memory to plan for |
| --- | --- | --- | --- |
| Qwen3.5-2B | 1.28 GB | 1.72 GB | 8 GB |
| Qwen3.5-4B | 2.74 GB | 3.03 GB | 8 GB is tight; 16 GB is comfortable |
| Qwen3.5-9B | 5.68 GB | 5.95 GB | 16 GB |
| Qwen3.8-27B | 16.81 GB | 16.05 GB | 24 GB with a modest context; 32 GB or more for long contexts |
| Qwen3.6-35B-A3B | 21.17 GB | 20.40 GB | 32 GB or more |
| Qwen3-Coder-30B-A3B | 18.56 GB | 17.18 GB | 32 GB |

_File sizes from each repository's file listing on Hugging Face (decimal GB). The last column is guidance derived from those sizes, not a measurement._

Q4_K_M and 4-bit are the usual starting points: about a quarter of the full-precision size with a small loss in quality. An 8-bit file is nearly twice as large (9.53 GB for Qwen3.5-9B, 28.60 GB for Qwen3.5-27B) and is worth it only when you have memory to spare. To see how much memory your Mac has, open Apple menu › About This Mac.

## Option 1: Noema, with no terminal

[Noema](https://noemaai.com/app) is our app. It is free and [open source](https://github.com/noemaai-labs/noema), needs an Apple silicon Mac on macOS 26 or later, and runs both GGUF and MLX builds of Qwen.

1. [Download Noema](https://noemaai.com/download) from the App Store, or the Direct edition from NoemaAI.
2. Open **Explore** and search for Qwen. Choose a model and a quantization.
3. Read the estimated fit for your Mac, the licence and the capability badges, then download.
4. Open **Stored**, check the model's settings and load it.
5. Select it in a chat. The context gauge shows how much of the model's context you have used.

A vision model's GGUF needs a projector file as well; Noema lists it as a dependency. If a file is marked incompatible, choose another format or quantization. [Running models locally](https://noemaai.com/docs/running-llms) explains the choices. The same steps work on iPhone and iPad with the smaller sizes, and NoemaAI's own [Noema 1.5 4B](https://noemaai.com/models/noema-1-5-4b) is post-trained from Qwen3.5-4B; its model page reports the measured differences from the base model.

## Option 2: Ollama

Install [Ollama](https://ollama.com/download), then run a model by its library tag. The first run downloads it.

```bash
# 8 GB Mac
ollama run qwen3.5:4b

# 16 GB Mac
ollama run qwen3.5:9b

# 24 GB or more
ollama run qwen3.8:27b
```

Two settings matter. Ollama picks a small default context on most Macs (4K tokens with under 24 GB of memory), which cuts off long chats and documents; raise it in the session with `/set parameter num_ctx 16384`, or set `OLLAMA_CONTEXT_LENGTH` before starting the server. To turn Qwen's thinking off, start with `--think=false` or type `/set nothink`. Use the library tags above: Unsloth notes that the Qwen3.5 GGUF repositories on Hugging Face do not load in Ollama because their vision projector is a separate file.

## Option 3: LM Studio

[LM Studio](https://lmstudio.ai) needs an Apple silicon Mac on macOS 14 or later. In the app, press Command-2 to open Discover, search for `qwen3.5`, choose a quantization and download. It offers both GGUF and MLX builds. From a terminal, its `lms` tool does the same:

```bash
lms get qwen/qwen3.5-9b
lms load qwen/qwen3.5-9b -c 8192
lms chat qwen/qwen3.5-9b

# Serve it to other apps
lms server start
```

## Option 4: llama.cpp

[llama.cpp](https://github.com/ggml-org/llama.cpp) is the engine most of these apps use for GGUF files. Version 1.0, released in November 2026, has a single `llama` command. `-hf` downloads a repository from Hugging Face, takes Q4_K_M unless you name another quantization, and fetches the vision projector automatically.

```bash
brew install llama.cpp

# Chat in the terminal
llama cli -hf unsloth/Qwen3.5-9B-GGUF:Q4_K_M

# Or serve an API and a web chat page on http://127.0.0.1:9931
llama serve -hf unsloth/Qwen3.5-9B-GGUF:Q4_K_M -c 16384 \
  --temp 0.7 --top-p 0.8 --top-k 20 --presence-penalty 1.5
```

If `llama` is not found, your build is older than 1.0: use `llama-cli` and `llama-server` with the same arguments, and expect the server on port 8080. To control thinking, add `--reasoning off` or `--reasoning on`.

## Option 5: MLX

MLX is Apple's machine-learning framework for Apple silicon, and the `mlx-community` organisation on Hugging Face publishes Qwen conversions. Qwen3.5, 3.6 and 3.8 accept images, and their MLX model cards use `mlx-vlm`. The older text-only Qwen3 models use `mlx-lm`. Both need Python 3.11 or later.

**Qwen3.5 and later**

```bash
pip install -U mlx-vlm
mlx_vlm.generate --model mlx-community/Qwen3.5-9B-4bit \
  --prompt "Explain unified memory in two sentences."
```

**Qwen3**

```bash
pip install -U mlx-lm
mlx_lm.generate --model mlx-community/Qwen3-8B-4bit \
  --prompt "Explain unified memory in two sentences." \
  --temp 0.7 --max-tokens 1024

# Interactive chat, or an OpenAI-style server on port 8080
mlx_lm.chat --model mlx-community/Qwen3-8B-4bit
mlx_lm.server --model mlx-community/Qwen3-8B-4bit
```

Set the temperature yourself with `mlx-lm`. It defaults to 0, which is greedy decoding, and Qwen's model cards advise against that.

## Settings that change the results

- **Thinking.** Qwen3.5 models of 4B and larger, Qwen3.6 and Qwen3.8 reason before answering by default; the 0.8B and 2B do not. Thinking improves hard answers and costs time and tokens. The `/think` and `/no_think` switches in a prompt work only on the older Qwen3; for newer models use the tool's own switch, as shown above. Some converted files change the default, so check what your download does.
- **Sampling.** Qwen's cards recommend temperature 1.0, top-p 0.95 and top-k 20 when thinking (0.6 for Qwen3), and temperature 0.7, top-p 0.8 and top-k 20 when not. Apps usually apply a model's defaults; command-line tools may not.
- **Context.** The newer models accept up to 262,144 tokens, but context uses memory. Start around 8K to 16K and raise it if a long document or a thinking model needs more.
- **Quantization.** If a model is slow or the Mac starts swapping, try a smaller model before a smaller quantization of the same one.

## Use your local Qwen from other apps

Each tool can serve the model to editors, scripts and coding agents through an OpenAI-style API on your own machine.

| Tool | How to start it | Address |
| --- | --- | --- |
| Noema | Start the Local API server on the Developer page | Shown in the app, with the model identifier |
| Ollama | Runs with the app or `ollama serve` | `http://localhost:11434/v1` |
| LM Studio | `lms server start`, or the Developer tab | `http://localhost:1234/v1` |
| llama.cpp | `llama serve` | `http://127.0.0.1:9931/v1` |
| mlx-lm | `mlx_lm.server` | `http://localhost:8080/v1` |

Noema's server also accepts the Responses and Anthropic Messages formats. The [Local API quickstart](https://noemaai.com/docs/local-api-quickstart) has ready-made requests for curl and the OpenAI and Anthropic SDKs.

## How fast will it be?

It depends on the chip, the model, the quantization and the context, and we have no figures with named hardware that we would repeat here. Measure on your own Mac: Noema's [Benchmarking Center](https://noemaai.com/docs/mac-tools-and-performance) reports prompt and generation speed with peak memory, `ollama run` with `--verbose` prints tokens per second after each reply, and LM Studio shows the rate under each response. As a guide to what to expect, a model that fits in memory with room to spare is responsive, and one that makes the Mac swap is not.

## Frequently asked questions

### What is the easiest way to run Qwen on a Mac?

Use an app with a model browser. In Noema, open Explore, search for Qwen, download a size that fits and chat; LM Studio's Discover tab works the same way. If you prefer a terminal, install Ollama and run `ollama run qwen3.5:9b`.

### Which Qwen model should I run on a 16 GB Mac?

Qwen3.5-9B at 4-bit. Its Q4_K_M file is 5.68 GB, which leaves room for macOS and a useful context. On 8 GB use Qwen3.5-2B or 4B; on 24 GB or more, Qwen3.8-27B.

### Can I run Qwen on a Mac with 8 GB of memory?

Yes, with the small sizes. Qwen3.5-2B (a 1.28 GB file at Q4_K_M) runs comfortably, and Qwen3.5-4B (2.74 GB) fits but leaves little room, so keep the context short and close other apps.

### Should I use GGUF or MLX for Qwen on Apple silicon?

Both work well. GGUF runs in the most tools (llama.cpp, Ollama, LM Studio, Noema) and has the widest choice of quantizations. MLX is built for Apple silicon and runs in LM Studio, Noema and the MLX command-line tools. Try both on your own Mac and keep the faster one.

### Does Qwen run offline?

Yes. After the model file is downloaded, generation happens on your Mac with no connection, and prompts are not sent anywhere. Only the download, and optional features such as web search, need the internet.

### How do I turn off Qwen's thinking?

Use the tool's switch. In Ollama, run with `--think=false` or type `/set nothink`. In llama.cpp, add `--reasoning off`. With mlx-vlm, add `--no-enable-thinking`. In Noema and LM Studio, use the model's settings. The `/no_think` prompt switch works only on the older Qwen3 models.

### Can I run Qwen on an iPhone or iPad?

Yes, the smaller sizes. Qwen3.5-2B and 4B fit recent iPhones and iPads in apps such as Noema. See [which local AI models fit your device](https://noemaai.com/guides/which-local-models-fit-your-device).

## Sources

- [Qwen3.8: repository and release notes](https://github.com/QwenLM/Qwen3.8)
- [Qwen3.8-27B: model card](https://huggingface.co/Qwen/Qwen3.8-27B)
- [Qwen3.6-35B-A3B: model card](https://huggingface.co/Qwen/Qwen3.6-35B-A3B)
- [Qwen3.5-9B: model card](https://huggingface.co/Qwen/Qwen3.5-9B)
- [Qwen3.5-4B: model card](https://huggingface.co/Qwen/Qwen3.5-4B)
- [Qwen3-8B: model card](https://huggingface.co/Qwen/Qwen3-8B)
- [Qwen3.5-9B GGUF files (Unsloth)](https://huggingface.co/unsloth/Qwen3.5-9B-GGUF/tree/main)
- [Qwen3.8-27B GGUF files (lmstudio-community)](https://huggingface.co/lmstudio-community/Qwen3.8-27B-GGUF/tree/main)
- [Qwen3.5-9B MLX 4-bit (mlx-community)](https://huggingface.co/mlx-community/Qwen3.5-9B-4bit)
- [Unsloth: running Qwen3.5](https://unsloth.ai/docs/models/qwen3.5)
- [Unsloth: running Qwen3.8](https://unsloth.ai/docs/models/qwen3.8)
- [Apple Machine Learning Research: exploring LLMs with MLX and M5](https://machinelearning.apple.com/research/exploring-llms-mlx-m5)
- [Ollama library: qwen3.5 tags](https://ollama.com/library/qwen3.5/tags)
- [Ollama library: qwen3.8 tags](https://ollama.com/library/qwen3.8/tags)
- [Ollama: context length](https://docs.ollama.com/context-length)
- [Ollama: thinking](https://ollama.com/blog/thinking)
- [LM Studio: Qwen3.5 in the model catalog](https://lmstudio.ai/models/qwen3.5)
- [LM Studio: system requirements](https://lmstudio.ai/docs/app/system-requirements)
- [LM Studio: download a model](https://lmstudio.ai/docs/app/basics/download-model)
- [llama.cpp: repository and quick start](https://github.com/ggml-org/llama.cpp)
- [llama.cpp: server documentation](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md)
- [llama.cpp: Homebrew formula](https://formulae.brew.sh/formula/llama.cpp)
- [mlx-lm: repository](https://github.com/ml-explore/mlx-lm)
- [mlx-vlm: repository](https://github.com/Blaizzy/mlx-vlm)
