Tutorial · Updated

How to run Qwen models locally on a Mac

Choose a Qwen model for your Mac's memory, then run it with Noema, Ollama, LM Studio, llama.cpp or MLX. Commands, file sizes and the settings that matter.

  • Mac

Short answer

Pick a Qwen model that fits your Mac's memory, then run it with whichever tool you prefer. On a 16 GB Mac, Qwen3.5-9B is the usual choice; with 8 GB use Qwen3.5-2B or 4B; with 24 GB or more you can run Qwen3.8-27B. All of these are Apache 2.0.

The quickest routes are ollama run qwen3.5:9b in a terminal, or an app with a model browser: search for Qwen in Noema's Explore tab or LM Studio's Discover tab, download, and chat. llama.cpp and MLX give you more control from the command line. Every option runs entirely on your Mac once the model is downloaded.

Which Qwen models you can run

Qwen is Alibaba's family of open-weight models. The ones people run locally in late 2026 are:

Open-weight Qwen families commonly run locally, from their Hugging Face model cards on November 18, 2026.
FamilySizesReleasedGood for
Qwen3.827BAugust 2026The newest general model; text and images; 262K-token context
Qwen3.627B, 35B-A3BApril 2026General use; the 35B-A3B is a mixture-of-experts model with about 3B active per token
Qwen3.50.8B, 2B, 4B, 9B, 27B, 35B-A3B, 122B-A10BFebruary and March 2026The widest range of sizes, from phones to workstations; text and images
Qwen30.6B, 1.7B, 4B, 8B, 14B, 32B, 30B-A3BApril 2025Older, text only, very widely supported
Qwen3-Coder30B-A3BJuly 2025Coding and agent tools

Every model in that table is released under Apache 2.0. A few of the largest Qwen3.8 releases use other licences, so read the model card before using one commercially. “A3B” means a mixture-of-experts model: 35B-A3B stores 35 billion parameters but uses about 3 billion for each token, so it needs the memory of a large model and runs closer to the speed of a small one.

Pick a size for your Mac's memory

A model has to fit in memory alongside macOS and your other apps. As a rule of thumb, a model needs about its file size plus 1 GB at a short context, and more as the context grows. Apple's MLX team measured this for Qwen3 at 4-bit with a 4,096-token prompt: the 8B used 5.61 GB, the 14B 9.16 GB and the 30B-A3B 17.31 GB, each a little more than the size of its file.

File sizes from each repository's file listing on Hugging Face (decimal GB). The last column is guidance derived from those sizes, not a measurement.
ModelGGUF Q4_K_MMLX 4-bitMac memory to plan for
Qwen3.5-2B1.28 GB1.72 GB8 GB
Qwen3.5-4B2.74 GB3.03 GB8 GB is tight; 16 GB is comfortable
Qwen3.5-9B5.68 GB5.95 GB16 GB
Qwen3.8-27B16.81 GB16.05 GB24 GB with a modest context; 32 GB or more for long contexts
Qwen3.6-35B-A3B21.17 GB20.40 GB32 GB or more
Qwen3-Coder-30B-A3B18.56 GB17.18 GB32 GB

Q4_K_M and 4-bit are the usual starting points: about a quarter of the full-precision size with a small loss in quality. An 8-bit file is nearly twice as large (9.53 GB for Qwen3.5-9B, 28.60 GB for Qwen3.5-27B) and is worth it only when you have memory to spare. To see how much memory your Mac has, open Apple menu › About This Mac.

Option 1: Noema, with no terminal

Noema is our app. It is free and open source, needs an Apple silicon Mac on macOS 26 or later, and runs both GGUF and MLX builds of Qwen.

  1. Download Noema from the App Store, or the Direct edition from NoemaAI.
  2. Open Explore and search for Qwen. Choose a model and a quantization.
  3. Read the estimated fit for your Mac, the licence and the capability badges, then download.
  4. Open Stored, check the model's settings and load it.
  5. Select it in a chat. The context gauge shows how much of the model's context you have used.

A vision model's GGUF needs a projector file as well; Noema lists it as a dependency. If a file is marked incompatible, choose another format or quantization. Running models locally explains the choices. The same steps work on iPhone and iPad with the smaller sizes, and NoemaAI's own Noema 1.5 4B is post-trained from Qwen3.5-4B; its model page reports the measured differences from the base model.

Option 2: Ollama

Install Ollama, then run a model by its library tag. The first run downloads it.

Ollama
# 8 GB Mac
ollama run qwen3.5:4b

# 16 GB Mac
ollama run qwen3.5:9b

# 24 GB or more
ollama run qwen3.8:27b

Two settings matter. Ollama picks a small default context on most Macs (4K tokens with under 24 GB of memory), which cuts off long chats and documents; raise it in the session with /set parameter num_ctx 16384, or set OLLAMA_CONTEXT_LENGTH before starting the server. To turn Qwen's thinking off, start with --think=false or type /set nothink. Use the library tags above: Unsloth notes that the Qwen3.5 GGUF repositories on Hugging Face do not load in Ollama because their vision projector is a separate file.

Option 3: LM Studio

LM Studio needs an Apple silicon Mac on macOS 14 or later. In the app, press Command-2 to open Discover, search for qwen3.5, choose a quantization and download. It offers both GGUF and MLX builds. From a terminal, its lms tool does the same:

LM Studio CLI
lms get qwen/qwen3.5-9b
lms load qwen/qwen3.5-9b -c 8192
lms chat qwen/qwen3.5-9b

# Serve it to other apps
lms server start

Option 4: llama.cpp

llama.cpp is the engine most of these apps use for GGUF files. Version 1.0, released in November 2026, has a single llama command. -hf downloads a repository from Hugging Face, takes Q4_K_M unless you name another quantization, and fetches the vision projector automatically.

llama.cpp
brew install llama.cpp

# Chat in the terminal
llama cli -hf unsloth/Qwen3.5-9B-GGUF:Q4_K_M

# Or serve an API and a web chat page on http://127.0.0.1:9931
llama serve -hf unsloth/Qwen3.5-9B-GGUF:Q4_K_M -c 16384 \
  --temp 0.7 --top-p 0.8 --top-k 20 --presence-penalty 1.5

If llama is not found, your build is older than 1.0: use llama-cli and llama-server with the same arguments, and expect the server on port 8080. To control thinking, add --reasoning off or --reasoning on.

Option 5: MLX

MLX is Apple's machine-learning framework for Apple silicon, and the mlx-community organisation on Hugging Face publishes Qwen conversions. Qwen3.5, 3.6 and 3.8 accept images, and their MLX model cards use mlx-vlm. The older text-only Qwen3 models use mlx-lm. Both need Python 3.11 or later.

pip install -U mlx-vlm
mlx_vlm.generate --model mlx-community/Qwen3.5-9B-4bit \
  --prompt "Explain unified memory in two sentences."

Set the temperature yourself with mlx-lm. It defaults to 0, which is greedy decoding, and Qwen's model cards advise against that.

Settings that change the results

  • Thinking. Qwen3.5 models of 4B and larger, Qwen3.6 and Qwen3.8 reason before answering by default; the 0.8B and 2B do not. Thinking improves hard answers and costs time and tokens. The /think and /no_think switches in a prompt work only on the older Qwen3; for newer models use the tool's own switch, as shown above. Some converted files change the default, so check what your download does.
  • Sampling. Qwen's cards recommend temperature 1.0, top-p 0.95 and top-k 20 when thinking (0.6 for Qwen3), and temperature 0.7, top-p 0.8 and top-k 20 when not. Apps usually apply a model's defaults; command-line tools may not.
  • Context. The newer models accept up to 262,144 tokens, but context uses memory. Start around 8K to 16K and raise it if a long document or a thinking model needs more.
  • Quantization. If a model is slow or the Mac starts swapping, try a smaller model before a smaller quantization of the same one.

Use your local Qwen from other apps

Each tool can serve the model to editors, scripts and coding agents through an OpenAI-style API on your own machine.

ToolHow to start itAddress
NoemaStart the Local API server on the Developer pageShown in the app, with the model identifier
OllamaRuns with the app or ollama servehttp://localhost:11434/v1
LM Studiolms server start, or the Developer tabhttp://localhost:1234/v1
llama.cppllama servehttp://127.0.0.1:9931/v1
mlx-lmmlx_lm.serverhttp://localhost:8080/v1

Noema's server also accepts the Responses and Anthropic Messages formats. The Local API quickstart has ready-made requests for curl and the OpenAI and Anthropic SDKs.

How fast will it be?

It depends on the chip, the model, the quantization and the context, and we have no figures with named hardware that we would repeat here. Measure on your own Mac: Noema's Benchmarking Center reports prompt and generation speed with peak memory, ollama run with --verbose prints tokens per second after each reply, and LM Studio shows the rate under each response. As a guide to what to expect, a model that fits in memory with room to spare is responsive, and one that makes the Mac swap is not.

Frequently asked questions

What is the easiest way to run Qwen on a Mac?

Use an app with a model browser. In Noema, open Explore, search for Qwen, download a size that fits and chat; LM Studio's Discover tab works the same way. If you prefer a terminal, install Ollama and run ollama run qwen3.5:9b.

Which Qwen model should I run on a 16 GB Mac?

Qwen3.5-9B at 4-bit. Its Q4_K_M file is 5.68 GB, which leaves room for macOS and a useful context. On 8 GB use Qwen3.5-2B or 4B; on 24 GB or more, Qwen3.8-27B.

Can I run Qwen on a Mac with 8 GB of memory?

Yes, with the small sizes. Qwen3.5-2B (a 1.28 GB file at Q4_K_M) runs comfortably, and Qwen3.5-4B (2.74 GB) fits but leaves little room, so keep the context short and close other apps.

Should I use GGUF or MLX for Qwen on Apple silicon?

Both work well. GGUF runs in the most tools (llama.cpp, Ollama, LM Studio, Noema) and has the widest choice of quantizations. MLX is built for Apple silicon and runs in LM Studio, Noema and the MLX command-line tools. Try both on your own Mac and keep the faster one.

Does Qwen run offline?

Yes. After the model file is downloaded, generation happens on your Mac with no connection, and prompts are not sent anywhere. Only the download, and optional features such as web search, need the internet.

How do I turn off Qwen's thinking?

Use the tool's switch. In Ollama, run with --think=false or type /set nothink. In llama.cpp, add --reasoning off. With mlx-vlm, add --no-enable-thinking. In Noema and LM Studio, use the model's settings. The /no_think prompt switch works only on the older Qwen3 models.

Can I run Qwen on an iPhone or iPad?

Yes, the smaller sizes. Qwen3.5-2B and 4B fit recent iPhones and iPads in apps such as Noema. See which local AI models fit your device.

About this guide

NoemaAI makes Noema, one of the apps described here, so read this as a maker's guide and not an independent review. What it says about Noema matches the documentation. What it says about other products comes from their own listings, sites, repositories and documentation, linked below, as they read on . These products change often. If something here is wrong or out of date, write to [email protected] and we will correct it.

Sources

This guide as Markdown

Noema is free on the App Store.

One download for iPhone, iPad, Mac and Apple Vision Pro. Local chat needs no account.