This is Olmo 3 32B Think converted to MLX 4-bit quantized format for efficient inference on Apple Silicon hardware. This variant is optimized for speed with minimal memory requirements while preserving Olmo's signature thinking capabilities.
MLX Model Variants - Complete Collection
Choose the best variant for your hardware and performance needs:
Optimized Performance: 20.96 tokens/sec (2.8x faster than bf16)
Minimal Memory: 18.4GB RAM usage (3.5x less than bf16)
Device Compatibility: Runs on 32GB+ Apple Silicon devices
Preserved Thinking: Full Olmo <think> capabilities intact
Quick Start
Command Line Interface
bash
1# Interactive chat (recommended)2uvx --from mlx-lm mlx_lm.chat --model Plurigrid/Olmo-3-32B-Think-MLX-4bit
34# Generate text with thinking5uvx --from mlx-lm mlx_lm.generate --model Plurigrid/Olmo-3-32B-Think-MLX-4bit \6 --prompt "Who would win in a fight - a dinosaur or a cow named Moo Moo?"\7 --max-tokens 500 --temp 0.6
Olmo 3 is a new family of open language models designed to enable the science of language models. The Think variant uses long chain-of-thought thinking to improve reasoning tasks like math and coding.
Key Capabilities:
Step-by-step reasoning with visible <think> tags
Advanced mathematical and scientific reasoning
Tool-use and multi-turn conversations
Research-grade analysis and problem-solving
Chat Template
Default System Message
The default system prompt for this model is:
<|im_start|>system
You are a helpful AI assistant.<|im_end|>
Chat Format
The chat template for this model is formatted as:
<|im_start|>system
You are a helpful AI assistant.
<|im_start|>user
Who would win in a fight - a dinosaur or a cow named Moo Moo?<|im_end|>
<|im_start|>assistant
<think>Okay, so the question is who would win in a fight between a dinosaur and a cow named Moo Moo.
Hmm, first I need to break this down. Let me think about the different factors involved here..... </think>
Moo Moo the cow would certainly win.
<|endoftext|>
Evaluation Results
Results from the original Olmo-3-32B-Think model (quality preserved in 4-bit variant):
Benchmark
Olmo 3 Think 32B
Qwen 3 32B
Gemma 3 27B Instruct
DeepSeek-R1-Distill-Qwen-32B
Math
MATH
96.1
95.4
87.4
92.6
AIME 2024
76.8
80.8
28.9
70.3
AIME 2025
72.5
70.9
22.9
56.3
Reasoning
BigBenchHard
89.8
90.6
82.4
89.7
ZebraLogic
76.0
88.3
24.8
69.4
AGI Eval English
88.2
90.0
76.9
88.1
Coding
HumanEvalPlus
91.4
91.2
79.2
92.3
MBPP+
68.0
70.6
65.7
70.1
LiveCodeBench v3
83.5
90.2
39.0
79.5
Advanced Usage
Multi-turn Conversation
python
1messages =[2{"role":"user","content":"What is category theory?"},3{"role":"assistant","content":"Category theory is a mathematical framework..."},4{"role":"user","content":"How does it apply to computer science?"}5]67if tokenizer.chat_template isnotNone:8 formatted_prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)9 response = generate(model, tokenizer, prompt=formatted_prompt, max_tokens=2048)
Research-Style Analysis
python
1research_prompt ="""
2Analyze the relationship between quantum mechanics and information theory.
3Think step by step and provide a comprehensive analysis.
4"""56response = generate(model, tokenizer, prompt=research_prompt, max_tokens=4096, verbose=True)
Like any base language model or fine-tuned model without safety filtering, these models can easily be prompted by users to generate harmful and sensitive content. Such content may also be produced unintentionally, especially in cases involving bias, so we recommend that users consider the risks when applying this technology. Additionally, many statements from OLMo or any LLM are often inaccurate, so facts should be verified.
Citation
bibtex
1@article{olmo3,
2 title = {{OLMo 3: Open Language Models for Research and Education}},
3 author = {{Allen Institute for AI}},
4 year = {2024},
5}
