The interesting thing about the original model is its training data. It was trained entirely on the Common Pile v0.1, an 8 TB corpus of public domain and openly licensed text. All credit belongs to EleutherAI and their collaborators.

This repository changes nothing but the numeric precision.

Why this exists

The existing MLX conversion, simonw/comma-v0.1-2t-mlx, is unquantized at roughly 14 GB. That is uncomfortable on a 16 GB Mac, where it competes with the operating system and everything else for memory.

At 4 bits the model is 3.7 GB on disk and peaks around 4.1 GB resident, which leaves room to use the machine while it runs.

Property	Original	This repo
Precision	bfloat16	4-bit affine
Size on disk	~14 GB	3.7 GB
Peak memory	~14 GB	~4.1 GB

Measured throughput on an M4 MacBook: roughly 26 tokens/sec.

This is a base model

Comma has no instruction tuning and no chat template. It continues text; it does not answer questions. Prompt it with the opening of the passage you want written, the way models were prompted before chat tuning became standard.

This works:

Facts about pelicans:

Asking "What are some facts about pelicans?" will likely produce more questions rather than an answer. That is the model behaving as designed, not an artefact of quantization.

Usage

bash

pip install mlx-lm

mlx_lm.generate \
  --model dokoissho/comma-v0.1-2t-mlx-4bit \
  --prompt 'Facts about pelicans:' \
  --max-tokens 200

python

from mlx_lm import load, generate

model, tokenizer = load("dokoissho/comma-v0.1-2t-mlx-4bit")
print(generate(model, tokenizer, prompt="Facts about pelicans:", max_tokens=200))

Conversion

bash

python -m mlx_lm convert \
  --hf-path common-pile/comma-v0.1-2t \
  --mlx-path ./comma-v0.1-2t-mlx-4bit \
  -q

Quantization settings, as recorded in config.json:

Setting	Value
Bits	4
Group size	64
Mode	affine
Effective	4.5 bits per weight

Caveats

Quantization costs accuracy. This build has not been benchmarked against the original, so treat any quality difference as unmeasured rather than absent. If you are evaluating what Comma can do, use the full-precision model.

The base model's own limits apply unchanged. It is a 7B model and it will state wrong things fluently.

Licence

Apache 2.0, inherited from the original. Please cite EleutherAI and the Common Pile rather than this repository, which contributes nothing but a smaller file.