Property	Value
Base model	meta-llama/Llama-3.1-8B-Instruct
Architecture	LlamaForCausalLM
Parameters	8.03B
Quantization	8-bit affine (group size 64) via MLX
Format	MLX safetensors
Context length	131,072 tokens
Model size on disk	~8.5 GB
License	Llama 3.1 Community License

Training

This model was fine-tuned from meta-llama/Llama-3.1-8B-Instruct using LoRA on a 2× NVIDIA RTX 3090 cluster. The fine-tuning focused on improving creative and narrative writing quality. After training, the LoRA adapter was merged back into the base model and the result was converted to MLX 8-bit quantized format.

Training Details

Method: LoRA (Low-Rank Adaptation)
Hardware: 2× NVIDIA RTX 3090 (48 GB VRAM total)
Framework: Unsloth
Dataset: Curated creative writing examples
Quantization tool: mlx_lm.convert

Intended Use

This model is designed for creative and narrative writing tasks — storytelling, prose generation, dialogue, and similar text generation. It follows the Llama 3.1 Instruct chat template.

How to Use

With MLX (Apple Silicon)

python

from mlx_lm import load, generate

model, tokenizer = load("warrenm/llama3.1-8B-writer-Q8")
response = generate(model, tokenizer, prompt="Write the opening paragraph of a noir detective story.", max_tokens=512)
print(response)

With Cogitae (macOS)

This model can be loaded directly in Cogitae via its built-in MLX engine. Place the model files in ~/.cogitae/models/warrenm/llama3.1-8B-writer-Q8/ and load from the MLX provider.

Files

File	Size	Description
`config.json`	1 KB	Model configuration
`tokenizer.json`	16.4 MB	Tokenizer vocabulary
`tokenizer_config.json`	345 B	Tokenizer settings
`chat_template.jinja`	4.5 KB	Llama 3.1 Instruct chat template
`model.safetensors.index.json`	62 KB	Weight shard index
`model-00001-of-00002.safetensors`	5.0 GB	Weight shard 1/2
`model-00002-of-00002.safetensors`	3.0 GB	Weight shard 2/2

Limitations

Fine-tuned primarily for creative writing; may underperform on reasoning, coding, or factual Q&A compared to the base instruct model.
8-bit quantization introduces minor quality loss compared to the full-precision model.
Inherits all limitations and biases of the base Llama 3.1 8B Instruct model.