Built with Llama. This repository contains only a GGUF quantization of the model — I did not train, fine-tune, or otherwise create the underlying model. All credit for the model belongs to the PLLuM consortium / HIVE AI (CYFRAGOVPL), who trained it on ~150B tokens of Polish text using the Bem2 HPC cluster. This upload exists solely to provide a llama.cpp-compatible GGUF, which was not otherwise available for the instruct-2512 variant at the time of upload.

Model details

Base model: Llama-3.1-70B (continued-pretrained on Polish + English, instruction-tuned by PLLuM)
Variant: instruct (instruction-following; not the preference-aligned chat variant)
Architecture: LlamaForCausalLM — 80 layers, hidden 8192, 64 attn heads / 8 KV heads (GQA), 131072 context, rope theta 500000
Language: Polish (primary), English (secondary)

Available quantizations

Quant	Size	Notes
Q8_0	~75 GB	Highest practical fidelity; round-to-nearest, no imatrix

Converted with llama.cpp:

bash

python convert_hf_to_gguf.py ./Llama-PLLuM-70B-instruct-2512 \
  --outfile Llama-PLLuM-70B-instruct-2512-Q8_0.gguf \
  --outtype q8_0

⚠️ Chat template — important

Despite the Llama base, this model does NOT use the standard Llama-3.1 header format (<|start_header_id|>...<|eot_id|>). It uses a Mistral-style [INST] ... [/INST] template, embedded in the GGUF metadata. A system message is folded into the first [INST] block, separated from the user content by \n\n:

<|begin_of_text|>[INST]{system}\n\n{user}[/INST]{assistant}<|end_of_text|>

BOS: <|begin_of_text|> (128000) · EOS: <|end_of_text|> (128001)
Roles must alternate user/assistant; an optional initial system message is allowed.
llama-server detects and applies this template automatically from GGUF metadata. If you bypass it, replicate the [INST] format above (do not use llama3 template).

Usage (llama.cpp)

bash

# server
llama-server -m Llama-PLLuM-70B-instruct-2512-Q8_0.gguf -c 16384 -ngl 99 --port 8080

# OpenAI-compatible request (system folds into first [INST] automatically)
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "messages": [
    {"role": "system", "content": "Jesteś pomocnym asystentem."},
    {"role": "user", "content": "Napisz krótkie podsumowanie."}
  ],
  "temperature": 0.1
}'

This is not a reasoning/thinking model (thinking = 0); for deterministic tasks (extraction, correction) low temperature (0.0–0.2) is recommended.

License

This GGUF is a derivative of a Llama-3.1-based model and is distributed under the Llama 3.1 Community License, in addition to any license terms set by the original authors (PLLuM / CYFRAGOVPL) on the base model card. Please review and comply with both before use. "Llama" is included in the model name in accordance with the Llama 3.1 license requirements for derivative models.

Acknowledgements

PLLuM consortium / HIVE AI (CYFRAGOVPL) — model training and release
Meta — Llama 3.1 base architecture
llama.cpp (ggml-org) — GGUF format and conversion tooling

marcinm1234/Llama-PLLuM-70B-instruct-2512-Q8_0-GGUF

Llama-PLLuM-70B-instruct-2512 — GGUF