⚠️ This is NOT an official release from HiTZ Zentroa. These GGUF files were converted from the official HiTZ/Latxa-Llama-3.1-70B-Instruct-v2 model by a third party. For the original model, training details, and official support, please refer to the official HiTZ model page.

About Latxa

Latxa is a family of large language models (LLMs) developed by HiTZ Zentroa at the University of the Basque Country, specifically for the Basque language (Euskara), while maintaining strong multilingual support in English, Spanish, French, Catalan, and Galician.

Latxa-Llama-3.1-70B-Instruct-v2 is a 70 billion parameter instruction-tuned model based on Meta's Llama 3.1 architecture, trained on the Latxa v2 datasets.

GGUF Conversions

The following GGUF files are provided, converted using llama.cpp:

File	Quantization	Size	Description
`Latxa-Llama-3.1-70B-Instruct-v2-FP16.gguf`	FP16	131 GB	Full precision (reference)
`Latxa-Llama-3.1-70B-Instruct-v2-Q8_0.gguf`	Q8_0	70 GB	Near-lossless, recommended for high-quality inference
`Latxa-Llama-3.1-70B-Instruct-v2-Q4_K_M.gguf`	Q4_K_M	40 GB	Very good quality/size trade-off, most practical

Usage

With llama.cpp

bash

# Build and run
./build/bin/llama-cli \
  -m Latxa-Llama-3.1-70B-Instruct-v2-Q4_K_M.gguf \
  -p "Azaldu euskararen historia labur-labur." \
  -n 512

With Ollama

Create a Modelfile:

FROM ./Latxa-Llama-3.1-70B-Instruct-v2-Q4_K_M.gguf

Then:

bash

ollama create latxa-70b-v2 -f Modelfile
ollama run latxa-70b-v2

Model Details

Architecture: Llama 3.1 (decoder-only transformer)
Parameters: 70B
Layers: 80
Context length: 131,072 tokens
Vocabulary size: 128,256
Attention: Grouped Query Attention (64 heads, 8 KV heads)

Original Model

For the official model, license, training data, evaluation results, and proper attribution, visit: https://huggingface.co/HiTZ/Latxa-Llama-3.1-70B-Instruct-v2

License

This model inherits the license from the original Llama 3.1 Community License. Please refer to the original model page for full license terms.

Citation

Cite the original Latxa models:

bibtex

@misc{latxa2024,
  author = {HiTZ Zentroa},
  title = {Latxa: Basque Language Models},
  year = {2024},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/HiTZ}},
}

These GGUF files were converted from HiTZ/Latxa-Llama-3.1-70B-Instruct-v2-FP8 (decompressed to FP16 during conversion) using llama.cpp.

itzune/Latxa-Llama-3.1-70B-Instruct-v2-GGUF

Latxa-Llama-3.1-70B-Instruct-v2 - GGUF