You can deploy this model and start earning money today!

blackshell69/
llama-sahabat-AI-v2-70B-GGUF-Relist

Name: llama-sahabat-AI-v2-70B-GGUF-Relist
Availability: InStock

gguf

Sahabat-AI/Llama-Sahabat-AI-v2-70B-IT

Views

No views yet

ALL QUANTIZATIONS BY https://huggingface.co/techhermit

Llama-Sahabat-AI-v2-70B-IT — GGUF Quantizations

GGUF quantizations of Sahabat-AI/Llama-Sahabat-AI-v2-70B-IT, quantized locally using llama.cpp.

About the base model

Sahabat-AI adalah model bahasa besar (LLM) yang dikembangkan secara kolaboratif oleh BRIN, GoTo, dan Bukalapak untuk mendukung ekosistem AI berbahasa Indonesia. Model ini dilatih menggunakan data bahasa Indonesia yang kaya dan beragam, sehingga mampu memahami konteks budaya, bahasa, dan kebutuhan spesifik pengguna Indonesia dengan lebih baik.

Sahabat AI is a Large Language Model (LLM) collaboratively developed by BRIN (National Research and Innovation Agency), GoTo, and Bukalapak to support the Indonesian-language AI ecosystem. Trained on rich and diverse Indonesian language data, it better understands the cultural context, language nuances, and specific needs of Indonesian users.

Available quantizations

Repo	Quantization	Size	BPW	imatrix	Best for
blackshell69/Llama-Sahabat-AI-v2-70B-IT-Q8_0	Q8_0	70 GB	8.50	No	Near-lossless, 80+ GB VRAM
blackshell69/Llama-Sahabat-AI-v2-70B-IT-Q3_K_S	Q3_K_S	29 GB	3.50	No	Best quality that fits a 32 GB GPU
blackshell69/Llama-Sahabat-AI-v2-70B-IT-Q2_K_S	Q2_K_S	23 GB	2.77	Yes	Smallest footprint, quality trade-off

Quantization notes

Q8_0 — converted directly from bf16 safetensors; effectively lossless
Q3_K_S — good quality-size trade-off; fits comfortably on a single 32 GB V100
Q2_K_S — uses an importance matrix (imatrix) generated from Q3_K_S + groups_merged.txt calibration data for better weight selection at extreme compression

Usage

Load with any llama.cpp-compatible runner:

llama-cli -m llama-sahabat-70b-Q3_K_S.gguf -p "Halo, apa kabar?" -ngl 40

Hardware requirements

Quantization	Min VRAM (full offload)	CPU RAM (no GPU)
Q2_K_S	~25 GB	~28 GB
Q3_K_S	~32 GB	~34 GB
Q8_0	~75 GB	~78 GB

blackshell69/llama-sahabat-AI-v2-70B-GGUF-Relist