Views
No views yet
🇹🇭 Thai Legal AI Model — IQ4_NL Quantized GGUF for llama.cpp
| Property | Value |
|---|---|
| Base Model | iapp/openthai2.0-legal-thaillm-nemotron-3-nano-30b-a3b |
| Architecture | Nemotron H (Mamba + Attention + MoE Hybrid) |
| Parameters | ~30B total, ~3.5B active per token |
| Quantization | IQ4_NL (Iterative Optimized 4-bit Non-Linear) |
| Bits-per-Weight | 4.61 BPW |
| File Size | 17 GB |
| Fallback Tensors | 5 of 401 (specialized SSM/Mamba layers) |
llama-quantize1llama-cli -m "OpenThai 2.0 Legal ThaiLLM (Nemotron-3-Nano-30B-A3B)_IQ4_NL.gguf" \
2 -n 1024 \
3 -t 8 \
4 -p "คำถามทางกฎหมายของคุณ..."1llama-server -m "OpenThai 2.0 Legal ThaiLLM (Nemotron-3-Nano-30B-A3B)_IQ4_NL.gguf" \
2 --host 0.0.0.0 \
3 --port 8080 \
4 --ctx-size 40961from llama_cpp import Llama
2
3model = Llama.from_pretrained(
4 repo_id="Naypa/OpenThai-2.0-Legal-ThaiLLM-Nemotron-3-Nano-30B-A3B-IQ4_NL-GGUF",
5 filename="OpenThai 2.0 Legal ThaiLLM (Nemotron-3-Nano-30B-A3B)_IQ4_NL.gguf",
6 n_ctx=4096,
7 n_gpu_layers=-1, # Offload all layers to GPU
8)
9
10output = model("คำถาม: กฎหมายไทยเกี่ยวกับ...", max_tokens=512)
11print(output["choices"][0]["text"])| Component | Minimum | Recommended |
|---|---|---|
| VRAM | 16 GB | 24 GB (RTX 3090) |
| RAM | 32 GB | 64 GB |
| Storage | 20 GB free | 30 GB free |