NVFP4 4-bit build. Runs on a single 24 GB GPU (RTX 4090, RTX 5090). This is the quantized edition of openthai2.0-legal-thaillm-nemotron-3-nano-30b-a3b; the full-precision model card applies unless noted below.
OpenThai 2.0 Legal: open-weight Thai legal LLM built on NVIDIA Nemotron 30B-A3B
An open-weight model exploring how far Thai legal knowledge and verifiable, section-level citation can be pushed on a self-hostable 30B base, with or without retrieval.
Highlights
Runs on one consumer GPU: NVFP4 4-bit, ~20 GB of weights, fits a single 24 GB card. 30B MoE, ~3B active per token.
Cites the law: exact law name + มาตรา in a JSON contract, for RAG and drafting tools.
Knows the statute book: ~4x Qwen3.6-35B on closed-book Civil and Commercial recall.
Writes like a lawyer: ahead of Qwen3.6-35B on all four legal-essay axes.
Grounded training data: Thai statutes and court rulings, each answer tied to its source section.
Benchmark results
Benchmark scoreboard: closed-book, open-book (RAG) and legal-essay axes vs Qwen3.6-35B and the Nemotron base
Evaluation
OpenThai2.0-Legal
Qwen3.6-35B
Nemotron-3-30B base
Closed-book: knows the law from memory
Civil & Commercial: recall the Civil and Commercial Code (n=3,729)
0.07
0.02
0.001
Tax: recall the Revenue Code (n=50)
0.40
0.36
0.31
Open-book: uses provided law (the RAG setting)
Civil & Commercial: cite the applicable sections from context (n=3,729)
0.99
0.99
0.98
Tax echo: cite the sections provided (n=50)
0.84
0.84
0.64
Tax selection: cite only the applicable sections among distractors (n=50)
0.69
0.64
0.45
Legal essays (Thai Supreme Court cases): legal substance (holding, coverage and fluency judged by Gemini 3.1 Flash Lite)
Citations: correct citations inside the essay (n=72)
0.25
0.09
0.02
Holding: reaches the correct legal conclusion (n=72)
0.57
0.50
0.31
Coverage: covers the key legal points (n=72)
0.60
0.55
0.27
Fluency: writing quality (n=72)
0.46
0.43
0.13
Scoring: 0 to 1, higher is better. Citation rows use NitiBench's citation-F1 scorer (single pass, temperature 0); essay holding, coverage and fluency are judged by Gemini 3.1 Flash Lite.
These are the full-precision parent's scores. This repository is the NVFP4 4-bit build of that model. It was calibrated on held-out Thai legal text and evaluated on the same benchmarks: the citation rows are preserved at 4-bit, but the essay rows are lower on this quantized build (long-form writing degrades at 4-bit; for essays use the full-precision model). See Quantization.
How it was trained
Three stages on the NVIDIA NeMo stack, each solving one problem:
CPT (continued pretraining) teaches the law itself: the model absorbs Thai statutes and real legal rulings so the knowledge lives in the weights.
SFT (supervised fine-tuning) teaches grounded answering: every answer follows a fixed JSON contract that cites only the sections it was given.
GRPO (reinforcement learning) teaches precision: the reward is the official benchmark's own citation F1, penalizing wrong or missing citations.
Training pipeline: Base 30B, CPT, SFT, GRPO, Released
Quantization
This build is the full-precision model compressed to NVFP4 (NVIDIA's 4-bit floating-point format) with NVIDIA's open quantization tooling.
What is quantized. The mixture-of-experts and attention weights, about 95% of the model, are stored in 4-bit. The Mamba state-space layers and the output head are kept in bfloat16, where low precision would hurt most. The result is roughly 20 GB of weights, down from about 63 GB, so the model fits a single 24 GB GPU.
Calibrated. Quantization was calibrated on a held-out sample of Thai legal text drawn from the same distribution the model serves, so the 4-bit weights are tuned to the real workload rather than generic data. The calibration sample is decontaminated against every evaluation set.
Evaluated. The quantized weights were run through the full citation benchmark suite (open-book Civil and Commercial, tax echo, tax selection, closed-book) and the citation results match the full-precision model within measurement noise: on structured citation and retrieval the two are equivalent. Long-form essay writing is a different story: it degrades noticeably at 4-bit. Errors that are invisible in a 120-token citation answer accumulate across a 1,000-token essay, so coverage and fluency drop. If your use is essay drafting or long free-form legal writing, use the full-precision model, not this one.
Verified. Every tensor was checked to load and serve correctly under vLLM before release.
Use this build for citation and RAG on one GPU. Use the full-precision model for essay drafting and the best all-round quality.
32,768 served window, prompt plus completion (architecture supports up to 256k)
Languages
Thai (primary), English (reasoning)
Precision
NVFP4 4-bit weights (experts + attention); Mamba layers and output head in bfloat16. ~20 GB, 5 shards
Training stack
NVIDIA NeMo (Megatron-Bridge for CPT/SFT, NeMo-RL for GRPO)
Serving
vLLM on a single GPU (verified); OpenAI-compatible
Status
Open-weight release
Evaluation
RAG mode means you put candidate law sections in the prompt and the model cites only the ones that apply. It works with any retriever you already have and needs no particular embedding or rerank model. Because it is also strong closed-book, many questions need no retrieval at all.
Four modes, covering how a legal model is actually used:
Mode
What it tests
Setup
Open-book "echo"
RAG best case: uses exactly the right documents
Only the relevant sections are provided; cite them all
Open-book "selection"
RAG realistic case: rejects near-miss documents
Relevant sections mixed with inapplicable ones; cite only the right subset
Closed-book
Pure memory, no retrieval
The question alone; recall law name and section from weights
Legal essays
Long-form legal writing
72 held-out Thai Supreme Court cases reformatted as essay questions, graded on citations, holding, coverage and fluency
What each mode looks like
All examples are verbatim model output on held-out benchmark questions.
Open-book selection, the hardest mode. Five sections supplied, three applicable.
EN (question): A Thai company posts executives to serve as directors of its overseas subsidiaries, where they receive salary and benefits. What are the withholding and personal income tax consequences?
EN (answer): The company must include income paid for duties performed for the Thai employer when computing withholding tax under Revenue Code section 50(1). An executive paid for the overseas directorship is not liable to Thai personal income tax if the money is not brought into Thailand in the same tax year.
Cited 41, 50 and 56; left out 40 and 48. Citation F1 1.00.
Closed-book, no statute text in the prompt at all.
EN (question): A company imports a product "C." used in place of Choline Chloride in animal feed, to support liver function. Is the import VAT-exempt?
EN (answer): The import qualifies as an import of animal feed exempt from VAT under Revenue Code section 81(1)(d) and section 81(2)(a), the company holding the controlled animal-feed import licence.
Section and both sub-paragraphs recalled from weights, with no statute provided. Citation F1 1.00.
EN (question): The defendant owed the co-plaintiff 95,600 baht in loan and chit-fund debt, recorded in the co-plaintiff's ledger with text and figures evidencing the debt, which the defendant had signed to acknowledge. The defendant later struck through the text and figures with pen marks and tore the document apart.
EN (answer, excerpt): The Supreme Court affirmed the Court of Appeal, holding the defendant guilty of damaging or destroying a document in a manner likely to cause injury to another, under Criminal Code section 188 ... Striking out the text and figures and then tearing the document apart constitutes damage or destruction likely to cause injury, even though the remaining fragments can still be read.
Deployment
Serve on a single GPU. The model exposes an OpenAI-compatible API, so the Python examples that follow work unchanged.
Tensor parallelism must be 1. The MoE layer width does not divide evenly for TP=2; to split across two cards use pipeline parallelism, not tensor parallelism.
On Blackwell (RTX 5090) the 4-bit weights run on native FP4 hardware. On Ada (RTX 4090) they run through a 4-bit weight path with bf16 compute: the memory saving applies on both, the extra math speedup only on Blackwell.
Do not enable prefix caching with this hybrid Mamba model; it sharply reduces how many requests fit.
Tested configuration: vLLM 0.19.1, tensor-parallel 1 on one H100 (80 GB), --max-model-len 32768, --gpu-memory-utilization 0.90, --trust-remote-code, MoE backend Marlin. Loads in ~18.7 GiB. Thinking on and off, and temperatures 0.0 and 0.7, all verified.
Quickstart
Grounded citation answering (RAG mode). This exact example was run against the released weights; the output shown is the model's real response:
python
1from openai import OpenAI
2client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")34SYSTEM =("You are OpenThaiGPT-Legal, an expert assistant on Thai law. You are given a legal "5"question and the exact statutory sections needed to answer it. Reason step by step in "6"English, then give the final answer in Thai. Cite ONLY sections present in the provided "7"context, using each section's exact law_name and bare section number (e.g. 132, 77/1). "8'Output the final answer as JSON: {"answer": "<Thai answer>", '9'"citations": [{"law": "<law_name>", "section": "<bare id>"}]}.')1011USER ="""Provided context:
12<law law_name="ประมวลกฎหมายแพ่งและพาณิชย์" section="420">
13ผู้ใดจงใจหรือประมาทเลินเล่อ ทำต่อบุคคลอื่นโดยผิดกฎหมายให้เขาเสียหาย ... จำต้องใช้ค่าสินไหมทดแทนเพื่อการนั้น
14</law>
1516Question (ตอบเป็นภาษาไทย):
17นาย ก. ขับรถโดยประมาทชนรถของนาย ข. เสียหาย นาย ก. ต้องรับผิดตามกฎหมายใด"""1819r = client.chat.completions.create(20 model="openthai2.0-legal-thaillm-nemotron-3-nano-30b-a3b-NVFP4",21 messages=[{"role":"system","content": SYSTEM},{"role":"user","content": USER}],22 temperature=0.0, max_tokens=2048,23 extra_body={"chat_template_kwargs":{"enable_thinking":False}})24print(r.choices[0].message.content)
Closed-book mode. No context at all. The model recalls the section from its weights.
python
1SYSTEM =("You are an expert on Thai law. You are given ONLY a legal question, with NO reference "2"material provided. Using your OWN knowledge of Thai statutes, answer in Thai and cite the "3"specific sections that apply (law name + bare section number, มาตรา). "4'Output ONLY a JSON object: {"answer":"<Thai answer>","citations":[{"law":"<law name>",'5'"section":"<bare section number e.g. 40 or 77/1>"}]}.')67USER ="""บริษัท ก. (บริษัทฯ) ได้หารือเกี่ยวกับภาษีมูลค่าเพิ่ม กรณีการนำเข้าสินค้าที่ใช้ชื่อ C. มีข้อเท็จจริงสรุปได้ว่า บริษัทฯ ได้นำเข้าสินค้าชื่อว่า “C.” เพื่อใช้ทดแทน Choline Chloride ในอาหารสัตว์ โดยมีสรรพคุณใช้เพื่อช่วยในการทำงานของตับ และป้องกันการรั่วไหลของเอนไซม์ transaminases ในอาหารสัตว์ที่ขาด Choline แต่เนื่องจาก ข. ไม่เป็นอาหารสัตว์ ที่จะต้องขึ้นทะเบียนอาหารสัตว์ตามพระราชบัญญัติควบคุมคุณภาพอาหารสัตว์ แต่ต้องขออนุญาตนำเข้าจากกรมปศุสัตว์ ซึ่งบริษัทฯ เป็นผู้รับใบอนุญาตนำเข้าอาหารสัตว์ ตามใบอนุญาตนำเข้าอาหารสัตว์ควบคุมเฉพาะ จึงขอหารือว่า การนำเข้าสินค้าดังกล่าว เป็นการนำเข้าอาหารสัตว์ซึ่งได้รับยกเว้นภาษีมูลค่าเพิ่ม ตามมาตรา 81 (1) (ง) และมาตรา 81 (2) (ก) แห่งประมวลรัษฎากร หรือไม่"""89r = client.chat.completions.create(10 model="openthai2.0-legal-thaillm-nemotron-3-nano-30b-a3b-NVFP4",11 messages=[{"role":"system","content": SYSTEM},{"role":"user","content": USER}],12 temperature=0.0, max_tokens=2048,13 extra_body={"chat_template_kwargs":{"enable_thinking":False}})14print(r.choices[0].message.content)
Essay mode. A short system prompt and the exam question. The model writes a full legal essay, citing มาตรา in prose rather than JSON, so raise max_tokens.
python
1SYSTEM ="You are a Thai legal expert. Answer the question with legal analysis and cite the relevant มาตรา."23USER ="""จำเลยเป็นหนี้เงินกู้ยืมและเงินเล่นแชร์จำนวน 95,600 บาทแก่โจทก์ร่วม ซึ่งได้จดทะเบียนไว้ในสมุดบันทึกของโจทก์ร่วม โดยมีข้อความและตัวเลขเป็นหลักฐานแห่งหนี้ และจำเลยได้ลงลายมือชื่อรับรองข้อความและตัวเลขในเอกสารนั้นไว้ ต่อมาจำเลยได้ใช้ปากกาขีดฆ่าเป็นเครื่องหมายกากบาทและเครื่องหมายวงกลมลงบนข้อความและตัวเลขในเอกสารดังกล่าว จากนั้นได้ฉีกเอกสารจนขาดออกจากกัน แม้เอกสารที่ยังเหลืออยู่จะสามารถอ่านเข้าใจข้อความและตัวเลขเดิมได้ โจทก์ร่วมจึงฟ้องว่าจำเลยมีความผิดฐานทำให้เสียหายหรือทำลายเอกสารในประการที่น่าจะเกิดความเสียหายแก่ผู้อื่น ขอให้ลงโทษ และคืนของกลางแก่เจ้าของ จำเลยให้การปฏิเสธว่าไม่ได้กระทำหรือหากกระทำก็ไม่มีความผิดสมบูรณ์เพราะเอกสารยังอ่านได้ โจทก์ร่วมจึงยื่นคำร้องขอเข้าร่วมเป็นโจทก์ ศาลชั้นต้นและศาลอุทธรณ์พิพากษาว่าจำเลยมีความผิด ให้จำคุกจำเลย 6 เดือน จำเลยอุทธรณ์และฎีกา ขอให้ยกฟ้องหรือพิจารณาใหม่โดยอ้างว่าการกระทำของตนยังไม่สมบูรณ์เพราะเอกสารยังอ่านได้ จึงเป็นเพียงการพยายามกระทำความผิดหรือไม่"""45r = client.chat.completions.create(6 model="openthai2.0-legal-thaillm-nemotron-3-nano-30b-a3b-NVFP4",7 messages=[{"role":"system","content": SYSTEM},{"role":"user","content": USER}],8 temperature=0.0, max_tokens=4096,9 extra_body={"chat_template_kwargs":{"enable_thinking":False}})10print(r.choices[0].message.content)
These are the exact prompts used to evaluate the model.
Recommended generation settings
Use case
temperature
top_p
max_tokens (suggested)
thinking
Citation answering (RAG or closed-book)
0.0
1.0
2048
off
Legal essay drafting
0.7
0.9
4096 (6144 with thinking)
on or off
General chat
0.7
0.9
2048
off
Notes:
Thai costs about twice the token budget of English. Thai legal prose tokenizes at roughly 1.85 characters per token with this tokenizer, against 2.6 to 3.9 for English, and Thai has no inter-word spaces. A 1,024-token budget is only about 1,900 Thai characters, roughly one page. Set max_tokens to about twice what the same task would need in English.
max_tokens is a per-request setting, not a model limit. The limit is the served context window, 32,768 tokens for prompt plus completion. At temperature 0 answers terminate naturally (citation answers have a median of 121 tokens), so a larger budget costs nothing.
Do not set it below 1,536 for citation answering. A truncated JSON answer loses its whole citations array, so the item scores zero rather than partially: measured on our own runs, a 1,024 cap truncated 5.3% of held-out tax-selection answers and cost about 0.03 macro-F1 on that split.
Thinking mode (chat_template_kwargs: {"enable_thinking": true}) is fully supported: the model reasons in English inside a think block, then answers in Thai. It helps on essays and complex analysis. The benchmark numbers above use thinking off, which is the tuned path for citation tasks; allow roughly 2x the token budget when thinking is on.
Always serve with --max-model-len 32768: real Revenue Code contexts overflow smaller windows.
Limitations and responsible use
Retrieval quality drives results. Given the right sections, citations are near-perfect; if the retriever supplies the wrong sections, the answer follows them. For production RAG, pair the model with a sound retriever and monitor retrieval quality.
Hardest task, near-miss rejection. Tax selection (picking the applicable section among close alternatives) is the lowest-scoring axis for every model, ours included; telling a governing section from a closely related one is where errors concentrate, so retrieval tuning and human review are advised for high-stakes use.
Feed a focused context. Citation accuracy is highest when the prompt holds a handful of candidate sections. The more unrelated sections you pass, the harder near-miss rejection becomes: pass a tightly retrieved set (a page or few, not the whole code) rather than relying on the full 32k window.
Structured output is the strong path. The JSON citation contract is ideal for automation and legal-tech pipelines; free-form chat is supported but less structured.
Scope. Trained on Thai statutory law (Revenue Code, Civil and Commercial Code, related acts) and Revenue Department rulings; coverage of niche areas or very recent amendments may be thinner, and statutes change over time.
Human oversight required. Outputs are decision support, not legal advice. A qualified professional should verify every citation against the current law before relying on it.
Data provenance. Some essay training data derives from Thai bar examination materials; confirm redistribution terms for your jurisdiction and use case.
