An open-weight model exploring how far Thai legal knowledge and verifiable, section-level citation can be pushed on a self-hostable 30B base, with or without retrieval.
Open weights: 30B MoE, ~3B active per token, runs on a single 80 GB GPU (or one 24 GB GPU with the NVFP4 build).
256k context: a 262,144-token window, so long statute sets and multi-document ruling bundles fit in a single prompt. The Mamba2-Transformer hybrid keeps this cheap — only 6 of 52 layers are attention, so a full 256k sequence costs ~1.5 GiB of KV cache instead of the ~24 GiB a dense 30B would need.
Cites the law: exact law name + มาตรา in a JSON contract, for RAG and drafting tools.
Knows the statute book: ~4x Qwen3.6-35B on closed-book Civil and Commercial recall.
Writes like a lawyer: ahead of Qwen3.6-35B on all four legal-essay axes.
Grounded training data: Thai statutes and court rulings, each answer tied to its source section.
Benchmark results
Benchmark scoreboard: closed-book, open-book (RAG) and legal-essay axes vs Qwen3.6-35B and the Nemotron base
Evaluation
OpenThai2.0-Legal
Qwen3.6-35B
Nemotron-3-30B base
Closed-book: knows the law from memory
Civil & Commercial: recall the Civil and Commercial Code (n=3,729)
0.07
0.02
0.001
Tax: recall the Revenue Code (n=50)
0.40
0.36
0.31
Open-book: uses provided law (the RAG setting)
Civil & Commercial: cite the applicable sections from context (n=3,729)
0.99
0.99
0.98
Tax echo: cite the sections provided (n=50)
0.84
0.84
0.64
Tax selection: cite only the applicable sections among distractors (n=50)
0.69
0.64
0.45
Legal essays (Thai Supreme Court cases): legal substance (holding, coverage and fluency judged by Gemini 3.1 Flash Lite)
Citations: correct citations inside the essay (n=72)
0.25
0.09
0.02
Holding: reaches the correct legal conclusion (n=72)
0.57
0.50
0.31
Coverage: covers the key legal points (n=72)
0.60
0.55
0.27
Fluency: writing quality (n=72)
0.46
0.43
0.13
Scoring: 0 to 1, higher is better. Citation rows use NitiBench's citation-F1 scorer (single pass, temperature 0); essay holding, coverage and fluency are judged by Gemini 3.1 Flash Lite.
How it was trained
Three stages on the NVIDIA NeMo stack, each solving one problem:
CPT (continued pretraining) teaches the law itself: the model absorbs Thai statutes and real legal rulings so the knowledge lives in the weights.
SFT (supervised fine-tuning) teaches grounded answering: every answer follows a fixed JSON contract that cites only the sections it was given.
GRPO (reinforcement learning) teaches precision: the reward is the official benchmark's own citation F1, penalizing wrong or missing citations.
Training pipeline: Base 30B, CPT, SFT, GRPO, Released
NVIDIA NeMo (Megatron-Bridge for CPT/SFT, NeMo-RL for GRPO)
Serving
vLLM (verified) and NVIDIA NIM (OpenAI-compatible)
Status
Open-weight release
Evaluation
RAG mode means you put candidate law sections in the prompt and the model cites only the ones that apply. It works with any retriever you already have and needs no particular embedding or rerank model. Because it is also strong closed-book, many questions need no retrieval at all.
Four modes, covering how a legal model is actually used:
Mode
What it tests
Setup
Open-book "echo"
RAG best case: uses exactly the right documents
Only the relevant sections are provided; cite them all
Open-book "selection"
RAG realistic case: rejects near-miss documents
Relevant sections mixed with inapplicable ones; cite only the right subset
Closed-book
Pure memory, no retrieval
The question alone; recall law name and section from weights
Legal essays
Long-form legal writing
72 held-out Thai Supreme Court cases reformatted as essay questions, graded on citations, holding, coverage and fluency
What each mode looks like
All examples are verbatim model output on held-out benchmark questions.
Open-book selection, the hardest mode. Five sections supplied, three applicable.
EN (question): A Thai company posts executives to serve as directors of its overseas subsidiaries, where they receive salary and benefits. What are the withholding and personal income tax consequences?
EN (answer): The company must include income paid for duties performed for the Thai employer when computing withholding tax under Revenue Code section 50(1). An executive paid for the overseas directorship is not liable to Thai personal income tax if the money is not brought into Thailand in the same tax year.
Cited 41, 50 and 56; left out 40 and 48. Citation F1 1.00.
Closed-book, no statute text in the prompt at all.
EN (question): A company imports a product "C." used in place of Choline Chloride in animal feed, to support liver function. Is the import VAT-exempt?
EN (answer): The import qualifies as an import of animal feed exempt from VAT under Revenue Code section 81(1)(d) and section 81(2)(a), the company holding the controlled animal-feed import licence.
Section and both sub-paragraphs recalled from weights, with no statute provided. Citation F1 1.00.
EN (question): The defendant owed the co-plaintiff 95,600 baht in loan and chit-fund debt, recorded in the co-plaintiff's ledger with text and figures evidencing the debt, which the defendant had signed to acknowledge. The defendant later struck through the text and figures with pen marks and tore the document apart.
EN (answer, excerpt): The Supreme Court affirmed the Court of Appeal, holding the defendant guilty of damaging or destroying a document in a manner likely to cause injury to another, under Criminal Code section 188 ... Striking out the text and figures and then tearing the document apart constitutes damage or destruction likely to cause injury, even though the remaining fragments can still be read.
Deployment
The model exposes an OpenAI-compatible API under both serving paths below, so the Python example that follows works unchanged either way.
vLLM (one 80 GB GPU, or two 40 to 48 GB GPUs):
bash
1# single 80 GB GPU (H100, A100-80GB):2vllm serve iapp/openthai2.0-legal-thaillm-nemotron-3-nano-30b-a3b \3 --served-model-name openthai2.0-legal-thaillm-nemotron-3-nano-30b-a3b \4 --tensor-parallel-size 1 --trust-remote-code \5 --max-model-len 32768 --gpu-memory-utilization 0.9067# two 40 to 48 GB GPUs: use --tensor-parallel-size 2
Serving the full 256k window. Set --max-model-len 262144. No extra environment flag is needed: 262,144 is the model's max_position_embeddings, so vLLM accepts it directly.
--max-num-batched-tokens 8192 keeps chunked prefill active, which is what bounds activation memory on a long prefill. It is the flag that matters most for stability at this length.
On one 80 GB card, weights take ~59 GiB and each full 256k sequence costs ~1.5 GiB of KV cache plus ~48 MiB of Mamba state, leaving room for roughly 8 concurrent maximum-length requests.
Prefill compute, not memory, is the practical limit: a genuinely full 256k prompt takes appreciable time to first token on a single GPU. Use --tensor-parallel-size 2 to roughly halve it and to raise concurrency (KV cache shards to ~0.75 GiB per sequence per GPU).
Do not raise tensor parallelism past 2 expecting further KV savings. The model has num_key_value_heads: 2, so at TP=4 or TP=8 the KV heads are replicated rather than split. Scale with additional TP=2 replicas instead.
NVIDIA NIM (the NVIDIA-native inference microservice; the base is an NVIDIA Nemotron, so it fits the NIM for LLMs path):
bash
1docker run --rm --gpus all --shm-size=16g \2 -v /path/to/downloaded/model:/model \3 -e NIM_MODEL_NAME=/model \4 -e NIM_SERVED_MODEL_NAME=openthai2.0-legal \5 -p 8000:8000 \6 nvcr.io/nim/nvidia/llm-nim:latest
Use the current container tag and configuration from the NVIDIA NIM for LLMs catalog; NIM exposes the same OpenAI-compatible endpoint.
Tested configuration: vLLM 0.19.1, verified both on a single H100 80GB (--tensor-parallel-size 1) and on 2x H100 (--tensor-parallel-size 2), --max-model-len 32768, --gpu-memory-utilization 0.90, --trust-remote-code. On one 80GB card the weights load in ~59 GiB, leaving room for roughly 40 concurrent 32k-token requests. Thinking on and off, and temperatures 0.0 and 0.7, all verified. The benchmark numbers in this card were produced at a 32,768 window; the 256k figures above follow from the model configuration and the hybrid's KV-cache footprint.
Quickstart
Grounded citation answering (RAG mode). This exact example was run against the released weights; the output shown is the model's real response:
python
1from openai import OpenAI
2client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")34SYSTEM =("You are OpenThaiGPT-Legal, an expert assistant on Thai law. You are given a legal "5"question and the exact statutory sections needed to answer it. Reason step by step in "6"English, then give the final answer in Thai. Cite ONLY sections present in the provided "7"context, using each section's exact law_name and bare section number (e.g. 132, 77/1). "8'Output the final answer as JSON: {"answer": "<Thai answer>", '9'"citations": [{"law": "<law_name>", "section": "<bare id>"}]}.')1011USER ="""Provided context:
12<law law_name="ประมวลกฎหมายแพ่งและพาณิชย์" section="420">
13ผู้ใดจงใจหรือประมาทเลินเล่อ ทำต่อบุคคลอื่นโดยผิดกฎหมายให้เขาเสียหาย ... จำต้องใช้ค่าสินไหมทดแทนเพื่อการนั้น
14</law>
1516Question (ตอบเป็นภาษาไทย):
17นาย ก. ขับรถโดยประมาทชนรถของนาย ข. เสียหาย นาย ก. ต้องรับผิดตามกฎหมายใด"""1819r = client.chat.completions.create(20 model="openthai2.0-legal-thaillm-nemotron-3-nano-30b-a3b",21 messages=[{"role":"system","content": SYSTEM},{"role":"user","content": USER}],22 temperature=0.0, max_tokens=2048,23 extra_body={"chat_template_kwargs":{"enable_thinking":False}})24print(r.choices[0].message.content)
Closed-book mode. No context at all. The model recalls the section from its weights.
python
1SYSTEM =("You are an expert on Thai law. You are given ONLY a legal question, with NO reference "2"material provided. Using your OWN knowledge of Thai statutes, answer in Thai and cite the "3"specific sections that apply (law name + bare section number, มาตรา). "4'Output ONLY a JSON object: {"answer":"<Thai answer>","citations":[{"law":"<law name>",'5'"section":"<bare section number e.g. 40 or 77/1>"}]}.')67USER ="""บริษัท ก. (บริษัทฯ) ได้หารือเกี่ยวกับภาษีมูลค่าเพิ่ม กรณีการนำเข้าสินค้าที่ใช้ชื่อ C. มีข้อเท็จจริงสรุปได้ว่า บริษัทฯ ได้นำเข้าสินค้าชื่อว่า “C.” เพื่อใช้ทดแทน Choline Chloride ในอาหารสัตว์ โดยมีสรรพคุณใช้เพื่อช่วยในการทำงานของตับ และป้องกันการรั่วไหลของเอนไซม์ transaminases ในอาหารสัตว์ที่ขาด Choline แต่เนื่องจาก ข. ไม่เป็นอาหารสัตว์ ที่จะต้องขึ้นทะเบียนอาหารสัตว์ตามพระราชบัญญัติควบคุมคุณภาพอาหารสัตว์ แต่ต้องขออนุญาตนำเข้าจากกรมปศุสัตว์ ซึ่งบริษัทฯ เป็นผู้รับใบอนุญาตนำเข้าอาหารสัตว์ ตามใบอนุญาตนำเข้าอาหารสัตว์ควบคุมเฉพาะ จึงขอหารือว่า การนำเข้าสินค้าดังกล่าว เป็นการนำเข้าอาหารสัตว์ซึ่งได้รับยกเว้นภาษีมูลค่าเพิ่ม ตามมาตรา 81 (1) (ง) และมาตรา 81 (2) (ก) แห่งประมวลรัษฎากร หรือไม่"""89r = client.chat.completions.create(10 model="openthai2.0-legal-thaillm-nemotron-3-nano-30b-a3b",11 messages=[{"role":"system","content": SYSTEM},{"role":"user","content": USER}],12 temperature=0.0, max_tokens=2048,13 extra_body={"chat_template_kwargs":{"enable_thinking":False}})14print(r.choices[0].message.content)
Essay mode. A short system prompt and the exam question. The model writes a full legal essay, citing มาตรา in prose rather than JSON, so raise max_tokens.
python
1SYSTEM ="You are a Thai legal expert. Answer the question with legal analysis and cite the relevant มาตรา."23USER ="""จำเลยเป็นหนี้เงินกู้ยืมและเงินเล่นแชร์จำนวน 95,600 บาทแก่โจทก์ร่วม ซึ่งได้จดทะเบียนไว้ในสมุดบันทึกของโจทก์ร่วม โดยมีข้อความและตัวเลขเป็นหลักฐานแห่งหนี้ และจำเลยได้ลงลายมือชื่อรับรองข้อความและตัวเลขในเอกสารนั้นไว้ ต่อมาจำเลยได้ใช้ปากกาขีดฆ่าเป็นเครื่องหมายกากบาทและเครื่องหมายวงกลมลงบนข้อความและตัวเลขในเอกสารดังกล่าว จากนั้นได้ฉีกเอกสารจนขาดออกจากกัน แม้เอกสารที่ยังเหลืออยู่จะสามารถอ่านเข้าใจข้อความและตัวเลขเดิมได้ โจทก์ร่วมจึงฟ้องว่าจำเลยมีความผิดฐานทำให้เสียหายหรือทำลายเอกสารในประการที่น่าจะเกิดความเสียหายแก่ผู้อื่น ขอให้ลงโทษ และคืนของกลางแก่เจ้าของ จำเลยให้การปฏิเสธว่าไม่ได้กระทำหรือหากกระทำก็ไม่มีความผิดสมบูรณ์เพราะเอกสารยังอ่านได้ โจทก์ร่วมจึงยื่นคำร้องขอเข้าร่วมเป็นโจทก์ ศาลชั้นต้นและศาลอุทธรณ์พิพากษาว่าจำเลยมีความผิด ให้จำคุกจำเลย 6 เดือน จำเลยอุทธรณ์และฎีกา ขอให้ยกฟ้องหรือพิจารณาใหม่โดยอ้างว่าการกระทำของตนยังไม่สมบูรณ์เพราะเอกสารยังอ่านได้ จึงเป็นเพียงการพยายามกระทำความผิดหรือไม่"""45r = client.chat.completions.create(6 model="openthai2.0-legal-thaillm-nemotron-3-nano-30b-a3b",7 messages=[{"role":"system","content": SYSTEM},{"role":"user","content": USER}],8 temperature=0.0, max_tokens=4096,9 extra_body={"chat_template_kwargs":{"enable_thinking":False}})10print(r.choices[0].message.content)
These are the exact prompts used to produce every number above.
Recommended generation settings
Use case
temperature
top_p
max_tokens (suggested)
thinking
Citation answering (RAG or closed-book)
0.0
1.0
2048
off
Legal essay drafting
0.7
0.9
4096 (6144 with thinking)
on or off
General chat
0.7
0.9
2048
off
Notes:
Thai costs about twice the token budget of English. Thai legal prose tokenizes at roughly 1.85 characters per token with this tokenizer, against 2.6 to 3.9 for English, and Thai has no inter-word spaces. A 1,024-token budget is only about 1,900 Thai characters, roughly one page. Set max_tokens to about twice what the same task would need in English.
max_tokens is a per-request setting, not a model limit. The limit is whatever window you serve, up to the model's 262,144 tokens for prompt plus completion. At temperature 0 answers terminate naturally (citation answers have a median of 121 tokens), so a larger budget costs nothing.
Do not set it below 1,536 for citation answering. A truncated JSON answer loses its whole citations array, so the item scores zero rather than partially: measured on our own runs, a 1,024 cap truncated 5.3% of held-out tax-selection answers and cost about 0.03 macro-F1 on that split.
Thinking mode (chat_template_kwargs: {"enable_thinking": true}) is fully supported: the model reasons in English inside a think block, then answers in Thai. It helps on essays and complex analysis. The benchmark numbers above use thinking off, which is the tuned path for citation tasks; allow roughly 2x the token budget when thinking is on.
Serve with at least --max-model-len 32768: real Revenue Code contexts overflow smaller windows. Raise it toward 262144 when you need to hold long statute sets or whole ruling bundles in one prompt.
Limitations and responsible use
Retrieval quality drives results. Given the right sections, citations are near-perfect; if the retriever supplies the wrong sections, the answer follows them. For production RAG, pair the model with a sound retriever and monitor retrieval quality.
Hardest task, near-miss rejection. Tax selection (picking the applicable section among close alternatives) is the lowest-scoring axis for every model, ours included; telling a governing section from a closely related one is where errors concentrate, so retrieval tuning and human review are advised for high-stakes use.
Feed a focused context. The 262,144-token window is there when you need it, but citation accuracy is highest when the prompt holds a handful of candidate sections. The more unrelated sections you pass, the harder near-miss rejection becomes: for citation tasks, pass a tightly retrieved set (a page or few, not the whole code) rather than filling the window. Reach for the long window when the task genuinely spans many documents, and note that the benchmark results above were measured at a 32,768 window.
Structured output is the strong path. The JSON citation contract is ideal for automation and legal-tech pipelines; free-form chat is supported but less structured.
Scope. Trained on Thai statutory law (Revenue Code, Civil and Commercial Code, related acts) and Revenue Department rulings; coverage of niche areas or very recent amendments may be thinner, and statutes change over time.
Human oversight required. Outputs are decision support, not legal advice. A qualified professional should verify every citation against the current law before relying on it.
Data provenance. Some essay training data derives from Thai bar examination materials; confirm redistribution terms for your jurisdiction and use case.
