Views
No views yet

| Benchmark/Model | OpenJAI-v1.0-14b | Qwen3-14b | Typhoon2.1-gemma3-12b | OpenThaiGPT1.5-14b | GPT-4.1-nano-2025-04-14 |
|---|---|---|---|---|---|
| Instruction Following | |||||
| IFBench-EN | 32.4 | 29.7 | 27.4 | 30.6 | 28.3 |
| IFBench-TH | 39.4 | 38.1 | 36.5 | 35.4 | 34.9 |
| Multi-turn Capability | |||||
| MT-Bench-EN | 8.4 | 8.4 | 8.3 | 7.8 | 8.5 |
| MT-Bench-TH | 8.1 | 8.0 | 8.1 | 6.9 | 8.0 |
| Long-context Understanding | |||||
| MRCR | 18.9 | 18.3 | 16.9 | 16.9 | 16.2 |
| LongBench-v2 | 33.6 | 32.4 | 29.2 | 33.6 | 28.8 |
| Tool Calling | |||||
| BFCL-v3-EN | 60.5 | 59.2 | 52.2 | 52.9 | 53.1 |
| BFCL-v3-TH | 47.0 | 46.0 | 45.0 | 44.9 | 41.1 |
| General Knowledge | |||||
| MMLU-ProX-lite-EN | 66.0 | 66.6 | 55.1 | 64.3 | 36.3 |
| MMLU-ProX-lite-TH | 54.7 | 57.5 | 45.2 | 49.3 | 39.8 |
transformers library.1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model_name = "JTS-AI/OpenJAI-v1.0-14B"
4
5# load the tokenizer and the model
6tokenizer = AutoTokenizer.from_pretrained(model_name)
7model = AutoModelForCausalLM.from_pretrained(
8 model_name,
9 torch_dtype="auto",
10 device_map="auto"
11)
12
13# prepare the model input
14prompt = "แนะนำที่เที่ยวแถวสยามหน่อย"
15messages = [
16 {"role": "user", "content": prompt}
17]
18text = tokenizer.apply_chat_template(
19 messages,
20 tokenize=False,
21 add_generation_prompt=True,
22 enable_thinking=False # Switches between thinking and non-thinking modes. Default is True.
23)
24model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
25
26# conduct text completion
27generated_ids = model.generate(
28 **model_inputs,
29 max_new_tokens=1024
30)
31output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()
32content = tokenizer.decode(output_ids, skip_special_tokens=True).strip("\n")
33
34print("Content:", content)[!NOTE] OpenJAI-v1.0 is optimized for non-thinking mode. While the base model's thinking mode may be accessible, its performance is not guaranteed.
Qwen-Agent by adapting the model configuration.1from qwen_agent.agents import Assistant
2
3# Define LLM, pointing to your OpenJAI-v1.0 endpoint
4llm_cfg = {
5 'model': 'JTS-AI/OpenJAI-v1.0-14B',
6
7 # Use a custom endpoint compatible with OpenAI API:
8 'model_server': 'http://localhost:8000/v1', # api_base
9 'api_key': 'EMPTY',
10}
11
12# Define Tools
13tools = [
14 {'mcpServers': { # You can specify the MCP configuration file
15 'time': {
16 'command': 'uvx',
17 'args': ['mcp-server-time', '--local-timezone=Asia/Shanghai']
18 },
19 "fetch": {
20 "command": "uvx",
21 "args": ["mcp-server-fetch"]
22 }
23 }
24 },
25 'code_interpreter', # Built-in tools
26]
27
28# Define Agent
29bot = Assistant(llm=llm_cfg, function_list=tools)
30
31# Streaming generation
32messages = [{'role': 'user', 'content': 'วาดกราฟแสดงราคาหุ้นของ JTS ในช่วง 1 เดือนที่ผ่านมา'}]
33for responses in bot.run(messages=messages):
34 pass
35print(responses)vLLM and SGLang support passing command-line arguments to enable RoPE scaling.vllm, you can usevllm serve ... --rope-scaling '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":32768}' --max-model-len 131072 sglang, you can usepython -m sglang.launch_server ... --json-model-override-args '{"rope_scaling":{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":32768}}'[!NOTE] All the notable open-source frameworks implement static YaRN, which means the scaling factor remains constant regardless of input length, potentially impacting performance on shorter texts. We advise adding therope_scalingconfiguration only when processing long contexts is required. It is also recommended to modify thefactoras needed. For example, if the typical context length for your application is 65,536 tokens, it would be better to setfactoras 2.0.
[!NOTE] The defaultmax_position_embeddingsinconfig.jsonis set to 40,960. This allocation includes reserving 32,768 tokens for outputs and 8,192 tokens for typical prompts, which is sufficient for most scenarios involving short text processing. If the average context length does not exceed 32,768 tokens, we do not recommend enabling YaRN in this scenario, as it may potentially degrade model performance.
@misc{trakuekul2025openjaiv10openthailarge,
title={OpenJAI-v1.0: An Open Thai Large Language Model},
author={Pontakorn Trakuekul and Attapol T. Rutherford and Jullajak Karnjanaekarin and Narongkorn Panitsrisit and Sumana Sumanakul},
year={2025},
eprint={2510.06847},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2510.06847},
}