LFM2.5-1.2B-JP-202606 is our latest general purpose Japanese chat model, delivering significant improvements in knowledge, instruction following, math, code, and tool-use over both the models of comparable size and LFM2.5-1.2B-JP. It sets a new benchmark for state-of-the-art performance in Japanese language understanding.
Ideal for developers building Japanese-language applications where cultural and linguistic nuance matter.
Find more information about LFM2.5 in our blog post.
📊 Performance
Liquid AI
We compared LFM2.5-1.2B-JP-202606 with relevant sub-2B models on a diverse suite of benchmarks.
Model
Size
Knowledge
Instruction Following
Math
Code
Tool Use
Domain Avg
JMMLU‑ProX
JMMLU
JCulture
JGPQA
Avg
J‑MIFEval
JFBench1
Avg
J‑GSM8K
J‑MATH500
Avg
JHumanEval+
J‑BFCLv32
LFM2.5‑1.2B‑JP‑202606
1.2B
36.23
54.19
35.77
28.69
38.72
79.08
54.77
66.93
62.20
62.80
62.50
49.39
48.00
53.11
LFM2.5‑1.2B‑Instruct
1.2B
31.42
47.61
28.42
31.72
34.79
40.44
36.67
38.56
50.20
50.00
50.10
28.66
46.29
39.68
Qwen3‑1.7B (Instruct)
1.7B
30.78
47.67
33.33
26.26
34.51
40.29
36.61
38.45
46.00
56.40
51.20
47.56
52.45
44.83
Granite‑4.0‑1B
1.5B
15.32
33.93
34.38
24.44
27.02
27.56
31.26
29.41
42.80
25.40
34.10
51.22
50.57
38.46
Llama‑3.2‑1B‑Instruct
1.2B
15.91
33.97
22.52
32.32
26.18
24.10
21.78
22.94
25.20
11.40
18.30
17.68
21.06
21.23
Gemma‑3‑1B‑it
1.0B
14.12
34.45
23.42
24.24
24.06
26.31
31.15
28.73
33.60
15.60
24.60
25.00
17.26
23.93
sarashina2.2‑1b‑instruct‑v0.1
1.4B
18.3
40.24
25.53
26.26
27.58
21.9
27.41
24.66
44.4
24.8
34.60
21.95
13.86
24.53
TinySwallow‑1.5B‑Instruct
1.5B
21.51
47.98
31.17
29.29
32.49
36.55
34.25
35.40
47.2
22.4
34.80
26.83
11.7
28.24
llm‑jp‑3.1‑1.8b‑instruct4
1.9B
17.44
43.05
27.42
17.68
26.40
33.77
30.92
32.35
52.8
17.0
34.90
35.37
11.76
28.16
RakutenAI‑2.0‑mini‑instruct
1.5B
11.46
31.84
29.67
22.22
23.80
28.06
24.66
26.36
24.8
11.4
18.10
28.6
11.85
21.74
1 JFBench is evaluated using single-instruction prompts. 2 quickTestingOSSHandler is used for models that do not support function calling (sarashina2.2‑1b‑instruct‑v0.1, TinySwallow‑1.5B‑Instruct, llm‑jp‑3.1‑1.8b‑instruct4, and RakutenAI‑2.0‑mini‑instruct).
MLX format for Apple Silicon. Optimized for fast inference on Mac devices using the MLX framework.
We recommend using it for agentic workflows, tool use, structured outputs, bilingual English–Japanese assistants, and on-device personal-assistant applications. It is not recommended for knowledge-intensive tasks. It performs best when given clear, explicit instructions that define the task, expected behavior, and output format.
Function definition: We recommend providing the list of tools as a JSON object in the system prompt. You can also use the tokenizer.apply_chat_template() function with tools.
Function call: By default, LFM2.5 writes Pythonic function calls (a Python list between <|tool_call_start|> and <|tool_call_end|> special tokens), as the assistant answer. You can override this behavior by asking the model to output JSON function calls in the system prompt.
Function execution: The function call is executed, and the result is returned as a "tool" role.
Final answer: LFM2 interprets the outcome of the function call to address the original user prompt in plain text.