As of 2026-08-18 no GGUF of the dense 33B variant existed (official GGUF releases cover only the 8b and 32b-a3b variants), and the upstream convert_hf_to_gguf.py cannot convert the llm-jp-4 Unigram tokenizer correctly, so this repository fills that gap. All credit for the model goes to the LLM-jp project.

llm-jp/llm-jp-4-33b-thinking (LLM-jp / 国立情報学研究所による日本語特化 dense 33B thinking モデル) の非公式 GGUF 変換です。dense 33B の GGUF は公式・コミュニティとも存在しなかったため変換・公開しています。

Files

File	Size	Note
`llm-jp-4-33b-thinking-Q4_K_M.gguf`	20.2 GB	recommended for inference
`llm-jp-4-33b-thinking-BF16.gguf`	66.4 GB	lossless; for re-quantization
`llmjp-b10442.patch`	—	required llama.cpp patches (see below)
`convert-llmjp4-33b.py`	—	conversion script used to produce these files

⚠️ Runtime requirements — read this first

llama.cpp needs two patches to run llm-jp-4 GGUF models correctly (llmjp-b10442.patch in this repo, written against b10442; both issues also affect the official llm-jp-4 8b/32b-a3b GGUFs):

trigger-lstrip (ported from the LLM-jp fork of llama.cpp, not yet upstream): without it, a spurious leading space is emitted after special tokens and Harmony-style chat parsing (reasoning_content extraction) breaks.
UGM byte_fallback (ours): upstream llama.cpp's UGM (Unigram) tokenizer has no byte-fallback, so every \n / \t in your prompt becomes <|unk|> (reproducible with the official 8b GGUF on upstream and on the LLM-jp fork). The patch implements the same "replace unk spans with <0xXX> byte tokens" post-processing as HF tokenizers' byte_fallback: true. With it, tokenization is byte-exact vs. AutoTokenizer (verified on Japanese text, multi-line code, CRLF and rare characters).

bash

git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
git checkout b10442        # patch written against this tag; nearby tags likely fine
git am llmjp-b10442.patch  # or: git apply
cmake -B build -DCMAKE_BUILD_TYPE=Release   # add -DGGML_VULKAN=ON / -DGGML_CUDA=ON as needed
cmake --build build -j --target llama-server llama-cli

Usage

bash

llama-server -m llm-jp-4-33b-thinking-Q4_K_M.gguf --jinja -ngl 99 \
  --ctx-size 32768 --host 0.0.0.0 --port 8093

Reasoning effort (low / medium / high, default medium) can be set per request:

json

{"model": "llm-jp-4-33b-thinking",
 "messages": [{"role": "user", "content": "..."}],
 "chat_template_kwargs": {"reasoning_effort": "low"}}

Responses separate the analysis channel into reasoning_content (English — by design of the model) and the final answer into content.

Measured on AMD Ryzen AI Max+ 395 (Strix Halo, Vulkan, 256-bit LPDDR5X): ~11 tok/s generation, ~146 tok/s prompt processing (Q4_K_M).

Conversion notes

Vocab was built directly from tokenizer.json (HF Unigram) as GGUF tokenizer.ggml.model = "t5" (UGM), mirroring the representation used by the official llm-jp-4-8b-thinking-gguf: 352 special tokens as CONTROL with score -1000.0, 256 byte tokens, add_space_prefix=true, remove_extra_whitespaces=false.
Before conversion, all 196,608 tokens/scores/types and every tokenizer.* metadata key including the chat template were verified identical to the official 8b GGUF (the two models share llm-jp-tokenizer v4.0). See convert-llmjp4-33b.py.
Q4_K_M was produced with llama-quantize from the BF16 GGUF, without an importance matrix (the official 8b release uses one; contributions of an imatrix-based re-quantization from the BF16 file are welcome).

License

Apache License 2.0, inherited from the base model. This is an unofficial conversion; the model was created and trained by the LLM-jp project.

idxx/llm-jp-4-33b-thinking-gguf

llm-jp-4-33b-thinking-gguf (unofficial community conversion)

Files

⚠️ Runtime requirements — read this first

Usage

Conversion notes

License

idxx/
llm-jp-4-33b-thinking-gguf