You can deploy this model and start earning money today!

tuandunghcmut/
gemma-4-E4B-it-text-only-non-thinking

Name: gemma-4-E4B-it-text-only-non-thinking
Availability: InStock

tuandunghcmut/gemma-4-E4B-it-text-only

finetune

gemma

Views

No views yet

gemma-4-E4B-it-text-only-non-thinking

A tokenizer-only variant of tuandunghcmut/gemma-4-E4B-it-text-only with a modified chat template that permanently forces non-thinking mode (enable_thinking = false), regardless of any argument passed by the caller.

⚠️ This repo contains no model weights. It is designed to be used as a --tokenizer override with vLLM (or any framework that supports separate model/tokenizer repos). The weights live in the source repo.

What changed

A single two-line override is prepended to the top of chat_template.jinja (and embedded in tokenizer_config.json):

jinja

{#- Force non-thinking mode: override any caller-supplied enable_thinking to false -#}
{%- set enable_thinking = false -%}

This ensures the <|think|> token is never injected into the system prompt, so the model always responds directly without a reasoning/thinking preamble.

Deploying with vLLM ✅

No weight download needed for this repo. Load the model weights from the source and the tokenizer (with non-thinking template) from this repo:

bash

vllm serve tuandunghcmut/gemma-4-E4B-it-text-only \
  --served-model-name gemma-4-E4B-it-text-only-non-thinking \
  --tokenizer tuandunghcmut/gemma-4-E4B-it-text-only-non-thinking \
  --max-model-len 32768 \
  --dtype bfloat16

Then query via the OpenAI-compatible API:

python

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy")

response = client.chat.completions.create(
    model="gemma-4-E4B-it-text-only-non-thinking",
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ],
)
print(response.choices[0].message.content)
# ➜ Direct answer, no thinking preamble

Deploying with 🤗 Transformers

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load weights from source, tokenizer (with non-thinking template) from this repo
model = AutoModelForCausalLM.from_pretrained(
    "tuandunghcmut/gemma-4-E4B-it-text-only",
    dtype=torch.bfloat16,
    device_map="auto",
)
tok = AutoTokenizer.from_pretrained(
    "tuandunghcmut/gemma-4-E4B-it-text-only-non-thinking"
)

messages = [{"role": "user", "content": "What is the capital of France?"}]

# enable_thinking=True is silently ignored — template always uses non-thinking
prompt = tok.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=True
)

inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=200)
print(tok.decode(out[0], skip_special_tokens=True))

Why this is useful

Scenario	Problem	Solution
vLLM / llama.cpp serving	No per-request `enable_thinking` toggle	Use `--tokenizer` pointing here
OpenAI API clients	Can't pass custom template vars	Template forces non-thinking always
Batched inference	Some callers accidentally pass `enable_thinking=True`	Template override makes it safe
Latency-sensitive apps	Don't want thinking overhead	No thinking tokens = faster TTFT

Behaviour comparison

Caller code	`gemma-4-E4B-it-text-only`	this repo
`enable_thinking=True`	✅ thinking mode	❌ forced non-thinking
`enable_thinking=False`	✅ non-thinking	✅ non-thinking
no flag (default)	✅ non-thinking	✅ non-thinking

Source model

Tokenizer + template source: tuandunghcmut/gemma-4-E4B-it-text-only
Original model: google/gemma-4-E4B-it

License

Inherits the Gemma license from the base model.