Csermely is a 190M parameter decoder-only transformer trained exclusively on high-quality Hungarian text. It runs on edge devices and excels in summarization, grammar checking, and tone detection.

Model Details


Version	0.2
Parameters	190.2M
Context length	4,096 tokens (YaRN RoPE, 4× factor)
Architecture	LLaMA-style (decoder-only transformer)
Training context	1,024 tokens
Training precision	bfloat16 (MLX)
Published weights	float16
Vocabulary	32,000 (SentencePiece Unigram, Hungarian)
Training data	~2B tokens of Hungarian text
License	MIT

Architecture

16 transformer layers
896 hidden dimension
14 attention heads
2560 FFN intermediate size
RMSNorm pre-layer normalization
Rotary positional embeddings (RoPE) with YaRN extension (4× factor, base 1024)
SwiGLU feed-forward activation
Tied input/output embeddings

Tokenizer

Custom 32K vocabulary SentencePiece Unigram tokenizer trained on high-quality Hungarian corpora. ~30% more token-efficient than multilingual tokenizers for Hungarian text.

Available separately: emese-tech/emese-tokenizer-32k

Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("emese-tech/csermely")
model = AutoModelForCausalLM.from_pretrained("emese-tech/csermely")

input_text = "A magyar nyelv"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

The default generation config uses temperature=0.7, top_p=0.9, and repetition_penalty=1.2 to reduce repetitive output.

Citation

bibtex

@misc{emese-csermely-2026,
  title={Csermely: A Hungarian Language Model},
  author={Emese Tech},
  year={2026},
  url={https://huggingface.co/emese-tech/csermely}
}