squeal_ai_20m-instruct is a compact ~20M parameter language model, fine-tuned for instruction following on Russian-language data. It belongs to the squeal_ai_ model family by Squeal Studio.

This is the instruct version of squeal_ai_20m-base, fine-tuned with Supervised Fine-Tuning (SFT) on an instruction dataset. No RLHF or DPO alignment stage was applied.

Research and Educational Model. This model is designed for research, educational purposes, and experimentation. Given its parameter count and training data volume, performance on complex instructions or factual tasks will be limited.

Model Description

Architecture: Qwen2.5-style transformer decoder with GQA (Grouped Query Attention)
Parameters: ~20M
Tokenizer: Custom BPE, vocab_size = 24,000
Context length: 1,024 tokens
Base model: squeal_ai_20m-base
Fine-tuning method: Supervised Fine-Tuning (SFT) only, no RLHF/DPO

Architecture Details

Parameter	Value
hidden_size	352
num_hidden_layers	8
num_attention_heads	8
num_key_value_heads	4
intermediate_size	1024
max_position_embeddings	1024
vocab_size	24,000

Training Details

Base model: squeal_ai_20m-base, pretrained on cultura_ru_edu, fineweb2_ru, and Russian Wikipedia
SFT dataset: This model was fine-tuned using ru_turbo_alpaca, created by IlyaGusev
- Dataset License: CC-BY 4.0
- Modifications: Added examples of self-identification and reformatted to a plain User: / Bot: prompt structure
Training setup: Trained on a Tesla T4 (fp16) up to step 2,400
Chat template: none — no chat_template.jinja is included; prompts must be manually formatted as shown below

Usage

This model does not use tokenizer.apply_chat_template. Prompts must follow the plain-text format below:

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Squeal-Studio/squeal_ai_20m-instruct")
tokenizer = AutoTokenizer.from_pretrained("Squeal-Studio/squeal_ai_20m-instruct")

prompt = "User: Как поднять настроение?\nBot:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Scope & Limitations

Very small parameter count means limited factual knowledge and reasoning ability
SFT only, no RLHF or DPO, so alignment and instruction-following robustness are limited compared to larger aligned models
No chat template is provided, the model expects the exact User: ... \nBot: format shown above and may behave unpredictably with other prompt structures
May produce repetitive, incoherent, or factually incorrect responses
Not intended for production, safety-critical, or high-stakes use cases
Best suited for architectural experiments, baseline comparisons, and educational purposes

License

Apache 2.0