from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ThingAI/Dwarf-15M-Instruct", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("ThingAI/Dwarf-15M-Instruct", trust_remote_code=True)

prompt = "<|user|>\nFind all Python files modified in the last 3 days\n<|end|>\n<|assistant|>\n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=False))
# → find . -name '*.py' -mtime -3

What It Does

Dwarf-15M takes natural language descriptions of tasks and produces the corresponding Linux/bash command:

Prompt	Output
Show current date	`date`
List files	`ls`
Kill process 1234	`kill 1234`
Delete all .tmp files in current directory	`rm ./*.tmp`
Compress the /home/user/project folder	`tar -czf project.tar.gz /home/user/project/`
Check if port 8080 is in use	`ss -tlnp \| grep 8080`
Restart the nginx service	`sudo systemctl restart nginx`
Find all files containing TODO	`grep -rl 'TODO' .`
Change owner of /var/www to www-data	`sudo chown -R www-data:www-data /var/www`
Run script.sh in background and log output	`nohup ./script.sh > log.txt 2>&1 &`
Find and replace foo with bar in config.txt	`sed -i 's/foo/bar/g' config.txt`
What does chmod 755 do?	chmod 755 sets read+write+execute for owner, read+execute for group and others.
Write a bash function that counts lines	`count_lines() { wc -l < "$1"; }`

Architecture

Parameter	Value
Parameters	15.54M
Layers	12
Hidden dim	320
Attention	GQA (5 query, 1 KV head)
FFN	SwiGLU (d_ff=864)
Normalization	RMSNorm
Positional	RoPE (θ=10000)
Vocabulary	8,202 (DwarfGoToken)
Max sequence	2,048
Weight tying	Yes (embed ↔ lm_head)

Training

Pretraining: 21.6B tokens (ratio 1,390:1) on 11 datasets:

Shell/bash: The Stack (shell, batchfile), GunA-SD/bash_code — 38.5%
Code: The Stack (Python, C), CodeFeedback — 39.1%
Instructions: ShellLife (52K NL→command), rlvr-code-data-bash (133K problems) — 11%
English: helpful-instructions, FineWeb — 10.3%
CoT: Magpie-Reasoning — 1.1%

SFT: 557 curated Linux command pairs, 5 epochs, lr=4e-5. Training time: 19 seconds.

Tokenizer: DwarfGoToken — 8,202 token BPE with syntax-aware pre-tokenization for shell operators (2>&1, &&, >>).

Chat Template

<|user|>
Your question here
<|end|>
<|assistant|>
Model response here
<|end|>

Intended Use

Dwarf-15M is designed as a CLI assistant that suggests commands for user review before execution. It is NOT a general-purpose chatbot. Best results on:

Simple to medium Linux commands (file ops, process management, networking)
Bash one-liners and short functions
Command explanations ("What does chmod 755 do?")

Limitations

15M parameters — cannot handle complex multi-step reasoning
May produce incorrect commands for unusual or very specific requests
Should always be used with human review before executing any suggested command
English only
Trained primarily on Ubuntu/Debian commands

Hardware

Pretrained on RTX 3070 (8GB VRAM) at ~127K tokens/sec
Inference: runs on any hardware including CPU, ~30MB model size

License

Apache 2.0

Citation

bibtex

@misc{dwarf15m2026,
  title={Dwarf-15M-Instruct: A Shell Specialist Language Model},
  author={ThingsAI},
  year={2026},
  url={https://huggingface.co/ThingAI/Dwarf-15M-Instruct}
}