Chris Linux 124M is a 124,475,904-parameter GPT-2-style causal language model specialized for Linux terminal assistance and natural-language-to-command generation.
This repository contains the Hugging Face Transformers / SafeTensors release of the Core SFT v1, epoch 08 checkpoint:
linux_sft_core_v1/chris_linux_sft_epoch_08.pt
The model was not initialized from OpenAI GPT-2 weights. It was trained from scratch as Chris-GPT-2 124M, continued-pretrained on Linux-oriented technical material, and then instruction-tuned with a compact canonical Linux dataset.
Important: this is a small research model, not a command-execution safety system. Never automatically execute generated shell commands. Review every command before running it, especially commands involving disks, filesystems, permissions, packages, services, networking, or privileged operations.
Model summary
Property
Value
Model name
Chris Linux 124M
Published variant
Core SFT v1 — epoch 08
Architecture
GPT-2-compatible decoder-only Transformer
Parameters
124,475,904
Transformer blocks
12
Attention heads
12
Hidden dimension
768
Context length
1,024 tokens
MLP width
4 × hidden dimension
Activation
GELU, tanh approximation
Attention
causal scaled dot-product attention
Weight tying
token embedding = LM head
Embedding-matrix vocabulary
50,304 rows
Actual tokenizer vocabulary
50,257 tokens
Tokenizer
GPT-2 BPE
Output format
Hugging Face GPT2LMHeadModel
Weight format
SafeTensors
The model uses a 50,304-row embedding matrix for training efficiency, while the GPT-2 tokenizer exposes only 50,257 valid token IDs. The release generation_config.json suppresses IDs 50,257 through 50,303 during generation so that padding-only rows cannot be selected as output tokens.
The base model, Chris-GPT-2 124M, was trained from random initialization on 9,999,745,024 FineWeb-Edu tokens using the GPT-2 BPE tokenizer.
The fixed base checkpoint used for the Linux specialization had:
validation loss: 3.07248;
HellaSwag accuracy: 30.66%;
12 Transformer blocks;
12 attention heads;
768-dimensional hidden states;
1,024-token context length;
124,475,904 parameters.
The Linux specialization did not change the architecture.
2. Linux continued pretraining
The selected parent for Core SFT was the FULL 10-epoch Linux CPT checkpoint.
The Linux corpus was constructed from primary or human-curated technical sources, including:
Linux kernel Documentation/;
Linux man-pages;
tldr pages;
systemd documentation;
util-linux;
iproute2;
Git documentation;
curl;
OpenSSH Portable;
procps-ng;
Bash;
GNU coreutils;
findutils;
grep;
sed;
gawk;
tar;
gzip;
make;
glibc manual;
strace;
lsof;
selected Ubuntu/Debian package documentation and optional local man/info material.
The FULL tokenized corpus contained 21,968,794 training tokens and 375,458 validation tokens. Continued pretraining repeated the corpus for 10 epochs while probabilistically replaying FineWeb-Edu examples.
Selected FULL CPT run statistics:
Metric
Value
Optimizer steps
1,677
Global batch
131,072 tokens
Total token positions processed
219,807,744
Linux token positions
197,378,048
FineWeb-Edu replay token positions
22,429,696
Observed replay fraction
10.204%
Best/final Linux validation loss
2.1599783897
Mean throughput
~39,855 tokens/s
Wall time
~1.55 h
Training GPU
NVIDIA GeForce RTX 5070 Ti 16 GB
The parent CPT checkpoint SHA-256 recorded by the experiment is:
Early experiments showed that domain continued pretraining alone made the model more Linux-like but did not reliably map requests to correct terminal commands. A larger heterogeneous SFT dataset also produced frequent command-family and flag-composition errors.
Core SFT v1 was therefore designed around one simple principle:
diverse inputs, canonical outputs
Instead of assigning many competing shell commands to the same intent, the dataset uses multiple natural-language paraphrases and slot variations while keeping a stable target command for each intent.
Examples of deliberately separated concepts include:
text
1show disk usage
2→ df -h
34show listening TCP ports
5→ ss -lntp
67show active TCP connections
8→ ss -tnp
910what process is using port 8080
11→ sudo lsof -i :8080
1213show process with PID 8080
14→ ps -p 8080
1516show traffic control configuration
17→ tc qdisc show
1819find files larger than one gigabyte
20→ find . -type f -size +1G
2122find files smaller than one gigabyte
23→ find . -type f -size -1G
Core SFT v1 contains:
Split
Examples
Train
3,335
Validation
643
Total supervised
3,978
Held-out Core benchmark
307
The dataset contains 106 intents across 15 categories:
The published Core SFT epoch-08 checkpoint was evaluated greedily on all 271 command examples in the held-out Core benchmark.
Metric
Result
Command examples
271
Exact matches
181
Exact match
66.7897%
Extra unsafe suggestions
0
Extra unsafe rate
0.0%
Generated tokens
1,793
Aggregate Python generation throughput
~136.96 tokens/s
Mean latency
~48.3 ms/example
Median latency
~49.1 ms/example
The benchmark was evaluated with greedy decoding and seed 1337 on an NVIDIA GeForce RTX 5070 Ti.
How to interpret the benchmark
The 66.79% exact-match result is useful but narrow.
The benchmark is literally disjoint from the training and validation question templates, but it is intent-aligned with the same manually curated Core intent catalog. It should therefore be interpreted as a held-out test of the Core SFT design, not as a broad external benchmark of Linux expertise.
Exact string match is also conservative for shell commands: two syntactically different commands may be functionally equivalent. No execution-based equivalence claim is made for this score.
The 28 diagnostic and 8 safety examples in the 307-item benchmark are not included in the 271-command exact-match figure above.
Prompt format
The model was instruction-tuned with the following format:
For command synthesis, deterministic decoding is recommended. Sampling can make a small model more likely to combine nearby utilities, flags, or argument patterns incorrectly.
Conversion to Hugging Face format
The original Chris-GPT-2 implementation uses torch.nn.Linear for the projection layers, while Hugging Face GPT-2 represents the corresponding projections with its Conv1D module.
The converter therefore transposes the required matrices for:
attention c_attn;
attention c_proj;
MLP c_fc;
MLP c_proj.
The exported model is then reloaded with from_pretrained() and checked against the original checkpoint. A publication build should verify that the original and converted models have matching shapes, numerically close logits, and the same argmax over the real 50,257-token vocabulary.
The exact tokenizer file set may vary slightly with the Transformers version used during export.
Intended use
Chris Linux 124M is intended for:
research on small specialized language models;
natural-language-to-shell experiments;
offline/local Linux assistant prototypes;
command classification and intent experiments;
quantization and inference-runtime experiments;
studying the effect of canonical vs. heterogeneous instruction data;
educational analysis of end-to-end language-model training and deployment.
Out-of-scope use
This model should not be treated as:
an autonomous system administrator;
a secure command-execution engine;
a replacement for documentation or expert review;
a reliable source of security-critical commands;
a general-purpose coding assistant;
a guarantee that a generated command is portable across Linux distributions.
Limitations
At 124M parameters, the model has clear capacity limits.
Observed limitations include:
incomplete coverage outside the curated Core intent set;
imperfect generalization to unseen Linux utilities and uncommon phrasing;
possible confusion between nearby command families;
possible flag or argument composition errors;
limited factual and reasoning capability compared with modern general-purpose LLMs;
no guarantee of command portability across distributions or installed tool versions;
incomplete safety generalization for destructive requests expressed in unseen ways.
A small language model can produce a command that is syntactically plausible and still operationally wrong.
Safety
Generated commands are text suggestions only.
Do not automatically pipe model output to a shell. Before executing a command:
inspect the command manually;
understand its flags and target paths;
verify whether it requires elevated privileges;
check distribution/tool compatibility;
use a sandbox or disposable environment for uncertain commands;
add deterministic runtime validation if integrating the model into software.
For a production terminal assistant, a hybrid design is recommended: use the model to interpret natural language, but keep high-risk command construction and execution behind deterministic validation and policy checks.
Training-data provenance and licensing
The model has multiple training stages with different provenance.
Base pretraining used a public FineWeb-Edu 10B-token sample.
Linux CPT used documentation from multiple upstream projects with heterogeneous licenses.
Core SFT v1 was built from a manually curated intent catalog and deliberately excludes automatically generated man-page SFT and NL2Bash examples from its training split.
Because the upstream material does not have one uniform license, this repository uses:
license: other
This is intentionally conservative. Users should review the provenance and licenses of upstream data sources before redistribution or commercial use of derived weights.
Until the Linux-specialization paper has a stable public identifier, the project can be cited by its repository:
bibtex
1@misc{silva2026chrislinux,
2 author = {Christian Rafael de Souza Silva},
3 title = {Chris Linux 124M: A GPT-2-Style Language Model Specialized for Linux Terminal Assistance},
4 year = {2026},
5 howpublished = {GitHub repository},
6 url = {https://github.com/christianrss/chris-gpt-2}
7}
Disclaimer
Chris Linux 124M is an experimental research model. Its outputs may be incorrect or unsafe. The author and contributors do not guarantee the correctness, safety, or suitability of generated shell commands for any system.
