A 390M-parameter, from-scratch language model specialized in COBOL — small enough to run locally and on-prem, trained entirely in-house, that outperforms 7B general code models on COBOL code generation.

🔗 Watch it grow: https://skyl4r.ai

A from-scratch Skylar model generates COBOL; GnuCOBOL compiles and runs it

_{A from-scratch Skylar model completes a COBOL task → GnuCOBOL compiles it → it runs → correct output. No internet, no API, no third-party weights.}

TL;DR

~390M parameters (386M exact), decoder-only, Qwen3-style architecture, trained 100% from scratch (no warm-start from anyone else's weights).
Domain: COBOL — the language that still runs banking, insurance and public-administration back-ends, where general code models perform poorly and skilled developers are retiring.
Sovereign: runs on a single local GPU, no data leaves your infrastructure, no dependency on any external LLM provider.
On COBOLEval (146 HumanEval-derived COBOL problems, compiled and executed with GnuCOBOL), it beats Qwen2.5-Coder-7B, CodeLlama-7B and StarCoder2-7B on pass@1 — at 18× fewer parameters.

Results — head-to-head on COBOLEval

All models evaluated on the same harness, greedy decoding, seed 0, GnuCOBOL compile+execute, official {NAME}.TXT scoring.

Ordered by pass@1 (the benchmark). Skylar-390M-Cobol is #1 among all deployable models — only the much larger reference teacher scores higher.

Model	Params	CSR (compile rate)	pass@1
Claude Opus 4.8 (reference ceiling, not deployable)	—	96.6%	81.5%
Skylar-390M-Cobol (this model)	390M	82.2%	5.5%
Qwen2.5-Coder-7B-Instruct	7B	6.2%	2.1%
CodeLlama-7B-Instruct	7B	6.8%	0.7%
StarCoder2-7B	7B	48.6%	0.0%

COBOLEval — Skylar-390M-Cobol vs 7B code models, same harness

_{pass@1 = 8/146 problems solved. 95% binomial CI ≈ [2.4%, 10.5%] — low absolute, as expected for
a 390M model on an execution benchmark; the point is the relative result vs the 7B baselines on the
identical harness.}

Why this comparison is trustworthy: our measured baseline scores match the published literature (StarCoder2 ~0%, CodeLlama ~0%, Qwen2.5-Coder ~0.68% on COBOLEval). The harness does not favor us — it reproduces the competitors' official numbers, and on that same harness the specialized 390M wins.

What the numbers mean. The general 7B models often write pseudo-COBOL — they call mod(), substr(), abs() (functions from Python/Java that don't exist in COBOL), so their code looks right but doesn't compile. Skylar-390M-Cobol writes real COBOL: it compiles 82% of the time and solves the most problems. This is the value of specialization.

Decontamination

We audited overlap between training data and the COBOLEval test set. 4 problems (below_zero, count_upper, find_max, is_sorted) appeared in both; we removed them from the training set and retrained so the released model has never seen any test problem. The numbers above (5.5% / 82.2%) are reported on this clean model — and as a sanity check, none of the 4 formerly-overlapping problems are among the solved ones, confirming the score carries no leakage.

Intended use & limitations

Intended use: a supervised assistant for COBOL developers — scaffolding, boilerplate, DIVISION/LINKAGE structure, syntax, completion — where a human reviews the output.

Limitations (read this):

pass@1 is 5.5% in absolute terms — low. COBOLEval is hard (real execution), but this model is not capable of autonomously writing correct COBOL programs end-to-end. It is capacity- limited: it compiles well (82%) but its algorithmic reasoning is weak.
Do not use it unsupervised or for safety-critical code.
English/COBOL only; not a general chat model.

⚠️ Research preview (v0.1). This is an honest proof-of-concept, not a production tool. It demonstrates that a small, sovereign, specialized model can beat much larger generalist code LLMs on a niche legacy language. Read the Limitations before using it. Numbers are reported with full methodology and a decontamination audit — no hidden asterisks.

How it was built

Pretrain (from scratch): 8.19B tokens, mix of permissive code (oversampled), Italian/EU legal text, and COBOL, Chinchilla-optimal, code-aware BPE tokenizer (vocab 32768).
SFT: instruction tuning on COBOL completion, with verified distillation — a strong teacher solves COBOL problems, each solution is kept only if it compiles and passes its tests, then used as training data. Plus targeted synthetic data for structural gaps.
Decontamination + clean retrain (see above).

Roadmap

This 390M is the first step. A larger model (~1B) with more code pretraining and longer context is in progress — that is the version intended to become a genuinely usable COBOL assistant.

How to use it — a COBOL completer, not a chatbot

Skylar-390M-Cobol completes COBOL; it does not chat. You give it a COBOL stub — a skeleton with the task written as comments, ending at WORKING-STORAGE SECTION. — and it writes the WORKING-STORAGE entries + PROCEDURE DIVISION. Do not prompt it with free-form natural language ("write me a program that…", and especially not in Italian): it will ramble, not code.

bash

pip install skylar
skylar cobol --example                      # the built-in demo stub
skylar cobol --stub-file my_task.cbl --compile   # your own stub → completed + GnuCOBOL check

python

import skylar
m = skylar.load("Skyl4r-Ai/Skylar-390M-Cobol")
print(m.complete_cobol(my_stub))            # reassembles a full, compilable program

✅ Tasks it handles reliably — simple list / loop / accumulate / string work, e.g. max_element, sum_to_n, strlen, incr_list, is_palindrome, flip_case, pairs_sum_to_zero, count_up_to. Give it a stub shaped like these.

✅ Worked example — stub in → compilable program out (real output, cobc accepts it):

cobol

*> INPUT STUB (ends at WORKING-STORAGE SECTION.):
       01 LINKED-ITEMS.
           05 L-L OCCURS 3 TIMES INDEXED BY NI PIC S9(10).
           05 RESULT PIC S9(10).
      * Return maximum element in the list.
      * >>> max_element([5, 3, -5])  ->  5
       WORKING-STORAGE SECTION.

*> Skylar-390M-Cobol COMPLETES it:
       01 WS-I PIC 9(4).
       PROCEDURE DIVISION USING LINKED-ITEMS.
           MOVE L-L(1) TO RESULT
           PERFORM VARYING WS-I FROM 2 BY 1 UNTIL WS-I > 3
               IF L-L(WS-I) > RESULT  MOVE L-L(WS-I) TO RESULT  END-IF
           END-PERFORM
           GOBACK.
       END PROGRAM MAX-ELEMENT.

⚠️ Secondary mode: a precise technical English instruction also works, e.g. skylar generate --prompt "Write a COBOL PROCEDURE DIVISION that sums WS-NUMBERS into WS-TOTAL and displays it." — but the stub format above is where it's strongest.

❌ Don't: chat with it, prompt in natural Italian, or expect correct programs for hard tasks (pass@1 is 5.5% — it compiles ~82% of the time but its algorithmic reasoning is weak). Always review the output.

import skylar also registers the architecture with 🤗 Transformers, so AutoModelForCausalLM.from_pretrained("Skyl4r-Ai/Skylar-390M-Cobol") works too. The weights (model.safetensors) and tokenizer.json are standard; the architecture is a custom decoder (NanoTransformer, model_type: nano-transformer — Qwen3-style RMSNorm + RoPE + GQA + QK-Norm + SwiGLU), described in full in the accompanying paper.

License & attribution

Apache-2.0. IP: A. Ivanovitch (CEO of Skylar Project). Built in-house, no third-party pretrained weights.