PermuFormer is a small Llama-style causal language model trained on symbolic permutation tasks from algebraic combinatorics. It is intended as a specialist base model for permutation representation, reasoning, and finetuning experiments rather than as a general natural-language assistant.
The model operates on a compact word-level vocabulary for permutation syntax. Training examples are stored as pre-tokenized lists of tokens; at inference time, the Hugging Face tokenizer can also consume equivalent whitespace-separated strings. Prompts are formulaic equations: the left side specifies a permutation task and generation begins after the = token.
PermuFormer was trained autoregressively on synthetic permutation examples generated with exact combinatorial algorithms. The paper describes a dataset of 39.8M instances, approximately 2.66B tokens, over the symmetric groups S_2 through S_11.
Training tasks cover three broad families:
Translation between encodings: one-line notation, cycle notation, reduced Coxeter expressions, RSK tableaux, inversion vectors, and Lehmer codes.
Permutation statistics and properties: length, descents, fixed points, sign/parity, cycle type, RSK shape, pattern avoidance, longest increasing/decreasing subsequences, and related statistics.
Algebraic operations and comparisons: product/composition, inverse, powers, conjugation, commutator, relative products, multiplication by simple transpositions, complement, reverse, descent tests, and Bruhat order.
Some targets include computational witnesses before the final answer, for example inversion lists before a length answer or pattern witnesses before an avoidance answer.
Usage
Use deterministic decoding for most evaluation-style tasks. Make sure special token IDs come from the tokenizer.
Training data is represented as lists of token strings. When writing prompts as plain text, separate every token with spaces. Multi-digit integers, delimiters, and task names are individual tokens. A typical example starts with <|endoftext|>, then a size token such as n7, then the task expression, then =.
The training code evaluates by exact match on the generated right-hand side after =. The local training log for this repository reports, at step 2,522,000 on a 2,560-example stratified evaluation sample:
Overall exact match: 98.44%
Translation: 97.78%
Property/statistic tasks: 99.17%
Algebraic tasks: 98.36%
These figures are from the local log and should be treated as checkpoint-adjacent repository metadata, not a full benchmark report for every downstream setting.
The paper also reports that PermuFormer is substantially more accurate than frontier general-purpose LLMs on a small held-out sample from the model's symbolic test distribution, while noting that the comparison is imperfect because PermuFormer was trained directly in this syntax.
Finetuning
PermuFormer is designed to be finetuned on specialized permutation tasks. Experiments in the paper include:
231-avoidance and 2143-avoidance
mHeight
Schubert polynomial structure constants
Kazhdan-Lusztig polynomial degree prediction
The repository's finetuning scripts compare starting from this pretrained checkpoint with training the same architecture from scratch.
Limitations
This is a specialist symbolic model. It expects the exact whitespace-tokenized syntax used during training and is brittle to natural-language paraphrases or malformed prompts.
The model is trained on permutations of sizes represented in the training data, primarily S_2 through S_11; behavior outside that regime is not guaranteed.
Exact-match accuracy depends on canonical output formatting. Some mathematical tasks may have multiple valid answers, but evaluation expects the chosen canonical form.
The model focuses on permutations. It does not natively handle broader combinatorial structures such as arbitrary graphs or partitions unless encoded through the supported task syntax.
Outputs should be verified by exact combinatorial software for research-critical use.
Citation
If you use this model, please cite the accompanying PermuFormer paper once citation details are available.
