PSEng-14B-preview
A LoRA adapter that teaches
Phi-4 to write
PowerShell conforming to the
PowerShell Engineer
Standard without the Standard in the
context window. Trained locally on Apple Silicon with MLX. No hosted API touched the
data at any stage.
This is a preview from an in-progress run, not a finished model. Read the
limitations before quoting anything from it.
⚠️ This is an MLX adapter, not a PEFT adapter. PeftModel.from_pretrained() will
not load it. The adapter_config.json here uses MLX's schema (lora_parameters,
num_layers, keys), not PEFT's (peft_type, r, target_modules). Use mlx-lm,
as shown below. Apple Silicon required.
Use
The adapter was trained against a 4-bit quantized Phi-4. Build that exact base
first , it is a local conversion, not a Hub repo:
1python -m mlx_lm convert \
2 --hf-path mlx-community/phi-4-bf16 \
3 -q --q-bits 4 --q-group-size 64 \
4 --mlx-path phi-4-4bit
Then generate:
1python -m mlx_lm generate \
2 --model phi-4-4bit \
3 --adapter-path <path to this adapter> \
4 --max-tokens 1600 --temp 0.0 \
5 --prompt "Write a function that disables Active Directory accounts inactive for more than a given number of days."
Standard-conforming functions are long: comment-based help with three examples,
validated parameters, ShouldProcess where state changes. Give it 1,600 tokens. Greedy
decoding (--temp 0.0) is what it was evaluated under.
Applying the adapter to a differently-quantized base may not reproduce the results
below.
What it scores, and what that does and does not mean
Measured on a 48-task development split. This is not the project's frozen
benchmark, which is reserved for a pre-registered evaluation this model has not yet
undergone.
| Condition (same 48 dev tasks) | First attempt | After one repair |
|---|
| Phi-4 base, plain prompt | 16.9 | 31.2 |
| Phi-4 base, full Standard in system prompt | 18.2 | 41.2 |
| PSEng-14B-preview | 59.3 | 73.8 |
Scores are 0–100 under a static rubric: PowerShell's own parser, an AST walk for
Standard structure, PSScriptAnalyzer under pinned settings, and a validator that rejects
invented cmdlets and parameters. Any parse failure or invented surface scores the task
zero.
Putting the entire Standard in Phi-4's context bought about one point. Training on it
bought forty-two. That gap is a caution about prompt-based delivery generally, not a
claim that this model is good.
These numbers are not comparable to the PSEng-8B flagship's published benchmark
scores. Different task set, different base model. A like-for-like number will exist
only after the pre-registered evaluation.
Limitations
- Preview. Trained on 517 pairs, stopped at iteration 500 of a planned 1,200 on a
shallow, noisy validation minimum. A fuller corpus and a proper hyperparameter sweep
are the next steps and are expected to supersede this artifact.
- Invented surface is the dominant failure. 17 of 48 dev tasks scored zero, 12 of
them for cmdlets or parameters that do not exist. On tasks that clear the gates it
averages 91.8, so the distribution is bimodal. Verify any command it emits against
Get-Help before running it.
- Conformance is not correctness. Nothing in the evaluation executes the generated
code. It measures whether the code follows the Standard, not whether it does the right
thing.
- It will not beat a frontier model. It is small, private, offline, free, and good at
one language.
- Windows-only surface (Active Directory, Exchange) is validated against captured
manifests rather than a live system.
Training
- Base: Phi-4 (14B, MIT),
mlx-community/phi-4-bf16 revision e9ebdf8d, converted
locally to 4-bit (group size 64, affine).
- Method: QLoRA, rank 32, 20 of 40 layers, targeting Phi's fused
self_attn.qkv_proj
and mlp.gate_up_proj plus self_attn.o_proj and mlp.down_proj. lr 1e-5, batch 1,
sequence 3,072, completion-only loss, seed 20260805.
- Corpus: 517 train / 43 valid pairs, every accepted pair scoring 100/100 under the
frozen rubric. Teacher: Qwen3-Coder-30B-A3B-Instruct, run locally. Task prompts carry
no Standard text , the model must have internalised it.
- Data hygiene: development and benchmark tasks are excluded from training by fatal
invariant checks; completions that merely echo the Standard's own reference
implementation are filtered out.
Licence and attribution
MIT, inherited from Phi-4 (Copyright (c) Microsoft Corporation). Its notice travels with
this adapter in NOTICE.
What is in this download: weight deltas trained on top of Phi-4, and nothing else. No
third-party model weights are included, and no model other than Phi-4 is needed to run it.
The training corpus was generated during dataset construction by Qwen3-Coder-30B-A3B-Instruct
(Apache 2.0), running locally. That model is not in the inference path and none of its weights
are distributed here. Task material derived from MicrosoftDocs repositories is CC BY 4.0.
Full attribution for all three is in NOTICE.
PowerShell is a trademark of Microsoft Corporation. This project is not affiliated with,
endorsed by, or sponsored by Microsoft.
