The experiment reproduces the Phase 1 configuration in Liquid-CLI's train_unsloth_processed.py, changing only the base model and using native Transformers/TRL FlashAttention support for LFM2.5.

Training data

139,841 non-coding terminal-agent conversations prepared from the skill_based_easy, skill_based_medium, and skill_based_mixed configurations of nvidia/Nemotron-Terminal-Corpus using Liquid-CLI's prepare_data.py.

Training configuration

Full-parameter BF16 fine-tuning
Sequence length: 4,096
Packing: enabled
Assistant-only loss: enabled
Per-device batch size: 4
Gradient accumulation: 32 (effective batch size 128)
Epochs: 1
Learning rate: 2e-5
Optimizer: AdamW 8-bit
Warmup steps: 10
Weight decay: 0.01
Scheduler: linear
Seed: 3407
Attention: FlashAttention 2
Hardware: 1x NVIDIA H100 80GB

Training results

Optimizer steps: 1,092 / 1,092
Final epoch: 1.0
Training loss: 0.2867
Mean token accuracy: 0.8947
Tokens processed: 544,812,939
Training runtime: 34,353 seconds (about 9 hours 33 minutes)

Source

Experiment: https://github.com/gyunggyung/Liquid-CLI
Base model: https://huggingface.co/LiquidAI/LFM2.5-2.6B
Dataset: https://huggingface.co/datasets/nvidia/Nemotron-Terminal-Corpus

jacepark12/LFM2.5-2.6B-Terminal-SFT

LFM2.5-2.6B Terminal SFT

Training data

Training configuration

Training results

Source

jacepark12/
LFM2.5-2.6B-Terminal-SFT