A LoRA adapter fine-tuned on Meta-Llama-3-8B-Instruct using GRPO (Group Relative Policy Optimization) to generate correct Linux shell commands from natural language task descriptions.

Model Details

Model Description

Developed by: Jose Alvarez, Carson Chiem, Prisha Bhattacharyya, Vishal Tyagi
Model type: Causal Language Model (LoRA adapter)
Language(s) (NLP): English
License: Meta Llama 3 Community License
Finetuned from model: unsloth/llama-3-8b-Instruct

Model Sources

Repository: https://github.com/Alvarez-Jose/unsloth-grpo-project

Uses

Direct Use

Given a natural language description of a CLI task, the model outputs the correct shell command with no explanation, no markdown, and no backticks.

Example:

Input: "Count the number of lines in /tmp/data/log.txt"
Output: wc -l /tmp/data/log.txt

Out-of-Scope Use

Not intended for general conversation
Not suitable for tasks outside Linux CLI command generation
Should not be used for destructive or malicious shell commands

Bias, Risks, and Limitations

Model may generate incorrect or harmful shell commands — always review before executing
Trained on a limited set of ~60 task types, may not generalize to all CLI scenarios
Performance degrades on complex multi-step tasks

How to Get Started with the Model

python

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="jalva182/cli-agent-model",
    max_seq_length=512,
    load_in_4bit=True,
)

messages = [
    {"role": "system", "content": "You are a CLI expert. Given a task, output exactly the shell commands required. No explanation, no markdown, no backticks."},
    {"role": "user", "content": "Count the number of lines in /tmp/data/log.txt"},
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")
outputs = model.generate(input_ids=inputs, max_new_tokens=64)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

60 validated CLI tasks covering file operations, text processing (grep, awk, sed), sorting, archives, system info, permissions, and environment variables. Each task includes setup commands, expected output, and a reward function for GRPO training.

Training Hyperparameters

Training regime: bf16 mixed precision
Method: GRPO (Group Relative Policy Optimization)
Learning rate: 3e-6 with linear scheduler
Warmup ratio: 0.1
Batch size: 2 (per device)
Gradient accumulation steps: 2
Total steps: 10000
LoRA rank: 32, alpha: 64
KL coefficient: 0.05
Number of generations: 4
Max sequence length: 512

Speeds, Sizes, Times

Training time: ~3h 13min
Checkpoint size: ~524MB (LoRA adapter only)
Final train loss: 0.0141
Final reward: 8.0/8.0 on easy tasks, ~6.0 average

Evaluation

Metrics

Reward function scoring 0-8 per task:

+5 for correct output match
+3 for command success with partial match
-2 for command failure or wrong output

Results

Best reward: 8.0
Average reward (final steps): ~6.0
Train loss: 0.0141

Environmental Impact

Hardware Type: H100 SXM 80GB
Hours used: ~3.5 hours
Cloud Provider: Vast.ai

Technical Specifications

Model Architecture

Base: Meta-Llama-3-8B-Instruct
Adapter: LoRA (rank=32, alpha=64, dropout=0.05)
Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Software

unsloth 2026.3.3
trl 0.24.0
transformers 4.56.1
torch 2.6.0+cu124
PEFT 0.18.1

Model Card Authors

Jose Alvarez

Model Card Contact

https://github.com/Alvarez-Jose/unsloth-grpo-project

Framework versions

PEFT 0.18.1