You can deploy this model and start earning money today!

cagataydev/
qwen2.5-omni-3b-cagatay

Name: qwen2.5-omni-3b-cagatay
Availability: InStock

cagataydev/omni-commands

Qwen/Qwen2.5-3B-Instruct

adapter

apache-2.0

endpoints_compatible

Views

No views yet

🧠 Qwen 2.5 Omni 3B — Cagatay LoRA

A LoRA fine-tune of Qwen/Qwen2.5-3B-Instruct for robotics voice command understanding.

🎯 What is this?

This is a LoRA adapter fine-tuned to understand and generate responses for robotics voice commands — pick & place, navigation, manipulation, multi-step tasks, spatial reasoning, and safety commands.

Trained on 106K examples from the cagataydev/omni-commands dataset covering 10 categories of human→robot interaction.

📊 Training Details

Parameter	Value
Base Model	`Qwen/Qwen2.5-3B-Instruct`
Method	LoRA (PEFT)
Rank (r)	64
Alpha	128
Dropout	0.05
Target Modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Trainable Params	119.7M / 1.82B total (6.6%)
Training Examples	106,356
Epochs	2
Final Loss	0.846
Hardware	NVIDIA L4
Framework	PEFT v0.18.1

🚀 Quick Start

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model + LoRA adapter
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-3B-Instruct",
    torch_dtype="auto",
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, "cagataydev/qwen2.5-omni-3b-cagatay")
tokenizer = AutoTokenizer.from_pretrained("cagataydev/qwen2.5-omni-3b-cagatay")

# Robotics command
messages = [{"role": "user", "content": "Pick up the red cube on the table and bring it to me"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

🤖 Use Cases

Robot instruction following — Parse natural language into actionable robot commands
Voice-controlled robots — Understand spoken commands (pair with ASR)
Embodied AI assistants — Multi-step task planning from human speech
Neon VLA backbone — Powers the Neon VLA system

🗂️ Command Categories

Category	Description	Example
🫳 Pick & Place	Object manipulation	"Grab the bottle from the shelf"
🧭 Navigation	Movement commands	"Go to the kitchen"
🔧 Manipulation	Fine motor tasks	"Pour water into the cup"
📋 Multi-step	Chained actions	"Open drawer, take spoon, close it"
👁️ Observation	Visual queries	"What do you see on the table?"
📐 Spatial	Precise movements	"Move arm left 5 centimeters"
🏠 Household	Domestic tasks	"Clean the table"
⚠️ Safety	Emergency commands	"Stop immediately!"
💬 Conversational	Feedback	"Good job, try again"
🎯 Context-rich	Requires vision	"The one I pointed at"

📦 Related Resources

Resource	Description
vlm-voice-audio	10K audio recordings of robot commands
vlm-voice-commands	50K text robot commands
neon-vla	Live demo — Vision-Language-Action model
qwen3.5-4B-cagatay	Qwen 3.5 4B LoRA variant
qwen3.5-35B-A3B-cagatay	Qwen 3.5 35B MoE LoRA variant

🏗️ Training Pipeline

omni-commands dataset (106K examples)
    ↓
Qwen2.5-3B-Instruct (base)
    ↓  LoRA r=64, α=128
Fine-tuned adapter (120M params)
    ↓
Merged into Neon VLA pipeline

Built with DevDuck 🦆 | Part of the Neon VLA ecosystem