📢 Release Note Build Environment:

Training Method: Supervised Fine-Tuning (SFT)

Base Model: Qwen3.5-0.8B

Training Libraries: Hugging Face Transformers + TRL

💡 Model Introduction

Qwen3.5-0.8B-Claude-4.6-Opus-Reasoning-Distilled is a reasoning-focused model fine-tuned from Qwen3.5-0.8B using structured reasoning traces derived from Claude-4.6 Opus style reasoning datasets.

The model is trained to produce structured chain-of-thought reasoning, enabling it to:

Break complex problems into logical steps
Produce internal reasoning inside <think> blocks
Deliver accurate final answers after reasoning

The training dataset contains curated reasoning examples designed to teach the model step-by-step analytical thinking.

🚀 How to Run the Model

You can run the model using the transformers library.

1️⃣ Install Dependencies

pip install transformers torch accelerate

2️⃣ Run the Model

python

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "Ishant06/Qwen3.5-0.8B-Claude-4.6-Opus-Reasoning-Distilled"

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)

# Load model
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)

prompt = "Explain why the sky is blue."

messages = [
    {"role": "system", "content": "You are a helpful AI assistant."},
    {"role": "user", "content": prompt}
]

# Apply chat template (important for Qwen models)
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.7,
        top_p=0.9
    )

response = tokenizer.decode(outputs[0], skip_special_tokens=True)

print(response)

Tested with:

transformers >= 4.40
torch >= 2.0

🧠 Example Reasoning Pattern

The model follows a structured reasoning scaffold such as:

text

Let me analyze this request carefully:

Understand the problem.
Break it into smaller steps.
Analyze each step logically.
Combine the reasoning.
Produce the final answer.

🗺️ Training Pipeline Overview

text

Base Model (Qwen3.5-0.8B)
 │
 ▼
Supervised Fine-Tuning (SFT)
 │
 ▼
Final Model (Claude-4.6-Opus-Reasoning-Distilled)

📋 Training Details

🔹 Supervised Fine-Tuning (SFT)

Framework: Hugging Face Transformers + TRL

Training Strategy: Instruction → Response SFT

Goal: Teach the model structured reasoning and step-by-step problem solving.

Format Used During Training:

text

<think>
internal reasoning
</think>
final answer

📚 Dataset Used

Dataset	Description
crownelius/Opus-4.6-Reasoning-3300x	Claude-4.6 Opus style reasoning dataset containing structured chain-of-thought examples

🌟 Capabilities

The model performs well in tasks requiring reasoning such as:

Logical problem solving
Mathematical reasoning
Coding explanations
Step-by-step analysis
Instruction following

⚠️ Limitations

The model may still hallucinate factual information.
Performance is limited by the relatively small 0.8B parameter size.
Best suited for experimentation, lightweight reasoning tasks, and research.

🙏 Acknowledgements

Qwen Team for the base model.
The open-source community for providing reasoning datasets.

📖 Citation

bibtex

@misc{ishant_qwen35_opus_reasoning,
  title        = {Qwen3.5-0.8B-Claude-4.6-Opus-Reasoning-Distilled},
  author       = {Ishant Dere},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Ishant06/Qwen3.5-0.8B-Claude-4.6-Opus-Reasoning-Distilled}}
}