Important ⚠️
This repository contains LoRA adapter weights only, not the full base model.
You must load it together with the base model:
Qwen/Qwen3.5-0.8B
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model = "Qwen/Qwen3.5-0.8B"
adapter = "your-username/your-repo-name"
tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(base_model)
model = PeftModel.from_pretrained(model, adapter)
prompt = "### Instruction:\nWhat is the capital of India?\n\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
**inputs,
max_new_tokens=100,
temperature=0.7,
top_p=0.9
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Training Details
- Base Model: Qwen/Qwen3.5-0.8B
- Method: LoRA (PEFT)
- Dataset: databricks/databricks-dolly-15k
- Task Format: Instruction → Response
- Max Sequence Length: 1024
- Batch Size: 16
- Gradient Accumulation: 2
- Effective Batch Size: 32
- Epochs: 3
- Learning Rate: 2e-4
Capabilities
- Follows structured instruction prompts
- Answers general knowledge questions
- Generates concise and relevant responses
- Works well with the "Instruction → Response" format
Limitations
- Small model (0.8B) → limited reasoning ability
- Can repeat patterns if prompt format is incorrect
- Sensitive to prompt structure
- Not trained for complex multi-step reasoning
- Requires base model to function
Example
Instruction:
What is the capital of India?
Response:
The capital of India is New Delhi.
License
This model is released under the CC BY-NC 4.0 license.
