This is a LoRA fine-tuned version of Qwen/Qwen2.5-1.5B-Instruct trained for application-layer tool calling capabilities in the Dongshan knowledge domain.

Architecture: Application-Layer Tool Calling

This model uses a unique application-layer tool calling architecture where:

The application layer wraps the model and handles tool execution internally
The model outputs tool-CALL-COMPLETED results (not tool call requests)
Tools include: calculator, clock, search, lookup, and composite reasoning

Supported Tool Tags

Tool	Tag	Description
Calculator	`<calculator>expr</calculator>`	Mathematical computation
Clock	`<clock/>`	Current date/time
Search	`<search_1/>` ... `</search_1>`	Web search
Lookup	`<lookup/>`	Knowledge base lookup
Analyze	`<analyze/>`	Problem analysis
Evaluate	`<evaluate_1/>`	Quality evaluation
Synthesize	`<synthesize/>`	Information synthesis
Focus	`<focus/>`	Focus attention
Think	`<think/>`	Step-by-step reasoning
Verify	`<verify/>`	Result verification
Reflect	`<reflect/>`	Self-reflection
Correct	`<correct/>`	Error correction

Training Details

Base Model: Qwen/Qwen2.5-1.5B-Instruct
Method: LoRA (PEFT)
LoRA Config: r=16, alpha=32, dropout=0.05
Target Modules: q_proj, v_proj, k_proj, o_proj, gate_proj, up_proj, down_proj
Training Data: 5000 samples (mojoz/dongshan-tool-calling-5k)
Epochs: 3
Learning Rate: 2e-4
Batch Size: 2 (gradient_accumulation=4)
Quantization: INT4 (during training)

Tool Distribution in Training Data

Tool Type	Percentage
Calculator	33.1%
Search	21.5%
Lookup	20.1%
Clock	14.4%
No-tool	18.4%
Multi-tool	7.5%

Usage

With Application Layer

python

from tools.app_layer import DongshanAppLayer

app = DongshanAppLayer(model_name="mojoz/qwen2.5-1.5b-instruct-lora-finetuned")
result = app.query("今天星期几？现在几点了？")
print(result)

Direct Model Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load base model and LoRA adapter
base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(base_model, "mojoz/qwen2.5-1.5b-instruct-lora-finetuned")
tokenizer = AutoTokenizer.from_pretrained("mojoz/qwen2.5-1.5b-instruct-lora-finetuned")

# Chat with tool calling
messages = [
    {"role": "system", "content": "你是东山论知识助手..."},
    {"role": "user", "content": "帮我计算 123 × 456"}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0]))

Training Space: mojoz/train-qwen-1.5b-tool
Dataset: mojoz/dongshan-tool-calling-5k
7B Version: mojoz/qwen2.5-7b-instruct-lora-finetuned
GitHub: ctz168/ds_qwen_colab

mojoz/qwen2.5-1.5b-instruct-lora-finetuned

Qwen2.5-1.5B-Instruct LoRA Fine-tuned for Tool Calling

Model Description