This model is a LoRA fine-tuned version of DeepSeek-R1-Distill-Qwen-7B specifically trained for Chinese multiple choice question answering with reasoning capabilities.

Model Details

Model Description

This model has been fine-tuned using LoRA (Low-Rank Adaptation) on Chinese multiple choice questions with detailed reasoning explanations. The model is designed to understand Chinese questions, analyze multiple options (A, B, C, D), and provide reasoned answers with explanations.

Developed by: jasmine313112031 (YANG YING HSUAN)
Model type: Causal Language Model with LoRA adapter
Language(s) (NLP): Chinese
License: Apache 2.0
Finetuned from model: unsloth/DeepSeek-R1-Distill-Qwen-7B

Model Sources

Repository: GitHub Repository
Training Logs: W&B Dashboard

Uses

Direct Use

This model is designed for Chinese multiple choice question answering tasks. It can:

Answer multiple choice questions in Chinese
Provide reasoning explanations for answers
Handle questions across various domains

Downstream Use

The model can be integrated into:

Educational assessment systems
Chinese language learning platforms
Automated grading systems
Reasoning evaluation benchmarks

Out-of-Scope Use

This model should not be used for:

Tasks requiring factual accuracy beyond training data cutoff
High-stakes decision making without human oversight
Generating harmful or biased content

Bias, Risks, and Limitations

Known Issues:

Strong bias towards option A (97.6% of predictions in test set)
May reflect training data distribution imbalance
Limited to multiple choice format with 4 options

Recommendations

Users should be aware of the model's bias towards option A and consider:

Randomizing option order during inference
Using ensemble methods to reduce bias
Validating outputs against ground truth when available

How to Get Started with the Model

python

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load tokenizer and base model
tokenizer = AutoTokenizer.from_pretrained("unsloth/DeepSeek-R1-Distill-Qwen-7B")
base_model = AutoModelForCausalLM.from_pretrained(
    "unsloth/DeepSeek-R1-Distill-Qwen-7B",
    torch_dtype=torch.float16,
    device_map="auto"
)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "jasmine313112031/deepseek_wh2")

# Example usage
prompt = """問題：六四事件發生在哪一年？
選項：
A. 1989年
B. 1990年
C. 1991年
D. 1992年

請從上面的選項中選出最正確的答案。
你的答案必須使用「正確答案：X」的格式，X是A、B、C或D中的一個。

思考過程："""

inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=100, do_sample=False)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)