Recent advancements, such as o1, have demonstrated that inference-time scaling is an effective approach to enhance LLMs’ reasoning capabilities via Chain-of-Thought (CoT). Encouragingly, several open-source o1-like LLMs with long-form reasoning exhibit strong performances, such as QwQ-32B and Deepseek-R1. To leverage these capabilities, we conduct an initial experiment utilizing strong o1-like LLMs for model distillation. Specifically, we use Deepseek-R1-Distilled-Qwen-32B as our teacher model. Our data augmentation strategy follows a two-step approach:

(1) We prompt Qwen2.5-72B-Instruct to generate diverse synthetic questions based on existing datasets. (2) We then use Deepseek-R1-Distilled-Qwen-32B to generate responses for both the collected and synthetic instructions, resulting in an enriched dataset of 70K samples with extensive CoT reasoning steps.

After that, we use the 70K dataset Diabetica-o1-SFT to fine-tune Qwen2.5-7B-Instruct and get Diabetica-o1-7B.

Please refer to our Paper for more details.

Model Inference

bash

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

device = "cuda" # the device to load the model onto
model_path = 'WaltonFuture/Diabetica-o1'

model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_path)

def model_output(content):
    messages = [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": content}
    ]
    text = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True
    )
    model_inputs = tokenizer([text], return_tensors="pt").to(device)
    generated_ids = model.generate(
        model_inputs.input_ids,
        max_new_tokens=16384,
        do_sample=True,
    )
    generated_ids = [
        output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
    ]
    response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
    return response

prompt = "Hello! Please tell me something about diabetes."

response = model_output(prompt)
print(response)

Citation

@article{wei2024adapted,
  title={An adapted large language model facilitates multiple medical tasks in diabetes care},
  author={Wei, Lai and Ying, Zhen and He, Muyang and Chen, Yutong and Yang, Qian and Hong, Yanzhe and Lu, Jiaping and Li, Xiaoying and Huang, Weiran and Chen, Ying},
  journal={arXiv preprint arXiv:2409.13191},
  year={2024}
}