QWQ-500M is a fine-tuned variant of Qwen2.5-0.5B, optimized for text generation tasks, particularly conversational reasoning and complex problem-solving. This model contains 494 million parameters and uses FP16 tensor type for efficient inference. It leverages the robust architecture of Qwen2.5 and has undergone further enhancements to excel in generating high-quality text, structured outputs, and multilingual support.

Key Features

Base Model: Derived from Qwen/Qwen2.5-0.5B.
Finetuned on Instruction Data: Built upon Qwen2.5-0.5B-Instruct with specialized datasets for better instruction-following.
Specialization:
- Advanced conversational reasoning.
- Long-form content generation.
- Support for generating structured data (JSON, tables).
- Multilingual capabilities (over 29 languages).
Optimized for Long Context: Supports input contexts up to 128K tokens with generation capability up to 8K tokens.

Datasets Used

The model was fine-tuned on high-quality datasets explicitly curated for Chain of Thought (CoT) reasoning and long-context tasks. Notable datasets include:

amphora/QwQ-LongCoT-130K: 133k samples focused on complex CoT reasoning.
qingy2024/QwQ-LongCoT-Verified-130K: 467k verified samples emphasizing detailed step-by-step reasoning.
gghfez/QwQ-LongCoT-130K-cleaned: 125k cleaned samples for high-accuracy reasoning tasks.

Running the Model

To run the model using the Transformers library:

python

# Install necessary libraries
# pip install transformers torch

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("prithivMLmods/QWQ-500M")
model = AutoModelForCausalLM.from_pretrained(
    "prithivMLmods/QWQ-500M",
    torch_dtype=torch.float16,
    device_map="auto",
)

input_text = "Explain the concept of reinforcement learning."
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**input_ids, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations

Bias and Fairness: Despite fine-tuning efforts, biases from the training data may persist. Users should critically assess model outputs.
Contextual Understanding: While optimized for long contexts, the model may still occasionally misinterpret highly ambiguous prompts.
Real-Time Knowledge: The model's knowledge is limited to its training data and does not include real-time or post-training updates.
Safety Considerations: Safety alignment has been performed, but users should monitor outputs to avoid inappropriate content.
Resource Requirements: Running the model efficiently requires a GPU with sufficient memory.

Intended Use Cases

Conversational AI: Enhanced dialogue capabilities with nuanced understanding and context retention.
Educational Assistance: Generating detailed explanations, tutorials, and step-by-step guides.
Content Creation: Assisting in writing blogs, articles, and creative content.
Multilingual Applications: Supporting content generation and translation across multiple languages.
Data Generation: Producing structured outputs such as JSON and tables for various applications.