Important: This model does not have an embedded chat template. To ensure high-quality responses, you must manually format your prompts to match the structure used during training.

Prompt Format

The model expects the following turn-based structure:

text

User: [Your message here]
Assistant:

For multi-turn conversations, use:

text

User: [User message 1]
Assistant: [Model response 1]
User: [User message 2]
Assistant:

Python Example

python

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "FlameF0X/TinyChat-200m-2x16"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = "User: Explain how MoE works.\nAssistant:"

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

This model was fine-tuned on the following datasets using LoRA:

databricks-dolly-15k: Instruction/context/response pairs.
no_robots: SFT dataset.
ultrachat_200k: SFT dataset.

Preprocessing: All datasets were converted to a standard messages format and rendered into the simple User: ... / Assistant: ... text style shown above.

Training Specs:

Method: LoRA (Low-Rank Adaptation)
Hardware: Trained on CPU
Framework: Hugging Face transformers + peft
Tokenizer: Used the base model's fast tokenizer.