Views
No views yet
| Feature | What it means | Benefit |
|---|---|---|
| Auto Think | Diverse pre‑think data teaches the model to predict task difficulty | Better choice of when to think |
| Step‑SRPO | Token‑wise GRPO variant with process‑level rewards | More stable RL, higher “think” & “no‑think” accuracy |
| Agentic Data | Automated cot cold start data generation | Stronger inference models before reinforcement learning |
| KD + MTP | 1 teacher → many‑token prediction distillation | <1⁄30 pre‑train cost |
1from transformers import AutoTokenizer, AutoModelForCausalLM
2
3model_name = "Kwaipilot/KwaiCoder-AutoThink-preview"
4
5# load the tokenizer and the model
6tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
7model = AutoModelForCausalLM.from_pretrained(
8 model_name,
9 torch_dtype="auto",
10 device_map="auto"
11)
12
13# prepare the model input
14prompt = "Give me a short introduction to large language model."
15messages = [
16 {"role": "user", "content": prompt}
17]
18text = tokenizer.apply_chat_template(
19 messages,
20 tokenize=False,
21 add_generation_prompt=True
22)
23model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
24
25# conduct text completion
26generated_ids = model.generate(
27 **model_inputs,
28 max_new_tokens=32768,
29 temperature=0.6,
30 top_p=0.9,
31)
32output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()
33content = tokenizer.decode(output_ids, skip_special_tokens=True).strip("\n")
34print("prompt:\n", prompt)
35print("content:\n", content)
36"""
37prompt:
38Give me a short introduction to large language model.
39content:
40<judge>
41This is a definitional query seeking a basic explanation, which can be answered with straightforward factual recall or a concise summary. Requires think-off mode.
42</judge>
43
44<think off>
45Large Language Models (LLMs) are advanced artificial intelligence systems designed to understand and generate human-like text. They are trained on vast amounts of data to learn grammar, facts, reasoning, and context. Key features include:
46
47- **Scale**: Billions (or even trillions) of parameters, enabling complex pattern recognition.
48- **Versatility**: Can perform tasks like answering questions, writing code, summarizing text, and more.
49- **Adaptability**: Fine-tuned for specific uses (e.g., customer support, creative writing).
50
51Examples include OpenAI's GPT, Google's Gemini, and Meta's Llama. While powerful, LLMs may occasionally hallucinate or rely on outdated information. They’re transforming industries by automating text-based tasks and enhancing human productivity.
52
53Would you like a deeper dive into any aspect?
54"""