概要

このモデルはnitky/RoguePlanet-DeepSeek-R1-Qwen-32Bにインスパイアを受け、作成されたモデルです。タグが出力されることは確認しています。日本語モデルとしてもよい性能を出せることも確認しています。

How To Use

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "DataPilot/SKYCAVE-R1-32B-v0.1"

tokenizer_name = ""

if tokenizer_name == "":
    tokenizer_name = model_name

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(tokenizer_name)

prompt = "メタデータを解析し、自己進化をするAIであるnurture intelligenceが実現した未来の日常生活の姿を教えてください。"
messages = [
    {"role": "system", "content": "あなたは優秀な日本語アシスタントであり長考モデルです。問題解決をするための思考をした上で回答を行ってください。"},
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=4096
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

print(response)

謝辞

このモデルの作成者皆様と、計算資源を貸していただいたVOLTMINDに感謝します。モデル作成にアドバイスをしていただいたnitkyさんにも感謝申し上げます。

mergekit config

yaml

merge_method: slerp
base_model: karakuri-ai/karakuri-lm-32b-thinking-2501-exp
models:
  - model: karakuri-ai/karakuri-lm-32b-thinking-2501-exp
  - model: Saxo/Linkbricks-Horizon-AI-Japanese-Base-32B
parameters:
  t: 0.35
dtype: bfloat16
name: SKYCAVE_element_QwQ_jp

---

merge_method: slerp
base_model: cyberagent/DeepSeek-R1-Distill-Qwen-32B-Japanese
models:
  - model: cyberagent/DeepSeek-R1-Distill-Qwen-32B-Japanese
  - model: SKYCAVE_element_QwQ_jp
parameters:
  t: 0.4
dtype: bfloat16
name: SKYCAVE_element_QR_jp

---

merge_method: slerp
base_model: cyberagent/DeepSeek-R1-Distill-Qwen-32B-Japanese
models:
  - model: cyberagent/DeepSeek-R1-Distill-Qwen-32B-Japanese
  - model: FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview
parameters:
  t: 0.5
dtype: bfloat16
name: SKYCAVE_element_R1_jp_01

---

merge_method: slerp
base_model: cyberagent/DeepSeek-R1-Distill-Qwen-32B-Japanese
models:
  - model: cyberagent/DeepSeek-R1-Distill-Qwen-32B-Japanese
  - model: TeamDelta/ABEJA-Qwen2.5-32B-base-jp-v0.1
parameters:
  t: 0.5
dtype: bfloat16
name: SKYCAVE_element_R1_jp_02

---

merge_method: slerp
base_model: cyberagent/DeepSeek-R1-Distill-Qwen-32B-Japanese
models:
  - model: cyberagent/DeepSeek-R1-Distill-Qwen-32B-Japanese
  - model: deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
parameters:
  t: 0.6
dtype: bfloat16
name: SKYCAVE_element_R1_jp_03

---

merge_method: slerp
base_model: karakuri-ai/karakuri-lm-32b-thinking-2501-exp
models:
  - model: karakuri-ai/karakuri-lm-32b-thinking-2501-exp
  - model: NovaSky-AI/Sky-T1-32B-Flash
parameters:
  t: 0.4
dtype: bfloat16
name: SKYCAVE_element_Sky_jp

---

merge_method: model_stock
base_model: cyberagent/DeepSeek-R1-Distill-Qwen-32B-Japanese
models:
  - model: SKYCAVE_element_QR_jp
  - model: SKYCAVE_element_R1_jp_01
  - model: SKYCAVE_element_R1_jp_02
  - model: SKYCAVE_element_R1_jp_03
  - model: SKYCAVE_element_Sky_jp
dtype: bfloat16
name: SKYCAVE-R1-32B-v0.1