Quantization made by Richard Erkhov.
Chikuma is a 10.7B parameter model and is a merge of the following models using
LazyMergekit:
The name "Chikuma" is inspired by the
Chikuma River, the longest in Japan, known for its continuous flow and meandering path.
This metaphorically represents the model's depth, fluidity, and adaptability in processing and understanding language.
It also perfectly fits the approach taken here - Depth Upscaling, inspired by SOLAR 10.7B.
More details can be found
here1<|im_start|>GPT4 Correct system
2You are Chikuma, a constantly learning AI assistant who strives to be
3insightful, engaging, and helpful. You possess vast knowledge and creativity,
4but also a humble curiosity about the world and the people you interact
5with. If you don't know the answer to a question, please don't share false information.
6Always use <|end_of_turn|> when you want to end the answer.<|im_end|>
7<|im_start|>GPT4 Correct User:
8{{Input}}
9<|im_end|>GPT4 Correct Assistant:
ChatML also works, but make sure to add the sentence "Always use <|end_of_turn|> when you want to end the answer" as the default eos token is <|end_of_turn|>.
1slices:
2 - sources:
3 - model: sethuiyer/SynthIQ-7b
4 layer_range: [0, 24]
5 - sources:
6 - model: openchat/openchat-3.5-0106
7 layer_range: [8, 32]
8merge_method: passthrough
9dtype: bfloat16
Chikuma is on Ollama. You can use it by running the command
ollama run stuehieyr/chikuma in your
terminal. If you have limited computing resources, check out this
video to learn how to run it on
a Google Colab backend.
1sys_message = '''
2You are Chikuma, a constantly learning AI assistant who strives to be
3insightful, engaging, and helpful. You possess vast knowledge and creativity,
4but also a humble curiosity about the world and the people you interact
5with. If you don't know the answer to a question, please don't share false information.
6Always use <|end_of_turn|> when you want to end the answer.
7'''
8
9question = '''
10Tell me what is a large language model in under 250 words.
11'''
12
13messages = [{"role":"system", "content": sys_message}, {"role": "user", "content": question}]
14prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
15outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=4.0, top_k=50, top_p=0.01)
16print(outputs[0]["generated_text"])
Detailed results can be found
here
