This is Noromaid-10.7B-0.4-DPO, a depth-upscaled version of
NeverSleep/Noromaid-10.7B-0.4-DPO .
This model is intended to be used as a basis for further fine-tuning, or as a drop-in upgrade from the original 7 billion parameter model.
Paper detailing how Depth-Up Scaling works:
SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling
This is a merge of pre-trained language models created using
mergekit .
<|im_start|>system
{sysprompt}<|im_end|>
<|im_start|>user
{input}<|im_end|>
<|im_start|>assistant
{output}<|im_end|>This model was merged using the passthrough merge method.
1 dtype : bfloat16
2 merge_method : passthrough
3 slices :
4 - sources :
5 - layer_range : [ 0 , 24 ]
6 model : /Users/jsarnecki/opt/Workspace/NeverSleep - Noromaid - 0.4 - DPO/NeverSleep - Noromaid - 7B - 0.4 - DPO
7 - sources :
8 - layer_range : [ 8 , 32 ]
9 model : /Users/jsarnecki/opt/Workspace/NeverSleep - Noromaid - 0.4 - DPO/NeverSleep - Noromaid - 7B - 0.4 - DPO
10
This repo contains fp16 files of Noromaid-7b-v0.4-DPO.
Note: We have permission of all users to upload their ratings, we DONT screenshot random reviews without asking if we can put them here!
If you want your rating to be here, send us a message over on DC and we'll put up a screenshot of it here. DC name is "ikaridev" and "undi".
<|im_start|>system
{sysprompt}<|im_end|>
<|im_start|>user
{input}<|im_end|>
<|im_start|>assistant
{output}<|im_end|>This is a full finetune.
Undi: If you want to support me, you can
here .
