Views
No views yet
lm_head as an additional target, and twice with embed_tokens.grad_norm value would behave strangely on layer 3 and 27. The last layer, before the expansion used to be 27, while 3 is a mystery. I decided to use mergekit to copy layer 3 and insert it beside the original, and copy layer 27 and insert it at the end or top (the new 33, all 0 indexed), depending on your perspective.You agree not to use the Model or Derivatives of the Model:
- In any way that violates any applicable national or international law or regulation or infringes upon the lawful rights and interests of any third party;
- For military use in any way;
- For the purpose of exploiting, harming or attempting to exploit or harm minors in any way;
- To generate or disseminate verifiably false information and/or content with the purpose of harming others;
- To generate or disseminate inappropriate content subject to applicable regulatory requirements;
- To generate or disseminate personal identifiable information without due authorization or for unreasonable use;
- To defame, disparage or otherwise harass others;
- For fully automated decision making that adversely impacts an individual’s legal rights or otherwise creates or modifies a binding, enforceable obligation;
- For any use intended to or which has the effect of discriminating against or harming individuals or groups based on online or offline social behavior or known or predicted personal or personality characteristics;
- To exploit any of the vulnerabilities of a specific group of persons based on their age, social, physical or mental characteristics, in order to materially distort the behavior of a person pertaining to that group in a manner that causes or is likely to cause that person or another person physical or psychological harm;
- For any use intended to or which has the effect of discriminating against individuals or groups based on legally protected characteristics or categories.LLama-Factory/src/llmtuner/data/template.py1_register_template(
2 name="ninja_chatml",
3 format_user=StringFormatter(slots=["<|im_start|>user\n{{content}}\n<|im_end|>\n"]), # Works
4 format_assistant=StringFormatter(slots=["<|im_start|>assistant\n{{content}}\n<|im_end|>", {"eos_token"}]), # Works
5 format_system=StringFormatter(slots=["<|im_start|>system\n{{content}}\n<|im_end|>\n"]), # NinjaMouse does not like BOS!
6 format_function=FunctionFormatter(slots=["<|im_start|>assistant\n<tool_call>\n{\"name\":\"{{name}}\", \"arguments\":{{arguments}}}\n</tool_call>\n<|im_end|>", {"eos_token"}]), # Works
7 format_observation=StringFormatter(slots=["<|im_start|>tool\n<tool_response>\n{{content}}\n</tool_response>\n<|im_end|>\n"]), # Works
8 format_separator=EmptyFormatter(slots=["\n"]), # It makes sense to keep this a new line instead of </s> and apply the eos token directly
9 format_tools=ToolFormatter(tool_format="open_chatml"),
10)formatter.py in the same folder.1# At the top
2HERMES_TOOL_PROMPT = (
3 "\n<tools>\n"
4 "{funtion_description}\n"
5 "</tools>\n"
6)
7
8# I only added the elif
9@dataclass
10class ToolFormatter(Formatter):
11 def __post_init__(self):
12 if self.tool_format is None:
13 raise ValueError("Tool format was not found.")
14
15
16 def apply(self, **kwargs) -> SLOTS:
17 content = kwargs.pop("content")
18 try:
19 tools = json.loads(content)
20 if not len(tools):
21 return [""]
22
23 if self.tool_format == "default":
24 return [default_tool_formatter(tools)]
25 elif self.tool_format == "open_chatml": # This right here
26 return [OPEN_CHATML_TOOL_PROMPT.format(funtion_description=json.dumps(tools, ensure_ascii=False, indent=4))] # I used 4 but OpenChatML has 2
27 else:
28 raise NotImplementedError
29 except Exception:
30 return [""]
31
32
33 def extract(self, content: str) -> Union[str, Tuple[str, str]]:
34 if self.tool_format == "default":
35 return default_tool_extractor(content)
36 else:
37 raise NotImplementedErrorMistralForCausalLM(
(model): MistralModel(
(embed_tokens): Embedding(32009, 2560, padding_idx=0)
(layers): ModuleList(
(0-33): 34 x MistralDecoderLayer(
(self_attn): MistralSdpaAttention(
(q_proj): Linear(in_features=2560, out_features=2560, bias=False)
(k_proj): Linear(in_features=2560, out_features=640, bias=False)
(v_proj): Linear(in_features=2560, out_features=640, bias=False)
(o_proj): Linear(in_features=2560, out_features=2560, bias=False)
(rotary_emb): MistralRotaryEmbedding()
)
(mlp): MistralMLP( (gate_proj): Linear(in_features=2560, out_features=6912, bias=False)
(up_proj): Linear(in_features=2560, out_features=6912, bias=False)
(down_proj): Linear(in_features=6912, out_features=2560, bias=False)
(act_fn): SiLU()
)
(input_layernorm): MistralRMSNorm()
(post_attention_layernorm): MistralRMSNorm()
)
)
(norm): MistralRMSNorm()
)
(lm_head): Linear(in_features=2560, out_features=32009, bias=False)
)