You can deploy this model and start earning money today!

RichardErkhov/
0-hero_-_Matter-0.1-Slim-7B-C-gguf

Name: 0-hero_-_Matter-0.1-Slim-7B-C-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Matter-0.1-Slim-7B-C - GGUF

Model creator: https://huggingface.co/0-hero/
Original model: https://huggingface.co/0-hero/Matter-0.1-Slim-7B-C/

Name	Quant method	Size
Matter-0.1-Slim-7B-C.Q2_K.gguf	Q2_K	2.53GB
Matter-0.1-Slim-7B-C.IQ3_XS.gguf	IQ3_XS	2.81GB
Matter-0.1-Slim-7B-C.IQ3_S.gguf	IQ3_S	2.96GB
Matter-0.1-Slim-7B-C.Q3_K_S.gguf	Q3_K_S	2.95GB
Matter-0.1-Slim-7B-C.IQ3_M.gguf	IQ3_M	3.06GB
Matter-0.1-Slim-7B-C.Q3_K.gguf	Q3_K	3.28GB
Matter-0.1-Slim-7B-C.Q3_K_M.gguf	Q3_K_M	3.28GB
Matter-0.1-Slim-7B-C.Q3_K_L.gguf	Q3_K_L	3.56GB
Matter-0.1-Slim-7B-C.IQ4_XS.gguf	IQ4_XS	3.67GB
Matter-0.1-Slim-7B-C.Q4_0.gguf	Q4_0	3.83GB
Matter-0.1-Slim-7B-C.IQ4_NL.gguf	IQ4_NL	3.87GB
Matter-0.1-Slim-7B-C.Q4_K_S.gguf	Q4_K_S	3.86GB
Matter-0.1-Slim-7B-C.Q4_K.gguf	Q4_K	4.07GB
Matter-0.1-Slim-7B-C.Q4_K_M.gguf	Q4_K_M	4.07GB
Matter-0.1-Slim-7B-C.Q4_1.gguf	Q4_1	4.24GB
Matter-0.1-Slim-7B-C.Q5_0.gguf	Q5_0	4.65GB
Matter-0.1-Slim-7B-C.Q5_K_S.gguf	Q5_K_S	4.65GB
Matter-0.1-Slim-7B-C.Q5_K.gguf	Q5_K	4.78GB
Matter-0.1-Slim-7B-C.Q5_K_M.gguf	Q5_K_M	4.78GB
Matter-0.1-Slim-7B-C.Q5_1.gguf	Q5_1	5.07GB
Matter-0.1-Slim-7B-C.Q6_K.gguf	Q6_K	5.53GB
Matter-0.1-Slim-7B-C.Q8_0.gguf	Q8_0	7.17GB

Original model description:

license: apache-2.0

Matter 7B (Mistral 7B Finetune)

Matter 7B is contunied full-finetuned on the slim-C version of the Matter dataset, which is curated from over 35 datsets analyzing >6B tokens

Training

It took ~17 hours to train 3 epochs on 4x A100s (80GB) using Axolotl for a full-finetune

Prompt format: This model uses ChatML prompt format.

<|im_start|>system
You are a helpful AI assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

Function Calling

Model also supports function calling. Additional tokens for function calling

Model function call tokens

<|begin_func|> - Function call start token
<|end_func|> - Function call end token

Function call response tokens

<|begin_func_response|> - Function response start token
<|end_func_response|> - Function response end token

Example

<|im_start|>system
  You are a helpful assistant with access to the following functions. Use them if required -
  { "name": "get_news_headlines",
    "description": "Get the latest news headlines",
    "parameters":
      { "type": "object",
        "properties":
        { "country":
          { "type": "string",
            "description": "The country for which to fetch news"
          }
        },
      "required": [ "country" ]
    }
  }
<|im_end|>
<|im_start|>user
   Can you tell me the latest news headlines for the United States?<|im_end|>
<|im_start|>assistant
  <|begin_func|>{"name": "get_news_headlines", "arguments": '{"country": "United States"}'}<|end_func|><|im_end|>
<|im_start|>user
  <|begin_func_response|>{
    "headlines":
      [
        "Biden announces new vaccine mandates",
        "Hurricane Ida devastates Louisiana",
        "Apple unveils new iPhone",
        "NASA's Perseverance rover collects first Mars rock sample"
      ]
  }<|end_func_response|>
<|im_end|>
<|im_start|>assistant
  Here are the latest news headlines for the United States:
  1. Biden announces new vaccine mandates
  2. Hurricane Ida devastates Louisiana
  3. Apple unveils new iPhone
  4. NASA's Perseverance rover collects first Mars rock sample
<|im_end|>

RichardErkhov/0-hero_-_Matter-0.1-Slim-7B-C-gguf