You can deploy this model and start earning money today!

RichardErkhov/
uukuguy_-_speechless-thoughts-mistral-7b-gguf

Name: uukuguy_-_speechless-thoughts-mistral-7b-gguf
Availability: InStock

endpoints_compatible

gguf

template

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

speechless-thoughts-mistral-7b - GGUF

Model creator: https://huggingface.co/uukuguy/
Original model: https://huggingface.co/uukuguy/speechless-thoughts-mistral-7b/

Name	Quant method	Size
speechless-thoughts-mistral-7b.Q2_K.gguf	Q2_K	2.53GB
speechless-thoughts-mistral-7b.IQ3_XS.gguf	IQ3_XS	2.81GB
speechless-thoughts-mistral-7b.IQ3_S.gguf	IQ3_S	2.96GB
speechless-thoughts-mistral-7b.Q3_K_S.gguf	Q3_K_S	2.95GB
speechless-thoughts-mistral-7b.IQ3_M.gguf	IQ3_M	3.06GB
speechless-thoughts-mistral-7b.Q3_K.gguf	Q3_K	3.28GB
speechless-thoughts-mistral-7b.Q3_K_M.gguf	Q3_K_M	3.28GB
speechless-thoughts-mistral-7b.Q3_K_L.gguf	Q3_K_L	3.56GB
speechless-thoughts-mistral-7b.IQ4_XS.gguf	IQ4_XS	3.67GB
speechless-thoughts-mistral-7b.Q4_0.gguf	Q4_0	3.83GB
speechless-thoughts-mistral-7b.IQ4_NL.gguf	IQ4_NL	3.87GB
speechless-thoughts-mistral-7b.Q4_K_S.gguf	Q4_K_S	3.86GB
speechless-thoughts-mistral-7b.Q4_K.gguf	Q4_K	4.07GB
speechless-thoughts-mistral-7b.Q4_K_M.gguf	Q4_K_M	4.07GB
speechless-thoughts-mistral-7b.Q4_1.gguf	Q4_1	4.24GB
speechless-thoughts-mistral-7b.Q5_0.gguf	Q5_0	4.65GB
speechless-thoughts-mistral-7b.Q5_K_S.gguf	Q5_K_S	4.65GB
speechless-thoughts-mistral-7b.Q5_K.gguf	Q5_K	4.78GB
speechless-thoughts-mistral-7b.Q5_K_M.gguf	Q5_K_M	4.78GB
speechless-thoughts-mistral-7b.Q5_1.gguf	Q5_1	5.07GB
speechless-thoughts-mistral-7b.Q6_K.gguf	Q6_K	5.53GB
speechless-thoughts-mistral-7b.Q8_0.gguf	Q8_0	7.17GB

Original model description:

language:

en library_name: transformers pipeline_tag: text-generation datasets:
jondurbin/airoboros-2.2
Open-Orca/OpenOrca
garage-bAInd/Open-Platypus
WizardLM/WizardLM_evol_instruct_V2_196k
TokenBender/python_eval_instruct_51k
codefuse-ai/Evol-Instruction-66k tags:
llama-2
code license: llama2 model-index:
name: SpeechlessCoder results:
- task: type: text-generation dataset: type: openai_humaneval name: HumanEval metrics:
  - name: pass@1 type: pass@1 value: verified: false

speechless-thoughts-mistral-7b

code

speechless-thoughts-mistral-7b is fine-tuned as a baseline of the speechless-sparsetral-16x7b-MoE.

The specific datasets (speechless-thoughts-252k) are as follows:

jondurbin/airoboros-2.2: Filter categories related to coding, reasoning and planning. 23,462 samples.
Open-Orca/OpenOrca: Filter the 'cot' category in 1M GPT4 dataset. 74,440 samples.
garage-bAInd/Open-Platypus: 100%, 24,926 samples.
WizardLM/WizardLM_evol_instruct_V2_196k: Coding coversation part. 30,185 samples
TokenBender/python_eval_instruct_51k: “python” in output .40,309 samples
Spider: 8,659 samples
codefuse-ai/Evol-Instruction-66k: 100%, 66,862 samples

Alpaca Prompt Format

### Instruction:
<instruction>
### Response:

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name_or_path="uukuguy/speechless-thoughts-mistral-7b"
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=True)
  model = AutoModelForCausalLM.from_pretrained(model_name_or_path, device_map="auto", trust_remote_code=True).eval()

  system = ""Below is an instruction that describes a task.\nWrite a response that appropriately completes the request.\n\n""
  prompt = f"{system}\n\n### Instruction:\n{instruction}\n\n### Response:"

  inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
  pred = model.generate(**inputs, max_length=4096, do_sample=True, top_k=50, top_p=0.99, temperature=0.9, num_return_sequences=1)
  print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True))

HumanEval

Metric	Value
humaneval-python

lm-evaluation-harness

json

{'ARC (acc_norm)': ,
'HellaSwag (acc_norm)': ,
'MMLU (acc)': ,
'TruthfulQA (mc2)': ,
'Winoground (acc)': ,
'GSM8K (acc)': ,
'DROP (f1)': ,
'Open LLM Score': }

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	59.72
ARC (25-shot)	58.96
HellaSwag (10-shot)	80.71
MMLU (5-shot)	60.11
TruthfulQA (0-shot)	49.91
Winogrande (5-shot)	77.82
GSM8K (5-shot)	30.78

RichardErkhov/uukuguy_-_speechless-thoughts-mistral-7b-gguf