You can deploy this model and start earning money today!

RichardErkhov/
rombodawg_-_Open_Gpt4_8x7B_v0.2-gguf

Name: rombodawg_-_Open_Gpt4_8x7B_v0.2-gguf
Availability: InStock

endpoints_compatible

gguf

template

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Open_Gpt4_8x7B_v0.2 - GGUF

Model creator: https://huggingface.co/rombodawg/
Original model: https://huggingface.co/rombodawg/Open_Gpt4_8x7B_v0.2/

Name	Quant method	Size
Open_Gpt4_8x7B_v0.2.Q2_K.gguf	Q2_K	5.7GB
Open_Gpt4_8x7B_v0.2.IQ3_XS.gguf	IQ3_XS	18.02GB
Open_Gpt4_8x7B_v0.2.IQ3_S.gguf	IQ3_S	19.03GB
Open_Gpt4_8x7B_v0.2.Q3_K_S.gguf	Q3_K_S	19.03GB
Open_Gpt4_8x7B_v0.2.IQ3_M.gguf	IQ3_M	19.96GB
Open_Gpt4_8x7B_v0.2.Q3_K.gguf	Q3_K	21.0GB
Open_Gpt4_8x7B_v0.2.Q3_K_M.gguf	Q3_K_M	21.0GB
Open_Gpt4_8x7B_v0.2.Q3_K_L.gguf	Q3_K_L	22.51GB
Open_Gpt4_8x7B_v0.2.IQ4_XS.gguf	IQ4_XS	23.63GB
Open_Gpt4_8x7B_v0.2.Q4_0.gguf	Q4_0	24.63GB
Open_Gpt4_8x7B_v0.2.IQ4_NL.gguf	IQ4_NL	24.91GB
Open_Gpt4_8x7B_v0.2.Q4_K_S.gguf	Q4_K_S	24.91GB
Open_Gpt4_8x7B_v0.2.Q4_K.gguf	Q4_K	26.49GB
Open_Gpt4_8x7B_v0.2.Q4_K_M.gguf	Q4_K_M	26.49GB
Open_Gpt4_8x7B_v0.2.Q4_1.gguf	Q4_1	27.32GB
Open_Gpt4_8x7B_v0.2.Q5_0.gguf	Q5_0	30.02GB
Open_Gpt4_8x7B_v0.2.Q5_K_S.gguf	Q5_K_S	30.02GB
Open_Gpt4_8x7B_v0.2.Q5_K.gguf	Q5_K	30.95GB
Open_Gpt4_8x7B_v0.2.Q5_K_M.gguf	Q5_K_M	30.95GB
Open_Gpt4_8x7B_v0.2.Q5_1.gguf	Q5_1	32.71GB
Open_Gpt4_8x7B_v0.2.Q6_K.gguf	Q6_K	35.74GB
Open_Gpt4_8x7B_v0.2.Q8_0.gguf	Q8_0	46.22GB

Original model description:

license: cc-by-4.0 tags:

merge
moe model-index:
name: Open_Gpt4_8x7B_v0.2 results:
- task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge (25-Shot) type: ai2_arc config: ARC-Challenge split: test args: num_few_shot: 25 metrics:
  - type: acc_norm value: 68.69 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=rombodawg/Open_Gpt4_8x7B_v0.2 name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: HellaSwag (10-Shot) type: hellaswag split: validation args: num_few_shot: 10 metrics:
  - type: acc_norm value: 86.16 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=rombodawg/Open_Gpt4_8x7B_v0.2 name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: MMLU (5-Shot) type: cais/mmlu config: all split: test args: num_few_shot: 5 metrics:
  - type: acc value: 72.07 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=rombodawg/Open_Gpt4_8x7B_v0.2 name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: TruthfulQA (0-shot) type: truthful_qa config: multiple_choice split: validation args: num_few_shot: 0 metrics:
  - type: mc2 value: 71.92 source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=rombodawg/Open_Gpt4_8x7B_v0.2 name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: Winogrande (5-shot) type: winogrande config: winogrande_xl split: validation args: num_few_shot: 5 metrics:
  - type: acc value: 83.58 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=rombodawg/Open_Gpt4_8x7B_v0.2 name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: GSM8k (5-shot) type: gsm8k config: main split: test args: num_few_shot: 5 metrics:
  - type: acc value: 59.14 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=rombodawg/Open_Gpt4_8x7B_v0.2 name: Open LLM Leaderboard

Open_Gpt4_v0.2

This is the un-quantized fp16 version for training and merging. If you want the quantized version for inference please refer to the repo bellow:

https://huggingface.co/rombodawg/Open_Gpt4_8x7B_v0.2_q8_0_gguf

This model is a TIES merger of Mixtral-8x7B-Instruct-v0.1 and bagel-dpo-8x7b-v0.2 with MixtralOrochi8x7B being the Base model.

I was very impressed with MixtralOrochi8x7B performance and multifaceted usecases as it is already a merger of many usefull Mixtral models such as Mixtral instruct, Noromaid-v0.1-mixtral, openbuddy-mixtral and possibly other models that were not named. My goal was to expand the models capabilities and make it even more useful of a model, maybe even competitive with closed source models like Gpt-4. But for that more testing is required. I hope the community can help me determine if its deserving of its name. 😊

This is the second iteration of this model, using better models in the merger to improve performance (hopefully).

Base model:

https://huggingface.co/smelborp/MixtralOrochi8x7B

Merged models:

Instruct template: Alpaca

Merger config:

yaml

models:
  - model: Mixtral-8x7B-Instruct-v0.1

    parameters:
      density: .5
      weight: 1
  - model: bagel-dpo-8x7b-v0.2
    parameters:
      density: .5
      weight: .7


merge_method: ties
base_model: MixtralOrochi8x7B
parameters:
  normalize: true
  int8_mask: true
dtype: float16

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	73.59
AI2 Reasoning Challenge (25-Shot)	68.69
HellaSwag (10-Shot)	86.16
MMLU (5-Shot)	72.07
TruthfulQA (0-shot)	71.92
Winogrande (5-shot)	83.58
GSM8k (5-shot)	59.14

RichardErkhov/rombodawg_-_Open_Gpt4_8x7B_v0.2-gguf

Original model description:

Open LLM Leaderboard Evaluation Results

RichardErkhov/
rombodawg_-_Open_Gpt4_8x7B_v0.2-gguf