You can deploy this model and start earning money today!

RichardErkhov/
rombodawg_-_Open_Gpt4_8x7B_v0.1-gguf

Name: rombodawg_-_Open_Gpt4_8x7B_v0.1-gguf
Availability: InStock

endpoints_compatible

gguf

template

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Open_Gpt4_8x7B_v0.1 - GGUF

Model creator: https://huggingface.co/rombodawg/
Original model: https://huggingface.co/rombodawg/Open_Gpt4_8x7B_v0.1/

Name	Quant method	Size
Open_Gpt4_8x7B_v0.1.Q2_K.gguf	Q2_K	16.12GB
Open_Gpt4_8x7B_v0.1.IQ3_XS.gguf	IQ3_XS	18.02GB
Open_Gpt4_8x7B_v0.1.IQ3_S.gguf	IQ3_S	19.03GB
Open_Gpt4_8x7B_v0.1.Q3_K_S.gguf	Q3_K_S	19.03GB
Open_Gpt4_8x7B_v0.1.IQ3_M.gguf	IQ3_M	19.96GB
Open_Gpt4_8x7B_v0.1.Q3_K.gguf	Q3_K	21.0GB
Open_Gpt4_8x7B_v0.1.Q3_K_M.gguf	Q3_K_M	21.0GB
Open_Gpt4_8x7B_v0.1.Q3_K_L.gguf	Q3_K_L	22.51GB
Open_Gpt4_8x7B_v0.1.IQ4_XS.gguf	IQ4_XS	23.63GB
Open_Gpt4_8x7B_v0.1.Q4_0.gguf	Q4_0	24.63GB
Open_Gpt4_8x7B_v0.1.IQ4_NL.gguf	IQ4_NL	24.91GB
Open_Gpt4_8x7B_v0.1.Q4_K_S.gguf	Q4_K_S	24.91GB
Open_Gpt4_8x7B_v0.1.Q4_K.gguf	Q4_K	26.49GB
Open_Gpt4_8x7B_v0.1.Q4_K_M.gguf	Q4_K_M	26.49GB
Open_Gpt4_8x7B_v0.1.Q4_1.gguf	Q4_1	27.32GB
Open_Gpt4_8x7B_v0.1.Q5_0.gguf	Q5_0	30.02GB
Open_Gpt4_8x7B_v0.1.Q5_K_S.gguf	Q5_K_S	30.02GB
Open_Gpt4_8x7B_v0.1.Q5_K.gguf	Q5_K	30.95GB
Open_Gpt4_8x7B_v0.1.Q5_K_M.gguf	Q5_K_M	30.95GB
Open_Gpt4_8x7B_v0.1.Q5_1.gguf	Q5_1	32.71GB
Open_Gpt4_8x7B_v0.1.Q6_K.gguf	Q6_K	35.74GB
Open_Gpt4_8x7B_v0.1.Q8_0.gguf	Q8_0	46.22GB

Original model description:

license: cc-by-4.0 tags:

merge
moe

Open_Gpt4 cc

VERSION 0.2 OUT NOW:

Fp16:

https://huggingface.co/rombodawg/Open_Gpt4_8x7B_v0.2

q8_0.gguf:

https://huggingface.co/rombodawg/Open_Gpt4_8x7B_v0.2_q8_0_gguf

This model is a TIES merger of notux-8x7b-v1 and UNAversal-8x7B-v1beta with MixtralOrochi8x7B being the Base model.

I was very impressed with MixtralOrochi8x7B performance and multifaceted usecases as it is already a merger of many usefull Mixtral models such as Mixtral instruct,

Noromaid-v0.1-mixtral, openbuddy-mixtral and possibly other models that were not named. My goal was to expand the models capabilities and make it even more useful of a model, maybe even competitive with closed source models like Gpt-4. But for that more testing is required. I hope the community can help me determine if its deserving of its name. 😊

Base model:

https://huggingface.co/smelborp/MixtralOrochi8x7B

Merged models:

Instruct template: Alpaca

Merger config:

yaml

models:
  - model: notux-8x7b-v1
    parameters:
      density: .5
      weight: 1
  - model: UNAversal-8x7B-v1beta
    parameters:
      density: .5
      weight: 1


merge_method: ties
base_model: MixtralOrochi8x7B
parameters:
  normalize: true
  int8_mask: true
dtype: float16

RichardErkhov/rombodawg_-_Open_Gpt4_8x7B_v0.1-gguf

Original model description:

RichardErkhov/
rombodawg_-_Open_Gpt4_8x7B_v0.1-gguf