You can deploy this model and start earning money today!

RichardErkhov/
allknowingroger_-_llama3Yi-40B-gguf

Name: allknowingroger_-_llama3Yi-40B-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

llama3Yi-40B - GGUF

Model creator: https://huggingface.co/allknowingroger/
Original model: https://huggingface.co/allknowingroger/llama3Yi-40B/

Name	Quant method	Size
llama3Yi-40B.Q2_K.gguf	Q2_K	13.85GB
llama3Yi-40B.IQ3_XS.gguf	IQ3_XS	15.37GB
llama3Yi-40B.IQ3_S.gguf	IQ3_S	16.23GB
llama3Yi-40B.Q3_K_S.gguf	Q3_K_S	16.17GB
llama3Yi-40B.IQ3_M.gguf	IQ3_M	16.82GB
llama3Yi-40B.Q3_K.gguf	Q3_K	18.02GB
llama3Yi-40B.Q3_K_M.gguf	Q3_K_M	18.02GB
llama3Yi-40B.Q3_K_L.gguf	Q3_K_L	19.62GB
llama3Yi-40B.IQ4_XS.gguf	IQ4_XS	20.15GB
llama3Yi-40B.Q4_0.gguf	Q4_0	21.05GB
llama3Yi-40B.IQ4_NL.gguf	IQ4_NL	21.25GB
llama3Yi-40B.Q4_K_S.gguf	Q4_K_S	21.19GB
llama3Yi-40B.Q4_K.gguf	Q4_K	22.31GB
llama3Yi-40B.Q4_K_M.gguf	Q4_K_M	22.31GB
llama3Yi-40B.Q4_1.gguf	Q4_1	23.35GB
llama3Yi-40B.Q5_0.gguf	Q5_0	25.65GB
llama3Yi-40B.Q5_K_S.gguf	Q5_K_S	25.65GB
llama3Yi-40B.Q5_K.gguf	Q5_K	26.3GB
llama3Yi-40B.Q5_K_M.gguf	Q5_K_M	26.3GB
llama3Yi-40B.Q5_1.gguf	Q5_1	27.95GB
llama3Yi-40B.Q6_K.gguf	Q6_K	30.54GB
llama3Yi-40B.Q8_0.gguf	Q8_0	39.55GB

Original model description:

base_model:

01-ai/Yi-1.5-34B-Chat library_name: transformers tags:
mergekit
merge license: apache-2.0

merge

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the passthrough merge method.

Models Merged

The following models were included in the merge:

01-ai/Yi-1.5-34B-Chat

Configuration

The following YAML configuration was used to produce this model:

yaml

slices:
  - sources:
    - model: 01-ai/Yi-1.5-34B-Chat
      layer_range: [0, 39]
  - sources:
    - model: 01-ai/Yi-1.5-34B-Chat
      layer_range: [8, 39]
merge_method: passthrough
dtype: bfloat16

Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more quants, at much higher speed, than I would otherwise be able to.

RichardErkhov/allknowingroger_-_llama3Yi-40B-gguf