You can deploy this model and start earning money today!

PhilLovesCats/
LLaMA-1-65B-GGUF

LLaMA 1 65B - GGUF

Model creator: Meta Llama
The 'Provided files'-section layout, as well as the metadata layout are heavily inspired from TheBloke
The 'License'-section, aswell as the 'LICENSE'-file are burrowed by huggyllama

Please tell me when i have violated your copyright, so that i can remove the belonging section!

Description

Due the fact that the old LLaMA (1) models are in a deprecated format and really hard to use with modern tools, i converted them using the transformers library's convert_llama_weights_to_hf.py script to convert them into a PyTorch model.

Further i used the convert_hf_to_gguf.py script from llama.cpp and to bring them into the well-known GGUF file format. Afterwards, i've build llama.cpp and ran llama-quantize.exe to downscale the model files.

Models > 50 GB were split with the gguf-split tool from llama.cpp and using this guide: https://github.com/ggml-org/llama.cpp/discussions/6404

Provided files

The following table only lists the quantized (.gguf) files (without the PyTorch (.bin) ones).

Name	Quant method	Bits	Size	MD5-Hash	Use case
llama-1-65b.TQ1_0.gguf	TQ1_0	1	13.70 GB	14AFF138168B0C07E218661253669162	smallest, significant quality loss - not recommended for most purposes
llama-1-65b.TQ2_0.gguf	TQ2_0	2	16.50 GB	21029F9B17EF252BFD12060450575434	very small, significant quality loss - not recommended for most purposes
llama-1-65b.Q2_K.gguf	Q2_K	2	22.40 GB	24ECE55B5157469C9F343430D03C6B14	very small, significant quality loss - not recommended for most purposes
llama-1-65b.Q3_K_S.gguf	Q3_K_S	3	26.20 GB	182A25B6477F4B9E08A20DF7C53EA99C	very small, high quality loss
llama-1-65b.Q3_K_M.gguf	Q3_K_M	3	29.40 GB	CF1AC34185D7EB309EA7F0887405E760	very small, high quality loss
llama-1-65b.Q3_K_L.gguf	Q3_K_L	3	32.20 GB	C824C9BFE1B52C6A70C00326172E1D0C	small, substantial quality loss
llama-1-65b.Q4_0.gguf	Q4_0	4	34.20 GB	FDB69C9A0E4CA4413FDBD2EEA5C419D8	legacy; small, very high quality loss - prefer using Q3_K_M
llama-1-65b.Q4_K_S.gguf	Q4_K_S	4	34.50 GB	1BC80164AF921D33174590D216800F69	small, greater quality loss
llama-1-65b.Q4_K_M.gguf	Q4_K_M	4	36.60 GB	1C94842178FEE7062EDA5A91D76E8B91	medium, balanced quality - recommended
llama-1-65b.Q5_0.gguf	Q5_0	5	41.80 GB	2166618D0CA3FB8DBFA6E494D54A345F	legacy; medium, balanced quality - prefer using Q4_K_M
llama-1-65b.Q5_K_S.gguf	Q5_K_S	5	41.80 GB	3402780368E0C82A7D9B385ED95462C5	large, low quality loss - recommended
llama-1-65b.Q5_K_M.gguf	Q5_K_M	5	43.00 GB	CF486B3FD3CFEF6F4817456B51D8915A	large, very low quality loss - recommended
llama-1-65b.Q6_K.gguf	Q6_K	6	49.80 GB	D5EF35C61A7FEA2518A963202BA04D02	very large, extremely low quality loss
llama-1-65b.Q8_0.gguf	Q8_0	8	64.60 GB	C9E1EB00A0831F857B19D416A8B2D63D	very large, extremely low quality loss - not recommended
llama-1-65b.F16.gguf	F16	16	121.00 GB	A0AE5F69C985BA6CF48527E87CDC2AFA	original, no quality loss - not recommended
llama-1-65b.BF16.gguf	BF16	16	121.00 GB	8C6E8AAFAD5B0853F083A1DA871CE40E	original, no quality loss - not recommended

Notice: Hashes were calculated with Get-FileHash .\FILENAME -Algorithm MD5 on Microsoft Windows 11 64-bit (x86_x64).

License

This model is under a non-commercial license (see the LICENSE file). You should only use this repository if you have been granted access to the model by filling out this form but either lost your copy of the weights or got some trouble converting them to the Transformers format.

PhilLovesCats/LLaMA-1-65B-GGUF

LLaMA 1 65B - GGUF

Description

Provided files

License

PhilLovesCats/
LLaMA-1-65B-GGUF