Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
olmo-gguf – AI Model by calcuis | AlphaNeural AI
You can deploy this model and start earning money today!
calcuis
/
olmo-gguf
like
0
allenai/OLMo-7B-0724-Instruct-hf
quantized
conversational
allenai/tulu-v3.1-mix-preview-4096-OLMoE
en
endpoints_compatible
gguf
template
apache-2.0
us
text2text-generation
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
GGUF quantized version of OLMo-7B-0724-Instruct Model
project original source:
base model
Q_2_K (not nice)
Q_3_K_S (acceptable)
Q_3_K_M is acceptable (good for running with CPU)
Q_3_K_L (acceptable)
Q_4_K_S (okay)
Q_4_K_M is recommanded (balance)
Q_5_K_S (good)
Q_5_K_M (good in general)
Q_6_K is good also; if you want a better result; take this one instead of Q_5_K_M
Q_8_0 which is very good; need a reasonable size of RAM otherwise you might expect a long wait
f16 is similar to the original hf model; opt this one or hf also fine; make sure you have a good machine
how to run it
use any connector for interacting with gguf; i.e.,
gguf-connector
OLMo Logo
this picture is from the base model
