This repo contains gguf files for Emerhyst-20B. This was done with a modern llamacpp with the current modern gguf format; you shouldn't see warnings when loading it, compared to Undi's quants.
Yeah, I know everyone says the Llama2 days are over, so someone explain to me why current-era models are FUCKING HOT GARBAGE at ERP compared to an old-ass hack merge?
We got so close with Mixtral and then everyone just slathered LimaRP over it and called it a day.
Do you have any idea how much slop is in LimaRP? Do you have any idea how many Claude-isms and OAI-isms are in the shit people are using
these days? I'm serious when I say I'm just gonna go back to Emer. Like, bigger models are sweet and all, with better comprehension (sometimes) but
they write like hand-hammered dookie even with a 2000-token long token ban list.
Not to mention why in the living crap is everything like 70B? You have to quant that down to sweet F-A to even boot the thing. Do I seriously
expect to not die of old age running a 70 on a single video card? Mistral-Small-22 is god-tier slop infected. Llama3 was basically made by
scraping ChatGPT and Claude slop. The weird chinese models are weird.
Yeah, so it's time to go back to the roots. It's time to admit we fucked up, and we should Return to Monke. The current choices in the current
LLM world are "zomg 70b++++" and "barely functional 7B you can run on a phone." Like, what about a nice 20-30B? What about? Bros. Bro. My
brother in Christ. We need a model that runs on the kind of old-ass PC a kid would inherit from his dad. We need a model that runs on the
kind of hardware someone brings home from work.
I mean, shit, at this point we just need a model that fits in a fucking 3090. Like, what, I'm supposed to rent a 4xGPU runpod to
infer from? "Local LLM" my ass.
/End Rant
Anyway, there are no exl2 to be found here, or whatever weird torch quants you people use. Dude, invent the ability to spill over
into CPU. Oh, wait, EVERYONE BUT YOU OOBA DIPSHITS already has that.
So there's a 5_K_M and a Q6 and if you want something else the fp16 is in there. Maybe that robot that scrapes people's repos and
quants them will come along if you want some other size? I mean, that was half the point of uploading this, so that one guy's
bot will popularize an Emer that's usable by kcpp newer than two years old.
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
