Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
Noromaid-v0.1-mixtral-8x7b-Instruct-v3-3bpw-exl2 – AI Model by Kooten | AlphaNeural AI
You can deploy this model and start earning money today!
Kooten
/
Noromaid-v0.1-mixtral-8x7b-Instruct-v3-3bpw-exl2
like
0
transformers
pytorch
mixtral
text-generation
conversational
cc-by-nc-4.0
autotrain_compatible
text-generation-inference
endpoints_compatible
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Noromaid-v0.1-mixtral-8x7b-Instruct-v3 3bpw
Exllama quant of
NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3
You probably want to use the
3.5bpw
version, it lands very close to 24gb of vram with 8bit cache enabled. GGUF is probably preferable if you have less than 24gb.
Other BPW's
3.0bpw
,
3.5bpw
,
4.0bpw
Prompt format:
### Instruction: {system prompt} ### Input: {input} ### Response: {reply}
Sillytavern presets
Discussion with presets
Context :
https://files.catbox.moe/vnnsra.json
Instruct :
https://files.catbox.moe/7sudj5.json
Preset :
https://files.catbox.moe/og94e5.json
Contact
Kooten on discord.
