Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
BlackSheep-24B-GPTQ – AI Model by GusPuffy | AlphaNeural AI
You can deploy this model and start earning money today!
GusPuffy
/
BlackSheep-24B-GPTQ
like
0
transformers
safetensors
mistral
text-generation
mergekit
merge
llmcompressor
GPTQ
conversational
openerotica/erotiquant3
TroyDoesAI/BlackSheep-24B
quantized
artistic-2.0
autotrain_compatible
text-generation-inference
endpoints_compatible
compressed-tensors
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Sentient Simulations Plumbob
[🏠Sentient Simulations]
|
[Discord]
|
[Patreon]
BlackSheep-24B-GPTQ
This repository contains a 4 bit GPTQ-quantized version of the
TroyDoesAI/BlackSheep-24B model
using
llm-compressor
.
Quantization Settings
Attribute
Value
Algorithm
GPTQ
Layers
Linear
Weight Scheme
W4A16
Group Size
128
Calibration Dataset
openerotica/erotiquant3
Calibration Sequence Length
4900
Calibration Samples
1024
Dataset Preprocessing
The dataset was preprocessed with the following steps:
Extract and structure the conversation data using role-based templates (
SYSTEM
,
USER
,
ASSISTANT
).
Convert the structured conversations into a tokenized format using the model's tokenizer.
Filter out sequences shorter than 4096 tokens.
Shuffle and select 512 samples for calibration.
Quantization Process
View the shell and python script used to quantize this model.
1 local 3090 128GB of ram.
compress.sh
compress.py
Acknowledgments
Base Model:
TroyDoesAI/BlackSheep-24B
Calibration Dataset:
openerotica/erotiquant3
LLM Compressor:
llm-compressor
Everyone subscribed to the
Sentient Simulations Patreon
patreon.PNG
