Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
Legion-V2.1-LLaMa-70B-GPTQ – AI Model by GusPuffy | AlphaNeural AI
You can deploy this model and start earning money today!
GusPuffy
/
Legion-V2.1-LLaMa-70B-GPTQ
like
0
transformers
safetensors
llama
text-generation
mergekit
merge
llmcompressor
GPTQ
conversational
openerotica/erotiquant3
Tarek07/Legion-V2.1-LLaMa-70B
quantized
llama3.3
autotrain_compatible
text-generation-inference
endpoints_compatible
compressed-tensors
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Sentient Simulations Plumbob
[🏠Sentient Simulations]
|
[Discord]
|
[Patreon]
Llama-3.1-70B-ArliAI-RPMax-v1.3-GPTQ
This repository contains a 4 bit GPTQ-quantized version of the
Tarek07/Legion-V2.1-LLaMa-70B model
using
llm-compressor
.
Quantization Settings
Attribute
Value
Algorithm
GPTQ
Layers
Linear
Weight Scheme
W4A16
Group Size
128
Calibration Dataset
openerotica/erotiquant3
Calibration Sequence Length
4096
Calibration Samples
512
Dataset Preprocessing
The dataset was preprocessed with the following steps:
Extract and structure the conversation data using role-based templates (
SYSTEM
,
USER
,
ASSISTANT
).
Convert the structured conversations into a tokenized format using the model's tokenizer.
Filter out sequences shorter than 4096 tokens.
Shuffle and select 512 samples for calibration.
Quantization Process
View the shell and python script used to quantize this model.
2 rtx pro 6000 with 565GB of ram, 300GB of disk space was rented on runpod.
Quantization took approximately 3.5 hours with a total of $14.32 in compute costs.
compress.sh
compress.py
Acknowledgments
Base Model:
Tarek07/Legion-V2.1-LLaMa-70B
Calibration Dataset:
openerotica/erotiquant3
LLM Compressor:
llm-compressor
Everyone subscribed to the
Sentient Simulations Patreon
patreon.PNG
