Model Description
A LORA trained on the book 120 Days of Sodom. It Is intended for heavy NSFW content. Many users might find it disturbing.
- Developed by: Zattio770
- Model type: large language model LORA
- License:
- Finetuned from model [optional]: MythoMax-L2-13b, Llama-13B
Uses
Direct Use
This model is finetuned for heavy NSFW roleplay, while can still an assistant. (Might not be very helpfull)
Out-of-Scope Use
Be safe, I am not responsible for what this lora outputs. Do anything you want. I don't care
Bias, Risks, and Limitations
Model might have bias to NSFW due to the large % of NSFW data in the training set.
Training Details
Training Data
raw text
The 120 Days of Sodom by Marquis de Sade(1904)
Training Procedure
4bit MythoMaxL2
Rank: 128
Alpha: 256
Batch size: 128
Micro Batch: 1
Cutoff: 256
Epoch: 3
Learning Rate: 3e-4
Overlap Length: 64
Prefer Newline: 128
Training Hyperparameters
- Training regime: BF16, QLoRA, constant LR 5e-5
Compute Infrastructure
Hardware Type: 3080Ti
Hours used: 10
OogaBooga
