Created using the defaults from exllamav2 0.0.21 convert.py
3.0bpw to 6.0bpw head bits = 6
8.0bpw head bits = 8
length = 8192
dataset rows = 200
measurement rows = 32
measurement length = 8192
Dataset curation to remove slop-perceived expressions continues.
The below regexes return 0 matches. Bold entries are new since v0.3.
'barely above a whisper',
'barely audible',
'shiver([s]?) down',
' ministration',
'audible (["'"]?)p[l]?op',
'can't help but',
'buck([s]?) my ',
'buck([s]?) h[ei][rs] ',
'[Dd]espite h[ie][mr]self',
'slick slit',
'whatever it takes',
'unlike anything (s?)he',
'a mix([a-z]*) of',
'wave after wave',
'reckless abandon',
'[Mm]aybe, just maybe',
'eyes gleaming',
'mischievously',
"couldn't help but",
From testing so far, it feels like temperature 0.8-0.9 is a good starting point. I have mostly tested with everything neutralized. Please give feedback on which parameters work good for you.
