Metharme 7B 4bit
An instruction-tuned LLaMA biased towards fiction writing and conversation.
Model Details
Converted from the XORs weights from PygmalionAI's release
https://huggingface.co/PygmalionAI/metharme-7b I created several dozen quantized variations of this model and believe this variation to be "best."
Model: Wikitext2 Ptb-New C4-New Metharme 7b - 16bit 5.7208476066589355 41.61103439331055 7.512405872344971 Metharme-AO-TS-Trits-damp0.1 6.172733783721924 45.19890213012695 7.872506141662598
Other benchmark scores at the bottom of readme.
Metharme 7B is an instruct model based on Meta's LLaMA-7B.
This is an experiment to try and get a model that is usable for conversation, roleplaying and storywriting, but which can be guided using natural language like other instruct models. See the
prompting section below for examples.
It was trained by doing supervised fine-tuning over a mixture of regular instruction data alongside roleplay, fictional stories and conversations with synthetically generated instructions attached.
Prompting
The current model version has been trained on prompts using three different roles, which are denoted by the following tokens: <|system|>, <|user|> and <|model|>.
The <|system|> prompt can be used to inject out-of-channel information behind the scenes, while the <|user|> prompt should be used to indicate user input. The <|model|> token should then be used to indicate that the model should generate a response. These tokens can happen multiple times and be chained up to form a conversation history.
Prompting example
Here's a concrete example. Let's say you have the following system prompt:
This is a text adventure game. Describe the scenario to the user and give him three options to pick from on each turn.
And the user has typed in the following message:
To perform inference against the model, the prompt should be formatted like this:
<|system|>This is a text adventure game. Describe the scenario to the user and give him three options to pick from on each turn.<|user|>Start!<|model|>The model might generate something like:
You are standing in front of an old, abandoned house. The windows are boarded up, and there's no sign of life around it. As you approach, you notice a strange feeling emanating from within. Suddenly, you hear a voice calling out to you... 'Come inside!'
Go inside the house.
Ignore the call and move away.
Run as fast as you can.
It will automatically emit an end-of-text token (</s>) when it judges that the response is complete.
Pretend then that the user has replied with go inside. The next prompt should then be:
<|system|>This is a text adventure game. Describe the scenario to the user and give him three options to pick from on each turn.<|user|>Start!<|model|>You are standing in front of an old, abandoned house. The windows are boarded up, and there's no sign of life around it. As you approach, you notice a strange feeling emanating from within. Suddenly, you hear a voice calling out to you... 'Come inside!'
- Go inside the house.
- Ignore the call and move away.
- Run as fast as you can.<|user|>go inside<|model|>Which might generate something like:
You enter the house, and immediately, a wave of unsettling energy washes over you. Your senses seem heightened, but your mind feels foggy. A figure appears before you - a tall man with piercing eyes. He speaks to you in a language you don't understand.
Ask him what he wants.
Attack him.
Run away.
Same process applies. Usually, it is best to do a sliding window over the user and model turns, but keep the system prompt fixed at the start of the context window.
Limitations and biases
The intended use-case for this model is fictional writing for entertainment purposes. Any other sort of usage is out of scope.
As such, it was not fine-tuned to be safe and harmless: the base model and this fine-tune have been trained on data known to contain profanity and texts that are lewd or otherwise offensive. It may produce socially unacceptable or undesirable text, even if the prompt itself does not include anything explicitly offensive. Outputs might often be factually wrong or misleading.
Click to Expand Benchmarks of different quantized variations
The lower the number, the better the score.
Benchmarks Sorted by C4-New score
GPTQ Variation: Wikitext2 Ptb-New C4-New Metharme-7b-16bit 5.7208476066589355 41.61103439331055 7.512405872344971 Metharme-ao-ts-trits-damp0.1 6.172733783721924 45.19890213012695 7.872506141662598 Metharme-ao-trits-damp0.1 6.163661956787109 46.50249099731445 7.877425193786621 Metharme-ao-ts-damp0.1 6.184001445770264 46.17180633544922 7.880400657653809 Metharme-7b-4bit-ao-damp0.1 6.220707893371582 47.82929611206055 7.884565353393555 Metharme-ao-ts-trits 6.310682773590088 46.4483757019043 7.898126602172852 Metharme-7b-4bit-ao 6.281311511993408 46.79158401489258 7.906069755554199 Metharme-ao-trits 6.283935546875 46.57590103149414 7.907411575317383 Metharme-ao-ts 6.329496383666992 46.88129806518555 7.910323143005371 Metharme-ao-ts-sym-trits-damp0.1 6.232576370239258 48.081459045410156 7.95023250579834 Metharme-ao-ts-sym-damp0.1 6.210323333740234 47.66789245605469 7.952476978302002 Metharme-ao-sym-trits-damp0.1 6.329384803771973 48.06882858276367 7.959168910980225 Metharme-ao-ts-sym-trits 6.471063137054443 49.650611877441406 7.969552040100098 Metharme-ao-ts-sym 6.460526943206787 47.190460205078125 7.9732160568237305 Metharme-ao-sym-trits 6.390106678009033 48.15375900268555 7.9804582595825195 Metharme-7b-4bit-ao-sym 6.477842807769775 48.53507614135742 7.993765354156494 Metharme-ao-sym 6.477842807769775 48.53507614135742 7.993765354156494 Metharme-7b-4bit-ts-32g-trits 6.632943153381348 47.973228454589844 8.013848304748535 Metharme-7b-4bit-ts-32g-sym-damp0.1 6.274552822113037 47.35737228393555 8.06270980834961 Metharme-7b-4bit-ts-32g-sym-trits-damp0.1 6.266031265258789 47.346702575683594 8.068148612976074 Metharme-7b-4bit-32g-damp0.1 6.107605934143066 47.91380310058594 8.068695068359375 Metharme-7b-4bit-32g-trits-damp0.1 6.128157138824463 48.04175567626953 8.0708646774292 Metharme-7b-4bit-ts-32g-damp0.1 6.219024658203125 45.834869384765625 8.071272850036621 Metharme-7b-4bit-32g-trits 7.017086029052734 45.04129409790039 8.074845314025879 Metharme-7b-4bit-32g-sym-trits-damp0.1 6.109438896179199 47.35737228393555 8.075060844421387 Metharme-7b-4bit-ts-32g-trits-damp0.1 6.118431568145752 45.67333221435547 8.077078819274902 Metharme-7b-4bit-32g 6.902080535888672 50.237754821777344 8.081602096557617 Metharme-7b-4bit-ts-32g 6.424218654632568 48.48588943481445 8.089512825012207 Metharme-7b-4bit-ts-32g-sym-trits 6.82415771484375 48.82029724121094 8.090987205505371 Metharme-7b-4bit-32g-sym-damp0.1 6.566899299621582 48.0670166015625 8.095841407775879 Metharme-7b-4bit-ts-128g-trits-damp0.1 6.289113521575928 46.06787109375 8.122251510620117 Metharme-7b-4bit-32g-sym 6.518134117126465 49.66925811767578 8.13516616821289 Metharme-7b-4bit-32g-sym-trits 6.206963539123535 46.88833999633789 8.13610553741455 Metharme-7b-4bit-128g-damp0.1 6.242006301879883 45.30938720703125 8.14249324798584 Metharme-7b-4bit-ts-32g-sym 6.387663841247559 48.07244110107422 8.173730850219727 Metharme-7b-4bit-128g-trits-damp0.1 6.262309551239014 47.80055618286133 8.192194938659668 Metharme-7b-4bit-128g 10.206376075744629 49.00401306152344 8.198845863342285 Metharme-7b-4bit-ts-128g-damp0.1 6.17774772644043 46.47630310058594 8.20170783996582 Metharme-7b-4bit-ts-128g-sym-trits-damp0.1 6.225503921508789 53.12746047973633 8.240595817565918 Metharme-7b-4bit-128g-trits 8.68796443939209 49.73833465576172 8.2406587600708 Metharme-7b-4bit-ts-128g-sym-damp0.1 6.584965705871582 55.20026397705078 8.268644332885742 Metharme-7b-4bit-ts-128g-trits 7.350858688354492 44.25314712524414 8.274221420288086 Metharme-7b-4bit-128g-sym-trits-damp0.1 6.585468769073486 51.55869674682617 8.2803316116333 Metharme-7b-4bit-ts-128g-sym-trits 6.756448745727539 51.510311126708984 8.292160987854004 Metharme-7b-4bit-128g-sym-damp0.1 6.379064083099365 52.17233657836914 8.316649436950684 Metharme-7b-4bit-128g-sym-trits 7.056288242340088 48.983768463134766 8.339276313781738 Metharme-7b-4bit-ts-128g 9.475017547607422 52.358829498291016 8.340700149536133 Metharme-7b-4bit-128g-sym 6.9575653076171875 49.356834411621094 8.35644817352295 Metharme-7b-4bit-ts-128g-sym 6.819341659545898 55.28740310668945 8.377721786499023 Metharme-7b-4bit-ts-damp0.1 6.7783050537109375 51.81301498413086 8.621373176574707 Metharme-7b-4bit-ts-trits-damp0.1 6.631694793701172 51.7371711730957 8.656966209411621 Metharme-7b-4bit-damp0.1 6.495014190673828 49.39763641357422 8.68167781829834 Metharme-sym-damp0.1 6.896804332733154 57.4250602722168 8.703770637512207 Metharme-7b-4bit-ts-sym 7.270263671875 54.35262680053711 8.787986755371094 Metharme-7b-4bit-trits 7.832409858703613 55.383026123046875 8.806737899780273 Metharme-trits 7.832409858703613 55.383026123046875 8.806737899780273 Metharme-7b-4bit-ts-sym-damp0.1 6.7517595291137695 54.06147384643555 8.821818351745605 Metharme-7b-4bit-alone 6.997134685516357 58.87525177001953 8.824191093444824 Metharme-7b-4bit-ts-trits 7.2306809425354 66.78710174560547 8.879831314086914 Metharme-7b-4bit-ts-sym-trits-damp0.1 6.886506080627441 64.72743225097656 8.880627632141113 Metharme-7b-4bit-ts 7.735969543457031 62.92238235473633 8.913650512695312 Metharme-sym-trits-damp0.1 7.075908184051514 59.13897705078125 8.919178009033203 Metharme-7b-4bit-ts-sym-trits 7.599876403808594 55.75454330444336 8.932201385498047 Metharme-sym-trits 7.494253635406494 63.320709228515625 8.969240188598633 Metharme-sym 7.585672855377197 61.01168441772461 9.032520294189453 Metharme-7b-4bit-ao-128g 251321.265625 250117.859375 232929.234375 Metharme-7b-4bit-ao-32g 275425.5 267733.25 254506.71875
Benchmarks Sorted by Wikitext2
GPTQ Variation: Wikitext2 Ptb-New C4-New Metharme-7b-16bit 5.7208476066589355 41.61103439331055 7.512405872344971 Metharme-7b-4bit-32g-damp0.1 6.107605934143066 47.91380310058594 8.068695068359375 Metharme-7b-4bit-32g-sym-trits-damp0.1 6.109438896179199 47.35737228393555 8.075060844421387 Metharme-7b-4bit-ts-32g-trits-damp0.1 6.118431568145752 45.67333221435547 8.077078819274902 Metharme-7b-4bit-32g-trits-damp0.1 6.128157138824463 48.04175567626953 8.0708646774292 Metharme-ao-trits-damp0.1 6.163661956787109 46.50249099731445 7.877425193786621 Metharme-ao-ts-trits-damp0.1 6.172733783721924 45.19890213012695 7.872506141662598 Metharme-7b-4bit-ts-128g-damp0.1 6.17774772644043 46.47630310058594 8.20170783996582 Metharme-ao-ts-damp0.1 6.184001445770264 46.17180633544922 7.880400657653809 Metharme-7b-4bit-32g-sym-trits 6.206963539123535 46.88833999633789 8.13610553741455 Metharme-ao-ts-sym-damp0.1 6.210323333740234 47.66789245605469 7.952476978302002 Metharme-7b-4bit-ts-32g-damp0.1 6.219024658203125 45.834869384765625 8.071272850036621 Metharme-7b-4bit-ao-damp0.1 6.220707893371582 47.82929611206055 7.884565353393555 Metharme-7b-4bit-ts-128g-sym-trits-damp0.1 6.225503921508789 53.12746047973633 8.240595817565918 Metharme-ao-ts-sym-trits-damp0.1 6.232576370239258 48.081459045410156 7.95023250579834 Metharme-7b-4bit-128g-damp0.1 6.242006301879883 45.30938720703125 8.14249324798584 Metharme-7b-4bit-128g-trits-damp0.1 6.262309551239014 47.80055618286133 8.192194938659668 Metharme-7b-4bit-ts-32g-sym-trits-damp0.1 6.266031265258789 47.346702575683594 8.068148612976074 Metharme-7b-4bit-ts-32g-sym-damp0.1 6.274552822113037 47.35737228393555 8.06270980834961 Metharme-7b-4bit-ao 6.281311511993408 46.79158401489258 7.906069755554199 Metharme-ao-trits 6.283935546875 46.57590103149414 7.907411575317383 Metharme-7b-4bit-ts-128g-trits-damp0.1 6.289113521575928 46.06787109375 8.122251510620117 Metharme-ao-ts-trits 6.310682773590088 46.4483757019043 7.898126602172852 Metharme-ao-sym-trits-damp0.1 6.329384803771973 48.06882858276367 7.959168910980225 Metharme-ao-ts 6.329496383666992 46.88129806518555 7.910323143005371 Metharme-7b-4bit-128g-sym-damp0.1 6.379064083099365 52.17233657836914 8.316649436950684 Metharme-7b-4bit-ts-32g-sym 6.387663841247559 48.07244110107422 8.173730850219727 Metharme-ao-sym-trits 6.390106678009033 48.15375900268555 7.9804582595825195 Metharme-7b-4bit-ts-32g 6.424218654632568 48.48588943481445 8.089512825012207 Metharme-ao-ts-sym 6.460526943206787 47.190460205078125 7.9732160568237305 Metharme-ao-ts-sym-trits 6.471063137054443 49.650611877441406 7.969552040100098 Metharme-7b-4bit-ao-sym 6.477842807769775 48.53507614135742 7.993765354156494 Metharme-ao-sym 6.477842807769775 48.53507614135742 7.993765354156494 Metharme-7b-4bit-damp0.1 6.495014190673828 49.39763641357422 8.68167781829834 Metharme-7b-4bit-32g-sym 6.518134117126465 49.66925811767578 8.13516616821289 Metharme-7b-4bit-32g-sym-damp0.1 6.566899299621582 48.0670166015625 8.095841407775879 Metharme-7b-4bit-ts-128g-sym-damp0.1 6.584965705871582 55.20026397705078 8.268644332885742 Metharme-7b-4bit-128g-sym-trits-damp0.1 6.585468769073486 51.55869674682617 8.2803316116333 Metharme-7b-4bit-ts-trits-damp0.1 6.631694793701172 51.7371711730957 8.656966209411621 Metharme-7b-4bit-ts-32g-trits 6.632943153381348 47.973228454589844 8.013848304748535 Metharme-7b-4bit-ts-sym-damp0.1 6.7517595291137695 54.06147384643555 8.821818351745605 Metharme-7b-4bit-ts-128g-sym-trits 6.756448745727539 51.510311126708984 8.292160987854004 Metharme-7b-4bit-ts-damp0.1 6.7783050537109375 51.81301498413086 8.621373176574707 Metharme-7b-4bit-ts-128g-sym 6.819341659545898 55.28740310668945 8.377721786499023 Metharme-7b-4bit-ts-32g-sym-trits 6.82415771484375 48.82029724121094 8.090987205505371 Metharme-7b-4bit-ts-sym-trits-damp0.1 6.886506080627441 64.72743225097656 8.880627632141113 Metharme-sym-damp0.1 6.896804332733154 57.4250602722168 8.703770637512207 Metharme-7b-4bit-32g 6.902080535888672 50.237754821777344 8.081602096557617 Metharme-7b-4bit-128g-sym 6.9575653076171875 49.356834411621094 8.35644817352295 Metharme-7b-4bit-alone 6.997134685516357 58.87525177001953 8.824191093444824 Metharme-7b-4bit-32g-trits 7.017086029052734 45.04129409790039 8.074845314025879 Metharme-7b-4bit-128g-sym-trits 7.056288242340088 48.983768463134766 8.339276313781738 Metharme-sym-trits-damp0.1 7.075908184051514 59.13897705078125 8.919178009033203 Metharme-7b-4bit-ts-trits 7.2306809425354 66.78710174560547 8.879831314086914 Metharme-7b-4bit-ts-sym 7.270263671875 54.35262680053711 8.787986755371094 Metharme-7b-4bit-ts-128g-trits 7.350858688354492 44.25314712524414 8.274221420288086 Metharme-sym-trits 7.494253635406494 63.320709228515625 8.969240188598633 Metharme-sym 7.585672855377197 61.01168441772461 9.032520294189453 Metharme-7b-4bit-ts-sym-trits 7.599876403808594 55.75454330444336 8.932201385498047 Metharme-7b-4bit-ts 7.735969543457031 62.92238235473633 8.913650512695312 Metharme-7b-4bit-trits 7.832409858703613 55.383026123046875 8.806737899780273 Metharme-trits 7.832409858703613 55.383026123046875 8.806737899780273 Metharme-7b-4bit-128g-trits 8.68796443939209 49.73833465576172 8.2406587600708 Metharme-7b-4bit-ts-128g 9.475017547607422 52.358829498291016 8.340700149536133 Metharme-7b-4bit-128g 10.206376075744629 49.00401306152344 8.198845863342285 Metharme-7b-4bit-ao-128g 251321.265625 250117.859375 232929.234375 Metharme-7b-4bit-ao-32g 275425.5 267733.25 254506.71875
Benchmarks Sorted by PTB-new Score
GPTQ Variation: Wikitext2 Ptb-New C4-New Metharme-7b-16bit 5.7208476066589355 41.61103439331055 7.512405872344971 Metharme-7b-4bit-ts-128g-trits 7.350858688354492 44.25314712524414 8.274221420288086 Metharme-7b-4bit-32g-trits 7.017086029052734 45.04129409790039 8.074845314025879 Metharme-ao-ts-trits-damp0.1 6.172733783721924 45.19890213012695 7.872506141662598 Metharme-7b-4bit-128g-damp0.1 6.242006301879883 45.30938720703125 8.14249324798584 Metharme-7b-4bit-ts-32g-trits-damp0.1 6.118431568145752 45.67333221435547 8.077078819274902 Metharme-7b-4bit-ts-32g-damp0.1 6.219024658203125 45.834869384765625 8.071272850036621 Metharme-7b-4bit-ts-128g-trits-damp0.1 6.289113521575928 46.06787109375 8.122251510620117 Metharme-ao-ts-damp0.1 6.184001445770264 46.17180633544922 7.880400657653809 Metharme-ao-ts-trits 6.310682773590088 46.4483757019043 7.898126602172852 Metharme-7b-4bit-ts-128g-damp0.1 6.17774772644043 46.47630310058594 8.20170783996582 Metharme-ao-trits-damp0.1 6.163661956787109 46.50249099731445 7.877425193786621 Metharme-ao-trits 6.283935546875 46.57590103149414 7.907411575317383 Metharme-7b-4bit-ao 6.281311511993408 46.79158401489258 7.906069755554199 Metharme-ao-ts 6.329496383666992 46.88129806518555 7.910323143005371 Metharme-7b-4bit-32g-sym-trits 6.206963539123535 46.88833999633789 8.13610553741455 Metharme-ao-ts-sym 6.460526943206787 47.190460205078125 7.9732160568237305 Metharme-7b-4bit-ts-32g-sym-trits-damp0.1 6.266031265258789 47.346702575683594 8.068148612976074 Metharme-7b-4bit-ts-32g-sym-damp0.1 6.274552822113037 47.35737228393555 8.06270980834961 Metharme-7b-4bit-32g-sym-trits-damp0.1 6.109438896179199 47.35737228393555 8.075060844421387 Metharme-ao-ts-sym-damp0.1 6.210323333740234 47.66789245605469 7.952476978302002 Metharme-7b-4bit-128g-trits-damp0.1 6.262309551239014 47.80055618286133 8.192194938659668 Metharme-7b-4bit-ao-damp0.1 6.220707893371582 47.82929611206055 7.884565353393555 Metharme-7b-4bit-32g-damp0.1 6.107605934143066 47.91380310058594 8.068695068359375 Metharme-7b-4bit-ts-32g-trits 6.632943153381348 47.973228454589844 8.013848304748535 Metharme-7b-4bit-32g-trits-damp0.1 6.128157138824463 48.04175567626953 8.0708646774292 Metharme-7b-4bit-32g-sym-damp0.1 6.566899299621582 48.0670166015625 8.095841407775879 Metharme-ao-sym-trits-damp0.1 6.329384803771973 48.06882858276367 7.959168910980225 Metharme-7b-4bit-ts-32g-sym 6.387663841247559 48.07244110107422 8.173730850219727 Metharme-ao-ts-sym-trits-damp0.1 6.232576370239258 48.081459045410156 7.95023250579834 Metharme-ao-sym-trits 6.390106678009033 48.15375900268555 7.9804582595825195 Metharme-7b-4bit-ts-32g 6.424218654632568 48.48588943481445 8.089512825012207 Metharme-7b-4bit-ao-sym 6.477842807769775 48.53507614135742 7.993765354156494 Metharme-ao-sym 6.477842807769775 48.53507614135742 7.993765354156494 Metharme-7b-4bit-ts-32g-sym-trits 6.82415771484375 48.82029724121094 8.090987205505371 Metharme-7b-4bit-128g-sym-trits 7.056288242340088 48.983768463134766 8.339276313781738 Metharme-7b-4bit-128g 10.206376075744629 49.00401306152344 8.198845863342285 Metharme-7b-4bit-128g-sym 6.9575653076171875 49.356834411621094 8.35644817352295 Metharme-7b-4bit-damp0.1 6.495014190673828 49.39763641357422 8.68167781829834 Metharme-ao-ts-sym-trits 6.471063137054443 49.650611877441406 7.969552040100098 Metharme-7b-4bit-32g-sym 6.518134117126465 49.66925811767578 8.13516616821289 Metharme-7b-4bit-128g-trits 8.68796443939209 49.73833465576172 8.2406587600708 Metharme-7b-4bit-32g 6.902080535888672 50.237754821777344 8.081602096557617 Metharme-7b-4bit-ts-128g-sym-trits 6.756448745727539 51.510311126708984 8.292160987854004 Metharme-7b-4bit-128g-sym-trits-damp0.1 6.585468769073486 51.55869674682617 8.2803316116333 Metharme-7b-4bit-ts-trits-damp0.1 6.631694793701172 51.7371711730957 8.656966209411621 Metharme-7b-4bit-ts-damp0.1 6.7783050537109375 51.81301498413086 8.621373176574707 Metharme-7b-4bit-128g-sym-damp0.1 6.379064083099365 52.17233657836914 8.316649436950684 Metharme-7b-4bit-ts-128g 9.475017547607422 52.358829498291016 8.340700149536133 Metharme-7b-4bit-ts-128g-sym-trits-damp0.1 6.225503921508789 53.12746047973633 8.240595817565918 Metharme-7b-4bit-ts-sym-damp0.1 6.7517595291137695 54.06147384643555 8.821818351745605 Metharme-7b-4bit-ts-sym 7.270263671875 54.35262680053711 8.787986755371094 Metharme-7b-4bit-ts-128g-sym-damp0.1 6.584965705871582 55.20026397705078 8.268644332885742 Metharme-7b-4bit-ts-128g-sym 6.819341659545898 55.28740310668945 8.377721786499023 Metharme-7b-4bit-trits 7.832409858703613 55.383026123046875 8.806737899780273 Metharme-trits 7.832409858703613 55.383026123046875 8.806737899780273 Metharme-7b-4bit-ts-sym-trits 7.599876403808594 55.75454330444336 8.932201385498047 Metharme-sym-damp0.1 6.896804332733154 57.4250602722168 8.703770637512207 Metharme-7b-4bit-alone 6.997134685516357 58.87525177001953 8.824191093444824 Metharme-sym-trits-damp0.1 7.075908184051514 59.13897705078125 8.919178009033203 Metharme-sym 7.585672855377197 61.01168441772461 9.032520294189453 Metharme-7b-4bit-ts 7.735969543457031 62.92238235473633 8.913650512695312 Metharme-sym-trits 7.494253635406494 63.320709228515625 8.969240188598633 Metharme-7b-4bit-ts-sym-trits-damp0.1 6.886506080627441 64.72743225097656 8.880627632141113 Metharme-7b-4bit-ts-trits 7.2306809425354 66.78710174560547 8.879831314086914 Metharme-7b-4bit-ao-128g 251321.265625 250117.859375 232929.234375 Metharme-7b-4bit-ao-32g 275425.5 267733.25 254506.71875
Benchmarks Sorted in Alphabetical Order
GPTQ Variation: Wikitext2 Ptb-New C4-New Metharme-7b-16bit 5.7208476066589355 41.61103439331055 7.512405872344971 Metharme-7b-4bit-128g-damp0.1 6.242006301879883 45.30938720703125 8.14249324798584 Metharme-7b-4bit-128g-sym-damp0.1 6.379064083099365 52.17233657836914 8.316649436950684 Metharme-7b-4bit-128g-sym-trits-damp0.1 6.585468769073486 51.55869674682617 8.2803316116333 Metharme-7b-4bit-128g-trits-damp0.1 6.262309551239014 47.80055618286133 8.192194938659668 Metharme-7b-4bit-128g 10.206376075744629 49.00401306152344 8.198845863342285 Metharme-7b-4bit-32g-damp0.1 6.107605934143066 47.91380310058594 8.068695068359375 Metharme-7b-4bit-32g-sym-damp0.1 6.566899299621582 48.0670166015625 8.095841407775879 Metharme-7b-4bit-32g-sym-trits-damp0.1 6.109438896179199 47.35737228393555 8.075060844421387 Metharme-7b-4bit-32g-trits-damp0.1 6.128157138824463 48.04175567626953 8.0708646774292 Metharme-7b-4bit-32g 6.902080535888672 50.237754821777344 8.081602096557617 Metharme-7b-4bit-alone 6.997134685516357 58.87525177001953 8.824191093444824 Metharme-7b-4bit-ao-128g 251321.265625 250117.859375 232929.234375 Metharme-7b-4bit-ao-32g 275425.5 267733.25 254506.71875 Metharme-7b-4bit-ao-damp0.1 6.220707893371582 47.82929611206055 7.884565353393555 Metharme-7b-4bit-ao-sym 6.477842807769775 48.53507614135742 7.993765354156494 Metharme-7b-4bit-ao 6.281311511993408 46.79158401489258 7.906069755554199 Metharme-7b-4bit-damp0.1 6.495014190673828 49.39763641357422 8.68167781829834 Metharme-7b-4bit-trits 7.832409858703613 55.383026123046875 8.806737899780273 Metharme-7b-4bit-ts-128g-damp0.1 6.17774772644043 46.47630310058594 8.20170783996582 Metharme-7b-4bit-ts-128g-sym-damp0.1 6.584965705871582 55.20026397705078 8.268644332885742 Metharme-7b-4bit-ts-128g-sym-trits-damp0.1 6.225503921508789 53.12746047973633 8.240595817565918 Metharme-7b-4bit-ts-128g-trits-damp0.1 6.289113521575928 46.06787109375 8.122251510620117 Metharme-7b-4bit-ts-128g 9.475017547607422 52.358829498291016 8.340700149536133 Metharme-7b-4bit-ts-32g-damp0.1 6.219024658203125 45.834869384765625 8.071272850036621 Metharme-7b-4bit-ts-32g-sym-damp0.1 6.274552822113037 47.35737228393555 8.06270980834961 Metharme-7b-4bit-ts-32g-sym-trits-damp0.1 6.266031265258789 47.346702575683594 8.068148612976074 Metharme-7b-4bit-ts-32g-trits-damp0.1 6.118431568145752 45.67333221435547 8.077078819274902 Metharme-7b-4bit-ts-32g 6.424218654632568 48.48588943481445 8.089512825012207 Metharme-7b-4bit-ts-damp0.1 6.7783050537109375 51.81301498413086 8.621373176574707 Metharme-7b-4bit-ts-sym-damp0.1 6.7517595291137695 54.06147384643555 8.821818351745605 Metharme-7b-4bit-ts-sym-trits-damp0.1 6.886506080627441 64.72743225097656 8.880627632141113 Metharme-7b-4bit-ts-trits-damp0.1 6.631694793701172 51.7371711730957 8.656966209411621 Metharme-7b-4bit-ts 7.735969543457031 62.92238235473633 8.913650512695312 Metharme-7b-4bit-128g-sym-trits 7.056288242340088 48.983768463134766 8.339276313781738 Metharme-7b-4bit-128g-sym 6.9575653076171875 49.356834411621094 8.35644817352295 Metharme-7b-4bit-128g-trits 8.68796443939209 49.73833465576172 8.2406587600708 Metharme-7b-4bit-32g-sym-trits 6.206963539123535 46.88833999633789 8.13610553741455 Metharme-7b-4bit-32g-sym 6.518134117126465 49.66925811767578 8.13516616821289 Metharme-7b-4bit-32g-trits 7.017086029052734 45.04129409790039 8.074845314025879 Metharme-7b-4bit-ts-128g-sym-trits 6.756448745727539 51.510311126708984 8.292160987854004 Metharme-7b-4bit-ts-128g-sym 6.819341659545898 55.28740310668945 8.377721786499023 Metharme-7b-4bit-ts-128g-trits 7.350858688354492 44.25314712524414 8.274221420288086 Metharme-7b-4bit-ts-32g-sym-trits 6.82415771484375 48.82029724121094 8.090987205505371 Metharme-7b-4bit-ts-32g-sym 6.387663841247559 48.07244110107422 8.173730850219727 Metharme-7b-4bit-ts-32g-trits 6.632943153381348 47.973228454589844 8.013848304748535 Metharme-7b-4bit-ts-sym-trits 7.599876403808594 55.75454330444336 8.932201385498047 Metharme-7b-4bit-ts-sym 7.270263671875 54.35262680053711 8.787986755371094 Metharme-7b-4bit-ts-trits 7.2306809425354 66.78710174560547 8.879831314086914 Metharme-ao-sym-trits-damp0.1 6.329384803771973 48.06882858276367 7.959168910980225 Metharme-ao-sym-trits 6.390106678009033 48.15375900268555 7.9804582595825195 Metharme-ao-sym 6.477842807769775 48.53507614135742 7.993765354156494 Metharme-ao-trits-damp0.1 6.163661956787109 46.50249099731445 7.877425193786621 Metharme-ao-trits 6.283935546875 46.57590103149414 7.907411575317383 Metharme-ao-ts-damp0.1 6.184001445770264 46.17180633544922 7.880400657653809 Metharme-ao-ts-sym-damp0.1 6.210323333740234 47.66789245605469 7.952476978302002 Metharme-ao-ts-sym-trits-damp0.1 6.232576370239258 48.081459045410156 7.95023250579834 Metharme-ao-ts-sym-trits 6.471063137054443 49.650611877441406 7.969552040100098 Metharme-ao-ts-sym 6.460526943206787 47.190460205078125 7.9732160568237305 Metharme-ao-ts-trits-damp0.1 6.172733783721924 45.19890213012695 7.872506141662598 Metharme-ao-ts-trits 6.310682773590088 46.4483757019043 7.898126602172852 Metharme-ao-ts 6.329496383666992 46.88129806518555 7.910323143005371 Metharme-sym-damp0.1 6.896804332733154 57.4250602722168 8.703770637512207 Metharme-sym-trits-damp0.1 7.075908184051514 59.13897705078125 8.919178009033203 Metharme-sym-trits 7.494253635406494 63.320709228515625 8.969240188598633 Metharme-sym 7.585672855377197 61.01168441772461 9.032520294189453 Metharme-trits 7.832409858703613 55.383026123046875 8.806737899780273
