Roleplay Quantization in EXL2 format for Lumimaid 0.2 70B

• 2.0bpw8h quants (tested and barely working on one RTX 3090 at 32k context length)
• 2.2bpw8h quants
• 3.7bpw8h quants
• 3.75bpw8h quants (tested and working on dual RTX 3090s at 128k context length)
• 3.8bpw8h quants
• 4.0bpw8h quants (tested and working on dual RTX 3090s at 98k context length)
• 4.4bpw8h quants (tested and working on dual RTX 3090s at 64k context length)
• 6.0bpw8h quants
• 7.0bpw8h quants
• 8.0bpw8h quants

Original Model card

Lumimaid 0.2

This model is based on: Meta-Llama-3.1-70B-Instruct

Prompt template: Llama-3-Instruct

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>

{input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{output}<|eot_id|>

Credits:

• Undi
• IkariDev

Training data we used to make our dataset:

• Epiculous/Gnosis
• ChaoticNeutrals/Luminous_Opus
• ChaoticNeutrals/Synthetic-Dark-RP
• ChaoticNeutrals/Synthetic-RP
• Gryphe/Sonnet3.5-SlimOrcaDedupCleaned
• Gryphe/Opus-WritingPrompts
• meseca/writing-opus-6k
• meseca/opus-instruct-9k
• PJMixers/grimulkan_theory-of-mind-ShareGPT
• NobodyExistsOnTheInternet/ToxicQAFinal
• Undi95/toxic-dpo-v0.1-sharegpt
• cgato/SlimOrcaDedupCleaned
• kalomaze/Opus_Instruct_25k
• Doctor-Shotgun/no-robots-sharegpt
• Norquinal/claude_multiround_chat_30k
• nothingiisreal/Claude-3-Opus-Instruct-15K
• All the Aesirs dataset, cleaned, unslopped
• All le luminae dataset, cleaned, unslopped
• Small part of Airoboros reduced

• Opus_Instruct-v2-6.5K-Filtered-v2-sharegpt
• claude_sharegpt_trimmed
• CapybaraPure_Decontaminated-ShareGPT_reduced

Datasets credits:

• Epiculous
• ChaoticNeutrals
• Gryphe
• meseca
• PJMixers
• NobodyExistsOnTheInternet
• cgato
• kalomaze
• Doctor-Shotgun
• Norquinal
• nothingiisreal

Others