You can deploy this model and start earning money today!

RichardErkhov/
quantumaikr_-_quantum-dpo-v0.1-gguf

Name: quantumaikr_-_quantum-dpo-v0.1-gguf
Availability: InStock

endpoints_compatible

gguf

template

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

quantum-dpo-v0.1 - GGUF

Model creator: https://huggingface.co/quantumaikr/
Original model: https://huggingface.co/quantumaikr/quantum-dpo-v0.1/

Name	Quant method	Size
quantum-dpo-v0.1.Q2_K.gguf	Q2_K	2.53GB
quantum-dpo-v0.1.IQ3_XS.gguf	IQ3_XS	2.81GB
quantum-dpo-v0.1.IQ3_S.gguf	IQ3_S	2.96GB
quantum-dpo-v0.1.Q3_K_S.gguf	Q3_K_S	2.95GB
quantum-dpo-v0.1.IQ3_M.gguf	IQ3_M	3.06GB
quantum-dpo-v0.1.Q3_K.gguf	Q3_K	3.28GB
quantum-dpo-v0.1.Q3_K_M.gguf	Q3_K_M	3.28GB
quantum-dpo-v0.1.Q3_K_L.gguf	Q3_K_L	3.56GB
quantum-dpo-v0.1.IQ4_XS.gguf	IQ4_XS	3.67GB
quantum-dpo-v0.1.Q4_0.gguf	Q4_0	3.83GB
quantum-dpo-v0.1.IQ4_NL.gguf	IQ4_NL	3.87GB
quantum-dpo-v0.1.Q4_K_S.gguf	Q4_K_S	3.86GB
quantum-dpo-v0.1.Q4_K.gguf	Q4_K	4.07GB
quantum-dpo-v0.1.Q4_K_M.gguf	Q4_K_M	4.07GB
quantum-dpo-v0.1.Q4_1.gguf	Q4_1	4.24GB
quantum-dpo-v0.1.Q5_0.gguf	Q5_0	4.65GB
quantum-dpo-v0.1.Q5_K_S.gguf	Q5_K_S	4.65GB
quantum-dpo-v0.1.Q5_K.gguf	Q5_K	4.78GB
quantum-dpo-v0.1.Q5_K_M.gguf	Q5_K_M	4.78GB
quantum-dpo-v0.1.Q5_1.gguf	Q5_1	5.07GB
quantum-dpo-v0.1.Q6_K.gguf	Q6_K	5.53GB
quantum-dpo-v0.1.Q8_0.gguf	Q8_0	7.17GB

Original model description:

license: cc-by-nc-4.0 language:

en pipeline_tag: text-generation

quantumaikr/quantum-dpo-v0.1

Usage

Start chatting with quantumaikr/quantum-dpo-v0.1 using the following code snippet:

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

tokenizer = AutoTokenizer.from_pretrained("quantumaikr/quantum-dpo-v0.1")
model = AutoModelForCausalLM.from_pretrained("quantumaikr/quantum-dpo-v0.1", torch_dtype=torch.float16, device_map="auto")

system_prompt = "You are QuantumLM, an AI that follows instructions extremely well. Help as much as you can. Remember, be safe, and don't do anything illegal."

message = "Write me a poem please"
prompt = f"[INST] <<SYS>>\n{system_prompt}\n<</SYS>>\n\n{message}[/INST]"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
output = model.generate(**inputs, do_sample=True, temperature=0.7, top_p=0.95, top_k=30, max_new_tokens=2048)

print(tokenizer.decode(output[0], skip_special_tokens=True))

QuantumLM should be used with this prompt format:

### System:
This is a system prompt, please behave and help the user.

### User:
Your prompt here

### Assistant
The output of QuantumLM

Use and Limitations

Intended Use

These models are intended for research only, in adherence with the CC BY-NC-4.0 license.

Limitations and bias

Although the aforementioned dataset helps to steer the base language models into "safer" distributions of text, not all biases and toxicity can be mitigated through fine-tuning. We ask that users be mindful of such potential issues that can arise in generated responses. Do not treat model outputs as substitutes for human judgment or as sources of truth. Please use it responsibly.

RichardErkhov/quantumaikr_-_quantum-dpo-v0.1-gguf