You can deploy this model and start earning money today!

RichardErkhov/
g-ronimo_-_phi-2-OpenHermes-2.5-gguf

Name: g-ronimo_-_phi-2-OpenHermes-2.5-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

phi-2-OpenHermes-2.5 - GGUF

Model creator: https://huggingface.co/g-ronimo/
Original model: https://huggingface.co/g-ronimo/phi-2-OpenHermes-2.5/

Name	Quant method	Size
phi-2-OpenHermes-2.5.Q2_K.gguf	Q2_K	1.03GB
phi-2-OpenHermes-2.5.IQ3_XS.gguf	IQ3_XS	1.12GB
phi-2-OpenHermes-2.5.IQ3_S.gguf	IQ3_S	1.16GB
phi-2-OpenHermes-2.5.Q3_K_S.gguf	Q3_K_S	1.16GB
phi-2-OpenHermes-2.5.IQ3_M.gguf	IQ3_M	1.23GB
phi-2-OpenHermes-2.5.Q3_K.gguf	Q3_K	1.33GB
phi-2-OpenHermes-2.5.Q3_K_M.gguf	Q3_K_M	1.33GB
phi-2-OpenHermes-2.5.Q3_K_L.gguf	Q3_K_L	1.47GB
phi-2-OpenHermes-2.5.IQ4_XS.gguf	IQ4_XS	1.43GB
phi-2-OpenHermes-2.5.Q4_0.gguf	Q4_0	1.49GB
phi-2-OpenHermes-2.5.IQ4_NL.gguf	IQ4_NL	1.5GB
phi-2-OpenHermes-2.5.Q4_K_S.gguf	Q4_K_S	1.51GB
phi-2-OpenHermes-2.5.Q4_K.gguf	Q4_K	1.62GB
phi-2-OpenHermes-2.5.Q4_K_M.gguf	Q4_K_M	1.62GB
phi-2-OpenHermes-2.5.Q4_1.gguf	Q4_1	1.65GB
phi-2-OpenHermes-2.5.Q5_0.gguf	Q5_0	1.8GB
phi-2-OpenHermes-2.5.Q5_K_S.gguf	Q5_K_S	1.8GB
phi-2-OpenHermes-2.5.Q5_K.gguf	Q5_K	1.87GB
phi-2-OpenHermes-2.5.Q5_K_M.gguf	Q5_K_M	1.87GB
phi-2-OpenHermes-2.5.Q5_1.gguf	Q5_1	1.95GB
phi-2-OpenHermes-2.5.Q6_K.gguf	Q6_K	2.13GB
phi-2-OpenHermes-2.5.Q8_0.gguf	Q8_0	2.75GB

Original model description:

library_name: transformers license: mit datasets:

teknium/OpenHermes-2.5

microsoft/phi-2 + teknium/OpenHermes-2.5

Training

QLoRA rank 32, LR 2e-5, 1 epoch
effective batch size: 200
max. seq. length: 1024 tokens
code in code/

Evals

Model	AGIEval	GPT4All	TruthfulQA	Bigbench	Average
g-ronimo/phi-2-OpenHermes-2.5	30.27	71.18	43.87	35.9	45.3
minghaowu/phi-2-OpenHermes-2.5	27.95	67.55	48.07	36.17	44.94
phi-2	27.96	70.84	44.46	35.17	44.61

Inference

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

modelpath="g-ronimo/phi-2-OpenHermes-2.5"

model = AutoModelForCausalLM.from_pretrained(
    modelpath,    
    torch_dtype=torch.bfloat16,
    device_map="auto",
    # attn_implementation="flash_attention_2",
)
tokenizer = AutoTokenizer.from_pretrained(modelpath) 

messages = [
    {"role": "system", "content": "answer like a pirate"},
    {"role": "user", "content": "what does it mean to be successful?"},
]
        
input_tokens = tokenizer.apply_chat_template(
    messages, 
    add_generation_prompt=True,
    return_tensors="pt"
).to("cuda")
output_tokens = model.generate(input_tokens, max_new_tokens=500)
output = tokenizer.decode(output_tokens[0])

print(output)

Ahoy there, matey! To me, being successful means having the wind in your sails and reaching the treasure you've been dreaming of. It's about setting sail on a journey with clear goals, working hard, facing challenges head-on, and never losing sight of what truly matters. So, set your compass right, hoist your Jolly Roger high, and let's embark on this adventure together! ⚓️💰⛵️

RichardErkhov/g-ronimo_-_phi-2-OpenHermes-2.5-gguf