You can deploy this model and start earning money today!

RichardErkhov/
Vikhrmodels_-_Vikhr-7B-instruct_0.4-gguf

Name: Vikhrmodels_-_Vikhr-7B-instruct_0.4-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Vikhr-7B-instruct_0.4 - GGUF

Model creator: https://huggingface.co/Vikhrmodels/
Original model: https://huggingface.co/Vikhrmodels/Vikhr-7B-instruct_0.4/

Name	Quant method	Size
Vikhr-7B-instruct_0.4.Q2_K.gguf	Q2_K	2.74GB
Vikhr-7B-instruct_0.4.IQ3_XS.gguf	IQ3_XS	3.04GB
Vikhr-7B-instruct_0.4.IQ3_S.gguf	IQ3_S	3.19GB
Vikhr-7B-instruct_0.4.Q3_K_S.gguf	Q3_K_S	3.17GB
Vikhr-7B-instruct_0.4.IQ3_M.gguf	IQ3_M	3.29GB
Vikhr-7B-instruct_0.4.Q3_K.gguf	Q3_K	3.5GB
Vikhr-7B-instruct_0.4.Q3_K_M.gguf	Q3_K_M	3.5GB
Vikhr-7B-instruct_0.4.Q3_K_L.gguf	Q3_K_L	3.79GB
Vikhr-7B-instruct_0.4.IQ4_XS.gguf	IQ4_XS	3.92GB
Vikhr-7B-instruct_0.4.Q4_0.gguf	Q4_0	4.08GB
Vikhr-7B-instruct_0.4.IQ4_NL.gguf	IQ4_NL	4.12GB
Vikhr-7B-instruct_0.4.Q4_K_S.gguf	Q4_K_S	4.11GB
Vikhr-7B-instruct_0.4.Q4_K.gguf	Q4_K	4.32GB
Vikhr-7B-instruct_0.4.Q4_K_M.gguf	Q4_K_M	4.32GB
Vikhr-7B-instruct_0.4.Q4_1.gguf	Q4_1	4.5GB
Vikhr-7B-instruct_0.4.Q5_0.gguf	Q5_0	4.93GB
Vikhr-7B-instruct_0.4.Q5_K_S.gguf	Q5_K_S	4.93GB
Vikhr-7B-instruct_0.4.Q5_K.gguf	Q5_K	5.05GB
Vikhr-7B-instruct_0.4.Q5_K_M.gguf	Q5_K_M	5.05GB
Vikhr-7B-instruct_0.4.Q5_1.gguf	Q5_1	5.35GB
Vikhr-7B-instruct_0.4.Q6_K.gguf	Q6_K	5.83GB

Original model description:

library_name: transformers tags: []

Релиз вихря 0.3-0.4

Долили сильно больше данных в sft, теперь стабильнее работает json и multiturn, слегка подточили параметры претрена модели

collab

python

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model = AutoModelForCausalLM.from_pretrained("AlexWortega/v5-it",
                                             device_map="auto",
                                             attn_implementation="flash_attention_2",
                                             torch_dtype=torch.bfloat16)

tokenizer = AutoTokenizer.from_pretrained("AlexWortega/v5-it")
from transformers import  AutoTokenizer, pipeline
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
prompts = [
    "В чем разница между фруктом и овощем?",
    "Годы жизни колмагорова?"]

def test_inference(prompt):
    prompt = pipe.tokenizer.apply_chat_template([{"role": "user", "content": prompt}], tokenize=False, add_generation_prompt=True)
    print(prompt)
    outputs = pipe(prompt, max_new_tokens=512, do_sample=True, num_beams=1, temperature=0.25, top_k=50, top_p=0.98, eos_token_id=79097)
    return outputs[0]['generated_text'][len(prompt):].strip()


for prompt in prompts:
    print(f"    prompt:\n{prompt}")
    print(f"    response:\n{test_inference(prompt)}")
    print("-"*50)

RichardErkhov/Vikhrmodels_-_Vikhr-7B-instruct_0.4-gguf

Original model description:

library_name: transformers tags: []

Релиз вихря 0.3-0.4

RichardErkhov/
Vikhrmodels_-_Vikhr-7B-instruct_0.4-gguf