You can deploy this model and start earning money today!

RichardErkhov/
vvsotnikov_-_stablelm-tuned-alpha-3b-16bit-gguf

Name: vvsotnikov_-_stablelm-tuned-alpha-3b-16bit-gguf
Availability: InStock

endpoints_compatible

gguf

template

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

stablelm-tuned-alpha-3b-16bit - GGUF

Model creator: https://huggingface.co/vvsotnikov/
Original model: https://huggingface.co/vvsotnikov/stablelm-tuned-alpha-3b-16bit/

Name	Quant method	Size
stablelm-tuned-alpha-3b-16bit.Q2_K.gguf	Q2_K	1.34GB
stablelm-tuned-alpha-3b-16bit.Q3_K_S.gguf	Q3_K_S	1.54GB
stablelm-tuned-alpha-3b-16bit.Q3_K.gguf	Q3_K	1.81GB
stablelm-tuned-alpha-3b-16bit.Q3_K_M.gguf	Q3_K_M	1.81GB
stablelm-tuned-alpha-3b-16bit.Q3_K_L.gguf	Q3_K_L	1.96GB
stablelm-tuned-alpha-3b-16bit.IQ4_XS.gguf	IQ4_XS	1.88GB
stablelm-tuned-alpha-3b-16bit.Q4_0.gguf	Q4_0	1.96GB
stablelm-tuned-alpha-3b-16bit.IQ4_NL.gguf	IQ4_NL	1.98GB
stablelm-tuned-alpha-3b-16bit.Q4_K_S.gguf	Q4_K_S	1.98GB
stablelm-tuned-alpha-3b-16bit.Q4_K.gguf	Q4_K	2.18GB
stablelm-tuned-alpha-3b-16bit.Q4_K_M.gguf	Q4_K_M	2.18GB
stablelm-tuned-alpha-3b-16bit.Q4_1.gguf	Q4_1	2.16GB
stablelm-tuned-alpha-3b-16bit.Q5_0.gguf	Q5_0	2.36GB
stablelm-tuned-alpha-3b-16bit.Q5_K_S.gguf	Q5_K_S	2.36GB
stablelm-tuned-alpha-3b-16bit.Q5_K.gguf	Q5_K	2.52GB
stablelm-tuned-alpha-3b-16bit.Q5_K_M.gguf	Q5_K_M	2.52GB
stablelm-tuned-alpha-3b-16bit.Q5_1.gguf	Q5_1	2.56GB
stablelm-tuned-alpha-3b-16bit.Q6_K.gguf	Q6_K	2.78GB
stablelm-tuned-alpha-3b-16bit.Q8_0.gguf	Q8_0	3.6GB

Original model description:

language:

en tags:
causal-lm license: cc-by-nc-sa-4.0 datasets:
dmayhem93/ChatCombined
tatsu-lab/alpaca
nomic-ai/gpt4all_prompt_generations
Dahoas/full-hh-rlhf
jeffwan/sharegpt_vicuna
HuggingFaceH4/databricks_dolly_15k

StableLM-Tuned-Alpha 16-bit

Model Description

16-bit version of StableLM-Tuned-Alpha compressed for the sake of speed and memory usage. No other changes were made. Original model: https://huggingface.co/stabilityai/stablelm-tuned-alpha-3b

Usage

Get started chatting with StableLM-Tuned-Alpha 16-bit by using the following code snippet:

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, StoppingCriteria, StoppingCriteriaList
tokenizer = AutoTokenizer.from_pretrained("vvsotnikov/stablelm-tuned-alpha-3b-16bit")
model = AutoModelForCausalLM.from_pretrained("vvsotnikov/stablelm-tuned-alpha-3b-16bit", torch_dtype=torch.float16)
model.cuda()
class StopOnTokens(StoppingCriteria):
    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -> bool:
        stop_ids = [50278, 50279, 50277, 1, 0]
        for stop_id in stop_ids:
            if input_ids[0][-1] == stop_id:
                return True
        return False
system_prompt = """<|SYSTEM|># StableLM Tuned (Alpha version)
- StableLM is a helpful and harmless open-source AI language model developed by StabilityAI.
- StableLM is excited to be able to help the user, but will refuse to do anything that could be considered harmful to the user.
- StableLM is more than just an information source, StableLM is also able to write poetry, short stories, and make jokes.
- StableLM will refuse to participate in anything that could harm a human.
"""
prompt = f"{system_prompt}<|USER|>What's your mood today?<|ASSISTANT|>"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
tokens = model.generate(
  **inputs,
  max_new_tokens=64,
  temperature=0.7,
  do_sample=True,
  stopping_criteria=StoppingCriteriaList([StopOnTokens()])
)
print(tokenizer.decode(tokens[0], skip_special_tokens=True))

RichardErkhov/vvsotnikov_-_stablelm-tuned-alpha-3b-16bit-gguf

Original model description:

StableLM-Tuned-Alpha 16-bit

Model Description

Usage

RichardErkhov/
vvsotnikov_-_stablelm-tuned-alpha-3b-16bit-gguf