You can deploy this model and start earning money today!

RichardErkhov/
willnguyen_-_lacda-2-7B-chat-v0.1-gguf

Name: willnguyen_-_lacda-2-7B-chat-v0.1-gguf
Availability: InStock

endpoints_compatible

gguf

template

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

lacda-2-7B-chat-v0.1 - GGUF

Model creator: https://huggingface.co/willnguyen/
Original model: https://huggingface.co/willnguyen/lacda-2-7B-chat-v0.1/

Name	Quant method	Size
lacda-2-7B-chat-v0.1.Q2_K.gguf	Q2_K	2.36GB
lacda-2-7B-chat-v0.1.IQ3_XS.gguf	IQ3_XS	2.6GB
lacda-2-7B-chat-v0.1.IQ3_S.gguf	IQ3_S	2.75GB
lacda-2-7B-chat-v0.1.Q3_K_S.gguf	Q3_K_S	2.75GB
lacda-2-7B-chat-v0.1.IQ3_M.gguf	IQ3_M	2.9GB
lacda-2-7B-chat-v0.1.Q3_K.gguf	Q3_K	3.07GB
lacda-2-7B-chat-v0.1.Q3_K_M.gguf	Q3_K_M	3.07GB
lacda-2-7B-chat-v0.1.Q3_K_L.gguf	Q3_K_L	3.35GB
lacda-2-7B-chat-v0.1.IQ4_XS.gguf	IQ4_XS	3.4GB
lacda-2-7B-chat-v0.1.Q4_0.gguf	Q4_0	3.56GB
lacda-2-7B-chat-v0.1.IQ4_NL.gguf	IQ4_NL	3.58GB
lacda-2-7B-chat-v0.1.Q4_K_S.gguf	Q4_K_S	3.59GB
lacda-2-7B-chat-v0.1.Q4_K.gguf	Q4_K	3.8GB
lacda-2-7B-chat-v0.1.Q4_K_M.gguf	Q4_K_M	3.8GB
lacda-2-7B-chat-v0.1.Q4_1.gguf	Q4_1	3.95GB
lacda-2-7B-chat-v0.1.Q5_0.gguf	Q5_0	4.33GB
lacda-2-7B-chat-v0.1.Q5_K_S.gguf	Q5_K_S	4.33GB
lacda-2-7B-chat-v0.1.Q5_K.gguf	Q5_K	4.45GB
lacda-2-7B-chat-v0.1.Q5_K_M.gguf	Q5_K_M	4.45GB
lacda-2-7B-chat-v0.1.Q5_1.gguf	Q5_1	4.72GB
lacda-2-7B-chat-v0.1.Q6_K.gguf	Q6_K	5.15GB
lacda-2-7B-chat-v0.1.Q8_0.gguf	Q8_0	6.67GB

Original model description:

language:

en
vi license: llama2 datasets:
timdettmers/openassistant-guanaco model_name: LacDa2 7B inference: true model_creator: Will Nguyen model_link: https://huggingface.co/willnguyen/lacda-2-7B-chat-v0.1 model_type: llama base_model: meta-llama/llama-2-7b-hf

LacDa2 Model Card Readme

Model Information

Model Name: LacDa

Description: LacDa is a specialized language model that has been fine-tuned from the LLama2 model. It is designed to provide advanced natural language processing capabilities in specific domains or applications.

Fine-tuned from: LLama2

GitHub

Open LLM Leaderboard Evaluation Results

Metric	Value
Avg.	43.91
ARC (25-shot)	53.07
HellaSwag (10-shot)	77.57
MMLU (5-shot)	46.03
TruthfulQA (0-shot)	44.57
Winogrande (5-shot)	74.19
GSM8K (5-shot)	6.29
DROP (3-shot)	5.65

Instruction format

python

from transformers import  AutoModelForCausalLM, LlamaTokenizer, BitsAndBytesConfig, TextStreamer, StoppingCriteria, StoppingCriteriaList
import torch

class StopTokenCriteria(StoppingCriteria):
    def __init__(self, stop_tokens, tokenizer, prompt_length):
        self.stop_tokens = stop_tokens
        if tokenizer.pad_token not in stop_tokens:
            self.stop_tokens.append(tokenizer.pad_token)
        if tokenizer.bos_token not in stop_tokens:
            self.stop_tokens.append(tokenizer.bos_token)
        if tokenizer.eos_token not in stop_tokens:
            self.stop_tokens.append(tokenizer.eos_token)
        self.tokenizer = tokenizer
        self.prompt_length = prompt_length

    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -> bool:
        is_done = False
        tokens = tokenizer.decode(input_ids[0])[self.prompt_length:]
        for st in self.stop_tokens:
            if st in tokens:
                is_done = True
                break
        return is_done

model_name = "willnguyen/lacda-2-7B-chat-v0.1"
tokenizer = LlamaTokenizer.from_pretrained(
        model_name,
        use_fast=False,
        padding_side="right",
        tokenizer_type='llama',
)
tokenizer.pad_token_id = 0

model = AutoModelForCausalLM.from_pretrained(
    model_name,
     device_map="auto",
    torch_dtype=torch.float16,
)

prompt = "<s> [INST] who is Hồ Chí Minh [/INST]"

stopping_criteria = StoppingCriteriaList([StopTokenCriteria(["[INST]", "[/INST]"], tokenizer, len(prompt))])
with torch.inference_mode():
    input_ids = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).input_ids.to('cuda')
    streamer = TextStreamer(tokenizer)
    _ = model.generate(
        input_ids=input_ids,
        max_new_tokens=1024,
        do_sample=False,
        temperature=1.0,
        top_p=1.0,
        top_k=50,
        repetition_penalty=1.0,
        use_cache=True,
        streamer=streamer,
        stopping_criteria=stopping_criteria
    )

RichardErkhov/willnguyen_-_lacda-2-7B-chat-v0.1-gguf