You can deploy this model and start earning money today!

RichardErkhov/
migtissera_-_Tess-3-7B-SFT-gguf

Name: migtissera_-_Tess-3-7B-SFT-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Tess-3-7B-SFT - GGUF

Model creator: https://huggingface.co/migtissera/
Original model: https://huggingface.co/migtissera/Tess-3-7B-SFT/

Name	Quant method	Size
Tess-3-7B-SFT.Q2_K.gguf	Q2_K	2.54GB
Tess-3-7B-SFT.IQ3_XS.gguf	IQ3_XS	2.82GB
Tess-3-7B-SFT.IQ3_S.gguf	IQ3_S	2.97GB
Tess-3-7B-SFT.Q3_K_S.gguf	Q3_K_S	2.95GB
Tess-3-7B-SFT.IQ3_M.gguf	IQ3_M	3.06GB
Tess-3-7B-SFT.Q3_K.gguf	Q3_K	3.28GB
Tess-3-7B-SFT.Q3_K_M.gguf	Q3_K_M	3.28GB
Tess-3-7B-SFT.Q3_K_L.gguf	Q3_K_L	3.56GB
Tess-3-7B-SFT.IQ4_XS.gguf	IQ4_XS	3.68GB
Tess-3-7B-SFT.Q4_0.gguf	Q4_0	3.83GB
Tess-3-7B-SFT.IQ4_NL.gguf	IQ4_NL	3.87GB
Tess-3-7B-SFT.Q4_K_S.gguf	Q4_K_S	3.86GB
Tess-3-7B-SFT.Q4_K.gguf	Q4_K	4.07GB
Tess-3-7B-SFT.Q4_K_M.gguf	Q4_K_M	4.07GB
Tess-3-7B-SFT.Q4_1.gguf	Q4_1	4.24GB
Tess-3-7B-SFT.Q5_0.gguf	Q5_0	4.66GB
Tess-3-7B-SFT.Q5_K_S.gguf	Q5_K_S	4.66GB
Tess-3-7B-SFT.Q5_K.gguf	Q5_K	4.78GB
Tess-3-7B-SFT.Q5_K_M.gguf	Q5_K_M	4.78GB
Tess-3-7B-SFT.Q5_1.gguf	Q5_1	5.07GB
Tess-3-7B-SFT.Q6_K.gguf	Q6_K	5.54GB
Tess-3-7B-SFT.Q8_0.gguf	Q8_0	7.17GB

Original model description:

license: apache-2.0 tags:

generated_from_trainer base_model: mistralai/Mistral-7B-v0.3 model-index:
name: home/migel/tess-2.5-mistral-7B-phase-1 results: []

See axolotl config

axolotl version: 0.4.1

yaml

base_model: mistralai/Mistral-7B-v0.3
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer
tokenizer_use_fast: false

load_in_8bit: false
load_in_4bit: false
strict: false
model_config:

datasets:
  - path: /home/migel/ai_datasets/tess-v1.5b-chatml.jsonl
    type: sharegpt
    conversation: chatml
  - path: /home/migel/ai_datasets/Tess-3.0/Tess-3.0-multi_turn_chatml.jsonl
    type: sharegpt
    conversation: chatml
  - path: /home/migel/ai_datasets/Tess-3.0/Tess-3.0-single_turn_chatml.jsonl
    type: sharegpt
    conversation: chatml

chat_template: chatml

dataset_prepared_path: last_run_prepared_mistral
val_set_size: 0.0
output_dir: /home/migel/tess-2.5-mistral-7B-phase-1

resume_from_checkpoint: /home/migel/tess-2.5-mistral-7B-phase-1/checkpoint-440 
auto_resume_from_checkpoints: true

sequence_len: 16384
sample_packing: true
pad_to_sequence_len: true

gradient_accumulation_steps: 4
micro_batch_size: 4
num_epochs: 1
logging_steps: 1
optimizer: adamw_8bit
lr_scheduler: constant
learning_rate: 1e-6

wandb_project: mistral-7b
wandb_watch:
wandb_run_id:
wandb_log_model:

train_on_inputs: false
group_by_length: false
bf16: auto
fp16:
tf32: false

gradient_checkpointing: true
gradient_checkpointing_kwargs:
  use_reentrant: false
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true
saves_per_epoch: 10
evals_per_epoch: 10
save_total_limit: 3
save_steps:
eval_sample_packing: false
debug:
deepspeed: /home/migel/axolotl/deepspeed_configs/zero3_bf16.json
weight_decay: 0.0
fsdp:
fsdp_config:
special_tokens:
  bos_token: "<|im_start|>"
  eos_token: "<|im_end|>"
  pad_token: "<|end_of_text|>"

Tess-3-7B-SFT

Tess-3-7B is a finetuned version of the Mistral-7B-v0.3 base model. This version is the first phase of the final Tess-3 model, and have been trained with supervised fine-tuning (SFT) on a curated dataset of ~500K samples. The total SFT dataset contains about 1B tokens.

This model has 32K context length.

Sample code to run inference

Note that this model uses ChatML prompt format.

python

import torch, json
from transformers import AutoModelForCausalLM, AutoTokenizer
from stop_word import StopWordCriteria

model_path = "migtissera/Tess-3-7B-SFT"
output_file_path = "/home/migel/conversations.jsonl"

model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype=torch.float16,
    device_map="auto",
    load_in_4bit=False,
    trust_remote_code=True,
)

tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)

terminators = [
    tokenizer.convert_tokens_to_ids("<|im_end|>")
]

def generate_text(instruction):
    tokens = tokenizer.encode(instruction)
    tokens = torch.LongTensor(tokens).unsqueeze(0)
    tokens = tokens.to("cuda")

    instance = {
        "input_ids": tokens,
        "top_p": 1.0,
        "temperature": 0.75,
        "generate_len": 1024,
        "top_k": 50,
    }

    length = len(tokens[0])
    with torch.no_grad():
        rest = model.generate(
            input_ids=tokens,
            max_length=length + instance["generate_len"],
            use_cache=True,
            do_sample=True,
            top_p=instance["top_p"],
            temperature=instance["temperature"],
            top_k=instance["top_k"],
            num_return_sequences=1,
            pad_token_id=tokenizer.eos_token_id,
            eos_token_id=terminators,
        )
    output = rest[0][length:]
    string = tokenizer.decode(output, skip_special_tokens=True)
    return f"{string}"

conversation = f"""<|im_start|>system\nYou are Tesoro, a helful AI assitant. You always provide detailed answers without hesitation.<|im_end|>\n<|im_start|>user\n"""

while True:
    user_input = input("You: ")
    llm_prompt = f"{conversation}{user_input}<|im_end|>\n<|im_start|>assistant\n"
    answer = generate_text(llm_prompt)
    print(answer)
    conversation = f"{llm_prompt}{answer}\n"
    json_data = {"prompt": user_input, "answer": answer}

    with open(output_file_path, "a") as output_file:
        output_file.write(json.dumps(json_data) + "\n")

Join My General AI Discord (NeuroLattice):

https://discord.gg/Hz6GrwGFKD

Limitations & Biases:

While this model aims for accuracy, it can occasionally produce inaccurate or misleading results.

Despite diligent efforts in refining the pretraining data, there remains a possibility for the generation of inappropriate, biased, or offensive content.

Exercise caution and cross-check information when necessary. This is an uncensored model.

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	17.10
IFEval (0-Shot)	39.46
BBH (3-Shot)	24.12
MATH Lvl 5 (4-Shot)	3.32
GPQA (0-shot)	2.80
MuSR (0-shot)	10.28
MMLU-PRO (5-shot)	22.60

RichardErkhov/migtissera_-_Tess-3-7B-SFT-gguf