Tess-3-7B is a finetuned version of the Mistral-7B-v0.3 base model. This version is the first phase of the final Tess-3 model, and have been trained with supervised fine-tuning (SFT) on a curated dataset of ~500K samples. The total SFT dataset contains about 1B tokens.
This model has 32K context length.
Note that this model uses ChatML prompt format.
1import torch, json
2from transformers import AutoModelForCausalLM, AutoTokenizer
3from stop_word import StopWordCriteria
4
5model_path = "migtissera/Tess-3-7B-SFT"
6output_file_path = "/home/migel/conversations.jsonl"
7
8model = AutoModelForCausalLM.from_pretrained(
9 model_path,
10 torch_dtype=torch.float16,
11 device_map="auto",
12 load_in_4bit=False,
13 trust_remote_code=True,
14)
15
16tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
17
18terminators = [
19 tokenizer.convert_tokens_to_ids("<|im_end|>")
20]
21
22def generate_text(instruction):
23 tokens = tokenizer.encode(instruction)
24 tokens = torch.LongTensor(tokens).unsqueeze(0)
25 tokens = tokens.to("cuda")
26
27 instance = {
28 "input_ids": tokens,
29 "top_p": 1.0,
30 "temperature": 0.75,
31 "generate_len": 1024,
32 "top_k": 50,
33 }
34
35 length = len(tokens[0])
36 with torch.no_grad():
37 rest = model.generate(
38 input_ids=tokens,
39 max_length=length + instance["generate_len"],
40 use_cache=True,
41 do_sample=True,
42 top_p=instance["top_p"],
43 temperature=instance["temperature"],
44 top_k=instance["top_k"],
45 num_return_sequences=1,
46 pad_token_id=tokenizer.eos_token_id,
47 eos_token_id=terminators,
48 )
49 output = rest[0][length:]
50 string = tokenizer.decode(output, skip_special_tokens=True)
51 return f"{string}"
52
53conversation = f"""<|im_start|>system\nYou are Tesoro, a helful AI assitant. You always provide detailed answers without hesitation.<|im_end|>\n<|im_start|>user\n"""
54
55while True:
56 user_input = input("You: ")
57 llm_prompt = f"{conversation}{user_input}<|im_end|>\n<|im_start|>assistant\n"
58 answer = generate_text(llm_prompt)
59 print(answer)
60 conversation = f"{llm_prompt}{answer}\n"
61 json_data = {"prompt": user_input, "answer": answer}
62
63 with open(output_file_path, "a") as output_file:
64 output_file.write(json.dumps(json_data) + "\n")
While this model aims for accuracy, it can occasionally produce inaccurate or misleading results.
Despite diligent efforts in refining the pretraining data, there remains a possibility for the generation of inappropriate, biased, or offensive content.
Exercise caution and cross-check information when necessary. This is an uncensored model.
Detailed results can be found
here
