Views
No views yet
1import torch
2from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
3
4base_model_id = "mychen76/biomistral_medqa_v1"
5bnb_config = BitsAndBytesConfig(
6 load_in_4bit=True,
7 bnb_4bit_use_double_quant=True,
8 bnb_4bit_quant_type="nf4",
9 bnb_4bit_compute_dtype=torch.bfloat16
10)
11
12model = AutoModelForCausalLM.from_pretrained(base_model_id, quantization_config=bnb_config)
13tokenizer = AutoTokenizer.from_pretrained(
14 base_model_id,
15 add_eos_token=True,
16 add_bos_token=True,
17)
18
19## Uses
20eval_prompt = """From the MedQuad MedicalQA Dataset: Given the following medical question and question type, provide an accurate answer:
### Question type:
information
### Question:
What are the genetic changes related to X-linked lymphoproliferative disease ?
### Answer:
"""
model_input = eval_tokenizer(eval_prompt, return_tensors="pt").to("cuda")
ft_model.eval()
with torch.no_grad():
print(eval_tokenizer.decode(ft_model.generate(**model_input, max_new_tokens=300)[0], skip_special_tokens=True))From the MedQuad MedicalQA Dataset: Given the following medical question and question type, provide an accurate answer:
### Question type:
information
### Question:
What are the genetic changes related to X-linked lymphoproliferative disease ?
### Answer:
X-linked lymphoproliferative disease (XLP) is a rare primary immunodeficiency syndrome. XLP is caused by mutations in SH2D1A gene, which encodes the cytoplasmic signaling protein SLAM-associated protein ( client protein-SLAM). SLAM is a member of the signaling lymphocytic activation molecule family of receptors, which are involved in the regulation of lymphocyte activation and proliferation. The SLAM receptor is expressed on the surface of B and T lymphocytes, natural killer cells, and monocytes. Mutations in SH2D1A gene lead to impaired signaling through the SLAM receptor, resulting in a deficiency in the activation and proliferation of B and T lymphocytes. This leads to a decrease in the number of B and T lymphocytes, resulting in a weakened immune response.eval_prompt = """From the MedQuad MedicalQA Dataset: Given the following medical question and question type, provide an accurate answer:
### Question type:
frequency
### Question:
How many people are affected by Smith-Lemli-Opitz syndrome ?
### Answer:
"""
model_input = eval_tokenizer(eval_prompt, return_tensors="pt").to("cuda")
ft_model.eval()
with torch.no_grad():
print(eval_tokenizer.decode(ft_model.generate(**model_input, max_new_tokens=300)[0], skip_special_tokens=True))From the MedQuad MedicalQA Dataset: Given the following medical question and question type, provide an accurate answer:
### Question type:
frequency
### Question:
How many people are affected by Smith-Lemli-Opitz syndrome ?
### Answer:
Smith-Lemli-Opitz syndrome (SLOS) is a rare autosomal recessive disorder of human development. It is characterized by a wide range of symptoms, including growth and developmental delay, intellectual disability, characteristic facial features, and congenital heart defects. The prevalence of SLOS is estimated to be 1 in 15,000 to 1 in 25,000 live births.eval_prompt = """From the MedQuad MedicalQA Dataset: Given the following medical question and question type, provide an accurate answer:
### Question type:
symptoms
### Question:
What are the symptoms of Norrie disease ?
### Answer:
"""
model_input = eval_tokenizer(eval_prompt, return_tensors="pt").to("cuda")
ft_model.eval()
with torch.no_grad():
print(eval_tokenizer.decode(ft_model.generate(**model_input, max_new_tokens=300)[0], skip_special_tokens=True))Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
From the MedQuad MedicalQA Dataset: Given the following medical question and question type, provide an accurate answer:
### Question type:
symptoms
### Question:
What are the symptoms of Norrie disease ?
### Answer:
Norrie disease is a rare, X-linked recessive disorder of the blood vessels. It is characterized by a variety of symptoms, including glaucoma, mental retardation, seizures, and deafness.