This model is fine-tuned from gemma-2-2b-it using a custom dataset created by our team, providing a concise overview of its key details and purpose.
Women's health is a critical and often underserved topic, with limited accessible resources where women can ask questions and receive reliable answers about their health. Recognizing this gap, we embarked on this project to develop a compact language model (LLM) tailored to address these needs. Our goal is to create a model that not only provides valuable insights but also enables efficient on-device inference, ensuring greater accessibility and usability for women worldwide.
1
2from unsloth import FastLanguageModel
3import torch
4from transformers import TextStreamer
5
6
7max_seq_length = 2048
8dtype = None
9load_in_4bit = False
10lora_path = "altaidevorg/gemma-women-health-checkpoint-1292"
11use_streamer = False
12
13model, tokenizer = FastLanguageModel.from_pretrained(
14 lora_path,
15 max_seq_length=max_seq_length,
16 dtype=dtype,
17 load_in_4bit=load_in_4bit,
18)
19FastLanguageModel.for_inference(model)
20
21text_streamer = TextStreamer(tokenizer, skip_prompt = True)
22
23
24messages = [
25 {"role": "user", "content": user_prompt},
26]
27
28
29input_ids = tokenizer.apply_chat_template(
30 messages,
31 tokenize = True,
32 add_generation_prompt = True,
33 return_tensors = "pt",
34).cuda()
35
36terminators = [tokenizer.eos_token_id, tokenizer.convert_tokens_to_ids("<end_of_turn>")]
37
38outputs = model.generate(input_ids = input_ids,
39 streamer = text_streamer if use_streamer else None,
40 max_new_tokens = 1024,
41 eos_token_id=terminators,
42 use_cache=True, do_sample=True, temperature=0.6, top_p=0.9)
43if not use_streamer:
44 out = outputs[0][input_ids.shape[-1]:]
45 generated_text = tokenizer.decode(out, skip_special_tokens=True)
46 print(generated_text)
47
48
The dataset was prepared through a comprehensive process based on our synthetic dataset generation method. Health-related websites were scraped, open-source e-books and PDFs focusing on women's health were collected, and an instruction dataset was created from these sources. To generate high-quality questions, we utilized a technique that involves role-playing between two LLMs and contextually-augmented by RAG.
The dataset will be made publicly available through its dedicated repository
altaidevorg/women-health-mini. Additionally, the code used for dataset generation will be released in the future to promote transparency and enable reproducibility.
During testing, we observed that the LoRA checkpoint performed better in evaluations compared to the version where the LoRA checkpoint was
merged with the base model. Interestingly, the standalone LoRA checkpoint consistently delivered superior results, though we currently lack a concrete explanation for this phenomenon.
We are actively investigating the underlying cause, with our best hypothesis being that the merging process may introduce some form of precision loss. Further research is underway to validate this theory and optimize the performance.
Any output of this model should never be taken as a medical advice, and it is mainly intended for research purposes.
