Made with Exllamav2 0.2.3 with the default dataset. This model needs software with Exllamav2 library such as Text-Generation-WebUI, TabbyAPI, etc.
This model has to fit your GPU to be usable and it's mainly meant for RTX cards on Windows/Linux or AMD on Linux.
For computers with incompatible hardware it's better to use GGUF versions of the model, you can find them here.
Original model card
Democratizing Medical LLMs For Much More Languages
Covering 12 Major Languages including English, Chinese, French, Hindi, Spanish, Arabic, Russian, Japanese, Korean, German, Italian, Portuguese and 38 Minor Languages So far.
Prepare train data for specific model (Create tokenized data in advance):
You can adjust data Training order and Training Epoch in this step
bash 2.data_process_train.sh
Train the model
If you want to train in Multi Nodes please refer to ./src/sft/training_config/zero_multi.yaml
bash 3.single_node_train.sh
Evaluate your model: Generate score for benchmark
bash 4.eval.sh
Citation
Please use the following citation if you intend to use our dataset for training or evaluation:
@misc{zheng2024efficientlydemocratizingmedicalllms,
title={Efficiently Democratizing Medical LLMs for 50 Languages via a Mixture of Language Family Experts},
author={Guorui Zheng and Xidong Wang and Juhao Liang and Nuo Chen and Yuping Zheng and Benyou Wang},
year={2024},
eprint={2410.10626},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2410.10626},
}
