This modelcard aims to be a base template for new models. It has been generated using Baichuan-13B.

Use the code below to get started with the model.

MedDialog,cMedQA-v2,MedMCQA,DrugDB,Alpaca-GPT4-zh

MLEC-QA-Few-shot

MLEC-QA-Zero-shot

CMB

CMD

CMID

The overall architecture of MedLang-13B is based on the standard Transformer decoder, employing the same model design as LLaMA.
In terms of details, MedLang-13B adopts the following approaches:
(1) Using RoPE as the positional encoding, which is widely adopted in most models at this stage and exhibits excellent scalability;
(2) Setting the context window length to 4096;
(3) Utilizing SwiGLU as the activation function, which excels in handling complex semantic relationships and long-dependency issues in language modeling, hence widely used in large language models including MedLang-13B;
(4) Setting the hidden layer size to 11008 in the feedforward network;
(5) Adopting Pre-Normalization based on RMSNorm as layer normalization.

Although MedLang-13B performs well on both single-turn and multi-turn QA medical datasets, the high demands of reliability and safety in the medical field, especially concerning patient health and life, render the current MedLang-13B unsuitable for deployment in practical medical applications. I solemnly declare that this model is designed solely for the research and testing purposes of individual groups. Users are strongly advised to critically evaluate any information or medical advice provided by the model.