GPT-Fem-Compact is a version of GPT-Compact finetuned on 18 GB of text on and about women.
Model Description
GPT-Fem-Compact has 16 heads, 24 layers, and 1280 embeddings, similar to GPT2-Large, with a context window of 16,384 tokens. This maximized GPU VRAM while balancing size and functionality.
It was trained on a diverse corpus of over 17 billion tokens.
