You can deploy this model and start earning money today!

BikoRiko/
GPT-2.3-High

Name: GPT-2.3-High
Availability: InStock

lilacai/lilac-wikitext-2-raw-v1

text-generation-inference

endpoints_compatible

Views

No views yet

GPT-2.3-High

GPT-2.3-High is the definitive fine-tuned iteration of the GPT-2 architecture in this series, specifically optimized for high-coherence long-form text generation.

Technical Specifications

Model Name: GPT-2.3-High
Base Architecture: GPT-2 (Small)
Total Parameters: 125,226,240 (~124 Million)
Context Window: Upgraded to 2048 tokens
Training Dataset: Wikitext-2-raw-v1 (20% subset)
Training Epochs: 3
Framework: PyTorch & Hugging Face Transformers

Accuracy & Evaluation

Following the 'healing' fine-tune, the model was evaluated on the official unseen test split of the Wikitext-2 dataset.

Test Set Perplexity (PPL): 4.06
Training Set Perplexity (PPL): 2.25
IF your going to test it use the dataset i trained it on witch is Wikitext-2-raw-v1

Overview & Capabilities

GPT-2.3-High was developed to solve the 'word salad' and repetition issues found in previous 2048-token iterations. By performing a 'healing' fine-tune on a larger dataset slice (20%), the model learned to manage the expanded positional embeddings effectively.

Usage Instructions

To use GPT-2.3-High, ensure you have the transformers library installed. Due to the manual context expansion, the ignore_mismatched_sizes=True flag is required during loading.

python

from transformers import GPT2LMHeadModel, GPT2Tokenizer

repo_id = "BikoRiko/GPT-2.3-High"

model = GPT2LMHeadModel.from_pretrained(repo_id, ignore_mismatched_sizes=True)
tokenizer = GPT2Tokenizer.from_pretrained(repo_id)