You can deploy this model and start earning money today!

BikoRiko/
GPT-2.3

safetensors

gpt2

Views

No views yet

GPT-2.3

GPT-2.3 is an advanced fine-tuned version of the GPT-2 architecture, evolved directly from the Gpt-2.2 iteration.

Key Improvements over 2.2

Dataset Size: Trained on a 15% subset of Wikitext-2 (50% increase in training data volume).
Context Window: Upgraded to support up to 2048 tokens, allowing for much longer document generation and deeper context tracking.
Performance: Further optimized for causal language modeling with improved coherence.

Technical Specifications

Base Model: GPT-2 (small)
Training Epochs: 3
Max Sequence Length: 2048 Tokens
Framework: Hugging Face Transformers & PyTorch

Hardware Guidance

Inference: While it runs on a CPU, a GPU (e.g., Tesla T4) is highly recommended for generating long-form text (up to 2048 tokens) efficiently.

Usage Example

Users can load and run the model using the following code (note the ignore_mismatched_sizes flag):

python

from transformers import GPT2LMHeadModel, GPT2Tokenizer

repo_id = "BikoRiko/GPT-2.3"
# Note: ignore_mismatched_sizes=True is required because the context was expanded post-training
model = GPT2LMHeadModel.from_pretrained(repo_id, ignore_mismatched_sizes=True)
tokenizer = GPT2Tokenizer.from_pretrained(repo_id)

# Generate text
inputs = tokenizer("The future of technology is", return_tensors='pt')
outputs = model.generate(**inputs, max_length=150)
print(tokenizer.decode(outputs[0]))