from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
Set device
device = "cuda" if torch.cuda.is_available() else "cpu"
Model and tokenizer
model_name = "AventIQ-AI/gpt2-lmheadmodel-next-line-prediction-model"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name).to(device)
import html
Define test text
sample_text = "Artificial intelligence is transforming"
Tokenize input
inputs = tokenizer(sample_text, return_tensors="pt").to(device)
Generate prediction
with torch.no_grad():
output_tokens = model.generate(
**inputs,
max_length=50,
num_beams=5,
repetition_penalty=1.5,
temperature=0.7,
top_k=50,
top_p=0.9,
do_sample=True,
no_repeat_ngram_size=2,
num_return_sequences=1,
early_stopping=True,
length_penalty=1.0,
pad_token_id=tokenizer.eos_token_id,
eos_token_id=tokenizer.eos_token_id,
return_dict_in_generate=True,
output_scores=True
)
Decode and clean response
generated_response = tokenizer.decode(output_tokens.sequences[0], skip_special_tokens=True)
cleaned_response = html.unescape(generated_response).replace("#39;", "'").replace("quot;", '"')
print("\nGenerated Response:\n", cleaned_response)
GPT-2 for Next-line Prediction
This repository hosts a fine-tuned GPT-2 model optimized for next-line prediction tasks. The model has been fine-tuned on the OpenWebText dataset and quantized in FP16 format to enhance efficiency without compromising performance.
Model Details
Model Architecture: GPT-2 (Causal Language Model)Task: Next-line PredictionDataset: OpenWebText (subset: stas/openwebtext-10k)Quantization: FP16 for reduced model size and faster inferenceFine-tuning Framework: Hugging Face Transformers
Training Details
Number of Epochs: 3Batch Size: 4Evaluation Strategy: EpochLearning Rate: 5e-5
Evaluation Metrics (Perplexity Score)
Perplexity Score: 14.355693817138672
Limitations
The model is optimized for English-language next-word prediction tasks.
While quantization improves speed, minor accuracy degradation may occur.
Performance on out-of-distribution text (e.g., highly technical or domain-specific data) may be limited.
Usage Instructions
Installation
pip install transformers torchLoading the Model in Python
1 from transformers import AutoTokenizer , AutoModelForCausalLM
2 import torch
3
4 device = "cuda" if torch . cuda . is_available ( ) else "cpu"
5
6 model_name = "AventIQ-AI/gpt2-lmheadmodel-next-line-prediction-model"
7 tokenizer = AutoTokenizer . from_pretrained ( model_name )
8 model = AutoModelForCausalLM . from_pretrained ( model_name ) . to ( device )
Repository Structure
.
├── model/ # Contains the quantized model files
├── tokenizer_config/ # Tokenizer configuration and vocabulary files
├── model.safetensors/ # Quantized Model
├── README.md # Model documentationContributing
Contributions are welcome! Feel free to open an issue or submit a pull request if you have suggestions or improvements.
