ttmodel is a fine-tuned version of GPT-2 using a custom dataset related to adventure travel. This model is designed for generating coherent and contextually relevant text, particularly in the domain of travel and location-based descriptions. The training process involved fine-tuning the GPT-2 model on detailed location descriptions to enhance its ability to produce high-quality travel-related content.
Training Data:
The model was fine-tuned using a dataset containing travel-related text, specifically location names and information. The dataset includes various travel destinations and their detailed descriptions, providing the model with diverse data points for generating content in this domain.
Intended Use
This model is primarily intended for:
Text Generation: You can use it to generate location descriptions, travel-related content, or even assist in travel blog writing.
Tourism Applications: Useful for applications requiring automatic content generation for travel destinations, such as itinerary planning apps, travel guides, etc.
Limitations:
This model is specialized in travel and location descriptions and may not perform as well in other domains.
The model may repeat phrases or generate incoherent outputs if not controlled with proper sampling parameters.
How to Use
You can use this model with the Hugging Face transformers library. Here’s an example to generate text:
from transformers import GPT2Tokenizer, GPT2LMHeadModel
Load model and tokenizer
tokenizer = GPT2Tokenizer.from_pretrained("yoonusajward/ttmodel")
model = GPT2LMHeadModel.from_pretrained("yoonusajward/ttmodel")
Generate text
input_text = "Kitulgala"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(inputs.input_ids, max_length=150, do_sample=True, top_k=50, top_p=0.95, temperature=0.7)
generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(generated_text)
Training Hyperparameters
Model Architecture: GPT-2 (pretrained)
Learning Rate: 5e-5
Epochs: 20
Batch Size: 2
Evaluation Strategy: Steps-based evaluation (every 200 steps)
Early Stopping: Yes (patience of 5 evaluations)
