This model is part of the Deepstory project, which combines Natural Language Generation, Text-to-Speech, and animation technologies to create interactive storytelling experiences.

The model has been fine-tuned on The Witcher novels to generate text that captures the narrative style, vocabulary, and themes of the original books.

Model Architecture

This model is based on GPT-2 Medium architecture.

Parameter	Value
Architecture	GPT2LMHeadModel
Model Size	GPT-2 Medium
Number of Layers (n_layer)	24
Hidden Size (n_embd)	1024
Attention Heads (n_head)	16
Context Length (n_ctx)	1024
Max Positions (n_positions)	1024
Vocabulary Size	50,257
Activation Function	GELU (new)
Attention Dropout	0.1
Embedding Dropout	0.1
Residual Dropout	0.1
Layer Norm Epsilon	1e-05

Usage

With Transformers Library

python

from transformers import GPT2Tokenizer, GPT2LMHeadModel
import torch

# Load model and tokenizer
tokenizer = GPT2Tokenizer.from_pretrained("thetobysiu/gpt2-witcher-books")
model = GPT2LMHeadModel.from_pretrained("thetobysiu/gpt2-witcher-books")

# Generate text
prompt = "The witcher drew his silver sword and"
input_ids = tokenizer.encode(prompt, return_tensors='pt')

output = model.generate(
    input_ids=input_ids,
    max_length=150,
    temperature=0.9,
    top_p=0.95,
    top_k=50,
    do_sample=True,
    num_return_sequences=1
)

generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
print(generated_text)

Generation Parameters

For best results, we recommend the following generation parameters:

Parameter	Recommended Value	Description
temperature	0.7 - 1.0	Higher values = more creative
top_p	0.9 - 0.95	Nucleus sampling threshold
top_k	40 - 50	Top-k sampling
max_length	100 - 300	Maximum tokens to generate
do_sample	True	Enable sampling

Training Data

The model was fine-tuned on text from The Witcher book series, including:

The Last Wish
Sword of Destiny
Blood of Elves
Time of Contempt
Baptism of Fire
The Tower of the Swallow
The Lady of the Lake
Season of Storms

Training Procedure

The model was fine-tuned using the Hugging Face Transformers library starting from the pre-trained GPT-2 Medium checkpoint.

Intended Use

This model is intended for:

Creative writing assistance in fantasy genre
Interactive storytelling applications
Fan fiction generation
Research in language model fine-tuning
Educational purposes

Limitations

The model may generate content that doesn't perfectly match the original author's style
Context length is limited to 1024 tokens
May occasionally generate repetitive or incoherent text
The training data is based on copyrighted material

Ethical Considerations

This model is trained on copyrighted material and should be used for personal/research purposes
Generated content should not be presented as original work by the book author
Users should be aware that generated text may contain mature themes consistent with the source material

Citation

If you use this model, please cite the Deepstory project:

bibtex

@misc{deepstory,
  author = {Siu King Wai},
  title = {Deepstory},
  year = {2020},
  publisher = {GitHub},
  url = {https://github.com/thetobysiu/deepstory}
}

License

This model is released under the MIT License. Please note that The Witcher series is the intellectual property of Andrzej Sapkowski and CD Projekt.

Acknowledgments

OpenAI for the original GPT-2 model
Hugging Face for the Transformers library
Andrzej Sapkowski for The Witcher book series