You can deploy this model and start earning money today!

kerasformers/
gpt2_xl

Name: gpt2_xl
Availability: InStock

openai-community/gpt2-xl

finetune

mit

Views

No views yet

Run GPT-2 with Keras 3: JAX, PyTorch, or TensorFlow

GitHub

zeromodels/gpt2_xl

Paper: Language Models are Unsupervised Multitask Learners (Radford et al., 2019)

GPT-2 is OpenAI's decoder-only transformer language model trained on WebText: learned absolute position embeddings, pre-LayerNorm blocks, gelu_new activations, a tied output head, and a byte-level BPE tokenizer. This is the 1.5B variant, a base completion model (no chat template).

For more details, see the upstream model card.

Pure-Keras 3 conversion of openai-community/gpt2-xl for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX.

Quick start

python

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from zeromodels.models.gpt2 import GPT2TextGenerate, GPT2Tokenizer

model = GPT2TextGenerate.from_weights("zeromodels/gpt2_xl")
tokenizer = GPT2Tokenizer.from_weights("zeromodels/gpt2_xl")

inputs = tokenizer("The meaning of life is")
outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0]))

All GPT-2 sizes load the same way with from_weights("zeromodels/<variant>"):

Variant	Hub	Params
`gpt2`	`zeromodels/gpt2`	124M
`gpt2_medium`	`zeromodels/gpt2_medium`	355M
`gpt2_large`	`zeromodels/gpt2_large`	774M
`gpt2_xl`	`zeromodels/gpt2_xl`	1.5B

Tips

Set KERAS_BACKEND before importing Keras / zeromodels.
This is a base completion model: it continues a prompt and is not instruction-tuned.
See the GPT-2 docs and Loading Weights.
Upstream safetensors still work via the hf: prefix, e.g. GPT2TextGenerate.from_weights("hf:openai-community/gpt2-xl").

Special Thanks

A huge thank you to the OpenAI GPT-2 authors for creating and releasing these models.

License: MIT, inherited from the upstream OpenAI GPT-2 release.

kerasformers/gpt2_xl