You can deploy this model and start earning money today!

pcalhoun/
gpt-j-6b-8bit-pun-generator

Name: gpt-j-6b-8bit-pun-generator
Availability: InStock

apache-2.0

Views

No views yet

GPT-J-6B Pun Generator

A GPT-J-6B fine-tuned to invent puns that don't already exist — and to explain them. It's a base GPT-J-6B (no chat template) that was quantized to 8-bit and fine-tuned with low-rank adapters, trained alongside phonetic tasks (grapheme↔phoneme conversion and homophone lists) so it can build puns that turn on words which merely sound alike.

Background: "This is the moment I've been training for," said the pun-generating AI.

Some of its output:

Two guys argued about a painting. There was a rupture in the peace. (peace → piece)

Old pianists never die they just get tuned away.

A successful soft drink company is full of cool executives.

The story of a boy who was born with one eye in the wrong place was told from an unexpected angle.

"I love kiwis," said Tom kiwwisely. (they're not all zingers)

Files in this repo

There are two ready-to-run formats:

File	Format	Use
`model.safetensors` (+ `config.json`, tokenizer)	fp16 safetensors	loads with a plain `from_pretrained` on modern `transformers` — no custom code, no 8-bit tricks
`ggml-model-q4_0.bin`	ggml, 4-bit	CPU-only inference (~3.4 GB, ~4 GB RAM) via ggml's `gpt-j`
`eval.py`	reference	the original 2023 inference script — documents the prompt schema

The fp16 safetensors was produced from the original 8-bit training by dequantizing the weights and folding the trained adapters into the dense weights — it is numerically equivalent to the original (matching argmax and ≥0.999999 cosine on held-out probes). The original 8-bit .pt pickle has been removed in favour of this safetensors (safe-format) equivalent.

The prompt format

This is a base model with no chat template. Its entire task interface is a set of control tokens built from GPT-J's unused <|extratoken_N|> vocabulary. To generate a pun unconditionally, prompt with:

<|extratoken_60|><|extratoken_61|>   # TASK_START
<|extratoken_46|><|extratoken_47|>   # generate pun → explanation from keywords
<|extratoken_10|><|extratoken_11|>   # PUN begin
<|extratoken_24|><|extratoken_25|>   # GRAPHEMES (written form) begin

i.e. the concatenated string

<|extratoken_60|><|extratoken_61|><|extratoken_46|><|extratoken_47|><|extratoken_10|><|extratoken_11|><|extratoken_24|><|extratoken_25|>

, then sample. The pun is emitted as written text; the model also produces an explanation (often as ARPABET phonemes — a quirk of how deeply the phonetic training pervades it). Strip the <|extratoken_N|> tags from the output.

Usage — modern (fp16 safetensors)

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "pcalhoun/gpt-j-6b-8bit-pun-generator"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.float16).cuda().eval()

pre = ("<|extratoken_60|><|extratoken_61|>"
       "<|extratoken_46|><|extratoken_47|>"
       "<|extratoken_10|><|extratoken_11|>"
       "<|extratoken_24|><|extratoken_25|>")
ids = tok(pre, return_tensors="pt").to(model.device)
out = model.generate(ids.input_ids, do_sample=True, temperature=0.9,
                     top_k=50, top_p=0.98, max_new_tokens=128, pad_token_id=50256)
text = tok.decode(out[0])
import re
print(re.sub(r"<\|extratoken_\d+\|>", " ", text).strip())

Usage — CPU, 4-bit (ggml)

Note: mainline llama.cpp does not support GPT-J — the architecture is only implemented in the gpt-j example of the ggml repo. Build that, and patch its tokenizer to treat the <|extratoken_N|> tags as atomic special tokens (otherwise they get BPE-split and the model isn't conditioned):

bash

git clone https://github.com/ggml-org/ggml && cd ggml
cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build --target gpt-j -j
# download ggml-model-q4_0.bin from this repo, then:
./build/bin/gpt-j -m ggml-model-q4_0.bin -n 128 --top_k 50 --top_p 0.98 --temp 0.9 \
  -p '<|extratoken_60|><|extratoken_61|><|extratoken_46|><|extratoken_47|><|extratoken_10|><|extratoken_11|><|extratoken_24|><|extratoken_25|>'

Runs in ~3.4 GB on disk / ~4 GB RAM, CPU only.

License

Apache-2.0. The model was trained on data derived from public pun corpora; note that some of those corpora carry NonCommercial terms, so treat pun generation for redistribution accordingly.