Views
No views yet
talkie-lm/talkie-1930-13b-it. The original release ships as a raw torch state-dict (rl-refined.pt) plus a tiktoken vocab, with no config.json, tokenizer.json, or HF modeling code, so it can't be loaded by transformers or served by vLLM out of the box.talkie-1930-13b-it is a 13B parameter instruction-tuned model from the talkie-lm project. The base model was pretrained on ~260B tokens of pre-1931 English text; the IT variant was instruction-tuned on a dataset built from pre-1931 reference works (etiquette manuals, encyclopedias, letter-writing guides) and refined with online DPO.| File | What it is |
|---|---|
model.safetensors | bf16 weights, ~25 GB. lm_head_gain (a learned scalar) is pre-multiplied into lm_head.weight so vLLM's transformers backend doesn't need to know about it. |
config.json | TalkieConfig (vocab=65540, hidden=5120, 40 layers × 40 heads, head_dim=128, ctx=2048, RoPE θ=1e6) plus auto_map for AutoConfig/AutoModel/AutoModelForCausalLM. |
tokenizer.json, tokenizer_config.json | HF fast BPE built from the original vocab.txt, with the 5 chat specials at fixed ids 65535..65539. EOS = <|end|>, pad = <|endoftext|>. |
chat_template.jinja | Renders to <|system|>…<|end|><|user|>…<|end|><|assistant|>…<|end|><|assistant|>, byte-matching format_chat from the official inference repo. |
generation_config.json | eos_token_id=[65536, 65535], pad_token_id=65535. |
modeling_talkie.py, configuration_talkie.py | HF PreTrainedModel implementation with ALL_ATTENTION_FUNCTIONS dispatch (vLLM transformers-backend compatible). Adapted from ricdomolm/1930-coder; TalkieForCausalLM.lm_head is an nn.Linear so the same lm_head.weight blob loads for both HF and vLLM. |
1vllm serve awilliamson/talkie-1930-13b-it-vllm \
2 --model-impl transformers \
3 --trust-remote-code \
4 --dtype bfloat16 \
5 --max-model-len 20481curl http://localhost:8000/v1/chat/completions \
2 -H "Content-Type: application/json" \
3 -d '{
4 "model": "awilliamson/talkie-1930-13b-it-vllm",
5 "messages": [{"role":"user","content":"Write one sentence about the year 1925."}],
6 "temperature": 0.7,
7 "max_tokens": 80
8 }'temperature ≥ 0.5 — greedy decoding (temperature=0) can collapse into single-token loops on this model.top_p / top_k and repetition_penalty don't reliably help with that failure mode; temperature does.max_position_embeddings=2048 matches the original IT training. The talkie-coder SWE recipe extends to 64K with NTK rope_theta=4e7, but loses ~14% on short evals (GSM8K) — only worth it for long-context agentic use.1from transformers import AutoTokenizer, AutoModelForCausalLM
2import torch
3
4tok = AutoTokenizer.from_pretrained("awilliamson/talkie-1930-13b-it-vllm", trust_remote_code=True)
5m = AutoModelForCausalLM.from_pretrained(
6 "awilliamson/talkie-1930-13b-it-vllm",
7 trust_remote_code=True,
8 dtype=torch.bfloat16,
9).cuda().eval()
10
11chat = tok.apply_chat_template(
12 [{"role": "user", "content": "Write one sentence about the year 1925."}],
13 tokenize=False, add_generation_prompt=True,
14)
15ids = tok([chat], return_tensors="pt").to("cuda")
16out = m.generate(
17 **ids, max_new_tokens=80, do_sample=True, temperature=0.7, top_p=0.9,
18 pad_token_id=tok.pad_token_id,
19 eos_token_id=[tok.convert_tokens_to_ids("<|end|>"), tok.eos_token_id],
20)
21print(tok.decode(out[0, ids.input_ids.shape[1]:], skip_special_tokens=True))rl-refined.pt from talkie-lm/talkie-1930-13b-it (bf16, vocab=65540, lm_head_gain.w_g=3.890625 baked in).vocab.txt from the same release (truncated to ranks < 65535, then the 5 chat specials appended at fixed ids).TalkieForCausalLM.lm_head switched from nn.Parameter to nn.Linear so the baked-in lm_head.weight loads cleanly for both HF and vLLM.talkie-lm/talkie-1930-13b-it release.