This repository is a Transformers-compatible conversion of talkie-lm/talkie-1930-13b-base, the original Talkie base completion model.

The upstream model is a 13B vintage language model trained on 260B tokens of pre-1931 English-language text, according to the original model card.

The original base checkpoint is FP32. This repository stores a BF16 conversion of those weights and packages them for Transformers with custom trust_remote_code modules and BF16 sharded safetensors.

This is not an official Talkie release; refer to the upstream model card for the author-provided provenance and usage notes.

The public reference configuration originally advertised 2,048 positions, but the Talkie team later clarified that the model was trained with a 4,096-token context. This conversion uses the corrected 4,096-token limit.

For base completions:

The included remote-code model implements the Transformers attention-interface hooks expected by vLLM's Transformers modeling backend. For compatibility with that backend, the original single-scalar lm_head_gain is folded into lm_head.weight during conversion; the other Talkie gain parameters remain explicit model parameters. Using vLLM's logit_scale-style approach was not used because it applies scaling after the output matmul, while Talkie applies the gain to the head weight before the matmul. In BF16 this can introduce small rounding differences and, in smoke tests, changed one near-tied top-token ordering.

The BF16 checkpoint matched a runtime BF16 cast from the original FP32 checkpoint exactly on the tested forward pass. The Transformers safetensors model was also compared against the Talkie reference architecture; the top-10 next-token ordering matched exactly, with observed max absolute logit difference 0.03125.