13B model fine-tuned on agentic software-engineering trajectories from SWE-smith, starting from the talkie-web base (same architecture as talkie-1930 but pre-trained on web-style data). Tuned for the mini-swe-agent interaction format.

SWE-bench-Verified-Working-Harbor pass@1

metric	value
pass@1 (n=3 independent eval runs)	5.75% ± 1.04 pp
per-run resolved (out of 446)	31, 23, 23

Eval pipeline: vLLM (--model-impl transformers --max-model-len 32768 --dtype bfloat16) → mini-swe-agent (mini-extra swebench, temperature 0.7, max_tokens=4096), graded with the swebench harness against ricdomolm/SWE-bench_Verified-Working-Harbor.

Training recipe


Base model	`talkie-web-13b-base` (chat-token reinitialised)
Dataset	`talkie-web-swe-100k-64k` (100k SWE-smith trajectories, packed at 64k)
Trainer	TRL `SFTTrainer` via `accelerate` (8× A100)
Optimizer	`adamw_torch_fused`, β=(0.9, 0.95), ε=1e-8
LR	2e-5, `cosine_with_min_lr`, warmup 3%
Precision	bf16
Weight decay	0.1
Max grad norm	30
Max length	65,536
Packing	`bfd` + padding-free
Loss	`completion_only_loss=1` (loss only on assistant tokens)
Steps	2,016 (this is ckpt-2000)

Usage

This model uses custom modeling code (modeling_talkie.py, configuration_talkie.py). Load with trust_remote_code=True:

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "ricdomolm/talkie-web-coder",
    trust_remote_code=True,
    torch_dtype="bfloat16",
)
tokenizer = AutoTokenizer.from_pretrained("ricdomolm/talkie-web-coder")

For agentic eval, serve with vLLM and drive with mini-swe-agent:

bash

vllm serve ricdomolm/talkie-web-coder \
    --model-impl transformers --max-model-len 32768 --dtype bfloat16

Companion model

ricdomolm/talkie-1930-coder — same recipe, same SFT data, but starting from a different base model. Reaches 4.48% ± 0.69 pp on the same eval (n=5).