Views
No views yet
3) of
desh2806/simplestories-persona-clusters-augment, where the persona
label is the dataset's cluster column. One specialist is trained per cluster on
10,000 of that cluster's stories; no mixture model is trained. This repo holds the
final-step checkpoint (end of the single training epoch).| base model | SimpleStories/SimpleStories-V2-5M |
| run | cluster_3 (cluster specialist) |
| epochs | 1 (single epoch — every example seen once) |
| final step | 313 of 313 (313 steps/epoch) |
| train examples | 10000 |
| optimizer | AdamW, lr=0.0005, weight_decay=0.0 |
| batch size | 32 |
| precision | fp32 |
| seed | 42 |
val_own: 1.6922val_mix: 2.29891from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model = AutoModelForCausalLM.from_pretrained("desh2806/simplestories-cluster-cluster_3")
4tokenizer = AutoTokenizer.from_pretrained("desh2806/simplestories-cluster-cluster_3")
5
6# The base model has no BOS; seed generation with EOS (id=1) to start a new story.
7import torch
8seed = torch.tensor([[tokenizer.eos_token_id]])
9out = model.generate(seed, max_new_tokens=150, do_sample=True, temperature=1.0, top_p=0.95,
10 eos_token_id=tokenizer.eos_token_id, pad_token_id=tokenizer.eos_token_id)
11print(tokenizer.decode(out[0][1:], skip_special_tokens=True))add_special_tokens=False, EOS (id=1)
appended to every story, truncated to 512 tokens.