This is an experimental Chinese novel-agent supervised fine-tune of
RWKV-7 G1h 13.3B. It targets novel continuation/instruction following and
structured narrative tasks such as speaker attribution, narrative-type
classification, and scene-boundary detection.
This release is the full-pass checkpoint after one nominal packed-data pass
over the local packed training stream. Use it for research and evaluation
rather than production.
The starting checkpoint was
BlinkDL/rwkv7-g1/rwkv7-g1h-13.3b-20260710-ctx10240.pth. This repository does
not currently include a Transformers config or Safetensors conversion.
Prompt format
Use the fine-tuning prompt format below. Keep the single spaces after role
labels.
CUDA inference JIT-compiles the RWKV-7 extension on first use, so a working
CUDA toolkit, C++ compiler, and ninja executable are required. The model path
passed to RWKV must omit the .pth suffix.
For deterministic JSON-style evaluation, use greedy decoding:
temperature=0, top_p=0, alpha_frequency=0, alpha_presence=0,
chunk_len=256, stop token [0], and stop text \n\nUser:.
Training
The published checkpoint was produced in two stages:
Stage
Optimizer steps
Nominal token presentations
Notes
Pilot
977
120,053,760
Initial SFT run
Full-pass continuation
2,385
293,068,800
Continued from pilot weights with fresh optimizer state
Combined
3,362
413,122,560
One nominal packed-data pass
The full-pass continuation used 6 GPUs, BF16, DeepSpeed ZeRO-2, global batch
12 sequences (122,880 tokens/update), assistant-only loss masking, constant
learning rate 5e-7 after 10 warmup steps, weight decay 0.01, and gradient
clipping 0.5.
The packed training stream contains 131,920 conversations and 413,123,655
tokens, of which 329,536,148 were marked as supervised assistant targets.
Because packed sampling used the training script's magic-prime schedule, "one
nominal pass" should not be read as every raw token being seen exactly once.
Training data was derived from
mikuhhn1239/novel-agent-sft-dataset
at revision 5d3040d21f51b3ce90b9396b058e552c47f43cd5 and includes continuation,
instruction, attribution, narrative-analysis, and scene-boundary examples.
Evaluation
The checkpoint was evaluated with rwkv==0.8.32, PyTorch 2.7.1+cu128, BF16
CUDA inference, and greedy decoding on NVIDIA A800 80 GB GPUs. Prompts were
rendered with the same local packing code used for training, ending at
Assistant:. Invalid JSON counted as wrong or empty predictions.
*The dataset README's published scene F1 is for its v4-590 evaluation setup,
not this local merged v4/v4.1 held-out split, so it is a directional reference
rather than a strict apples-to-apples baseline.
The last recorded training loss is not an evaluation metric and is intentionally
omitted.
Intended use
Research on Chinese fiction generation and continuation.
Experiments in speaker attribution, narrative classification, and scene
boundary detection.
Further fine-tuning and evaluation of RWKV-7 recurrent inference.
Limitations and risks
This is an experimental SFT checkpoint; output quality may be unstable.
The configured 10,240-token training segment length is not evidence of
reliable performance across that full length.
Generated JSON may be malformed or violate the requested schema.
The model has not been evaluated for factuality, bias, memorization, harmful
content, or prompt-injection resistance.
Training data contains third-party Chinese novel text. Per-book rights and
provenance are not fully established, so users must assess copyright,
privacy, and downstream-use obligations for their jurisdiction and use case.
Do not rely on generated text for legal, medical, financial, safety-critical,
or other high-impact decisions.
License and attribution
The upstream RWKV-7 G1h checkpoint is distributed under Apache-2.0, and this
repository retains that model-repository license metadata. The dataset's
Apache-2.0 label does not establish rights to every underlying novel. The data
rights caveat above remains applicable to this derivative checkpoint.
