You can deploy this model and start earning money today!

Ilikemechuri/
rwkv7-g1f-7.2b-transformers

Name: rwkv7-g1f-7.2b-transformers
Availability: InStock

Views

No views yet

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "Ilikemechuri/rwkv7-g1f-7.2b-transformers"

tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo,
    trust_remote_code=True,
    dtype="auto",
    device_map="auto",
)
model.eval()

inputs = tok("User: hello\n\nAssistant:", return_tensors="pt").to("cuda")

with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens=50,
        do_sample=False,
        stop_strings=["\n\n"],
        tokenizer=tok,
    )

print(tok.decode(out[0].tolist()))

Requires transformers >= 5.6.2 you must use transformers >= 5.6.2

I will update readme this is my plan

Model Introduction (Why RWKV-7, O(1))
Quick Start (Inference)
Chat Template
State Manipulation (Key Points for Researchers)
Convert from .pth
Known Limitations
Researcher's Guide (Advanced)
BlinkDL Citation

Citation

If you use this model, please cite the original RWKV work: https://github.com/BlinkDL/RWKV-LM

bibtex

@software{peng_bo_2021_5196578,
  author    = {PENG Bo},
  title     = {BlinkDL/RWKV-LM: 0.01},
  month     = aug,
  year      = 2021,
  publisher = {Zenodo},
  version   = {0.01},
  doi       = {10.5281/zenodo.5196577},
  url       = {https://doi.org/10.5281/zenodo.5196577}
}