OpenMOSE/

rwkv-7b-jp-lora

license: apache-2.0

a LoRA fine-tuned model with Japanese dataset

LoRA Experiment

rwkv-7b-jp-280.pth is merged model with base

Base Model

RWKV-4-Raven-7B-v10-Eng89%25-Jpn10%25-Other1%25-20230420-ctx4096.pth

Parameters: Lora Rank 1024 Lora Alpha 2048 ctx length 1024 Lora Size: almost 1.6GB

Dataset