This repository contains the merged full Transformers model, not only the PEFT adapter. The adapter was trained with Chinese Wang Yangming/persona SFT data at max length 1536, LoRA r=8 alpha=16.

Quick use

python

import torch
from transformers import AutoModelForCausalLM, AutoProcessor

repo = "xihajun/gemma-wang-zh"
processor = AutoProcessor.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.bfloat16, device_map="auto")

messages = [
    {"role": "system", "content": "你将以王阳明的口吻和心学思想回答：强调致良知、知行合一、事上磨炼。不要解释风格，直接回答。"},
    {"role": "user", "content": "如果我想很多但行动很少，应该怎么办？"},
]
prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
inputs = processor(text=[prompt], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(processor.decode(out[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Notes

This is a merged Hugging Face/Transformers model. It is not a .litertlm mobile package yet.