The newest GemmaNPC models, with the new Gemma4-E4B model as the base model, trained using the newest RolePlay-NPC-Quest dataset.
Intended Usage
This model is trained to be used as a more game focused NPC rolaplaying model.
Training Parameters
For this model, we employed a slightly less conservative parameter, which resulted in some beautiful training loss(Tensorboard attached).
We trained this model as a r=32, alpha=64 LoRA adapter with 2 epochs over RolePlay-NPC-Quest using a 80GB vRAM A100 in Google Colab. For this run, we employed a learning rate of 1e-4 and an effective batch size of 24. A cosine learning rate scheduler was used with an 500-step warmup. With a gradient clipping of 1.0.
Notes
As Unsloth noted in their official guide, training Gemma4 with text only would lead to a higher than usual loss and grad_norm, which we observed during training.
The performance of this model, especially the intruction-following capabilities is a huge step up compared to Gemma3/3n.
Inference Guidelines
Recommended Settings:
temp = 1.0, top_p = 0.95 and top_k = 64.
Optimal System Prompt:
System Prompt without Objective:
Enter Roleplay Mode. You are <|character name|>.
Background: <|Character background/bio|>
Location: <|Description of the current location|>
Roleplaying Instructions: <|Instructions|>
System Prompt with Objective:
Enter Roleplay Mode.
You are <|character name|>.
Background: <|Character background/bio|>
Location: <|Description of the current location|>
Quest: <|Quest description|>
Roleplaying Instructions: <|Instructions|>
Example of Roleplaying Instructions:
Here is an example of the roleplaying instructions we used to train the model:
Roleplaying Instructions:
- Speak using appropriate tone and vocabulary
- Reference your background and current surroundings naturally
- Keep responses conversational and authentic
- React to the player's words and intentions.
Your first response should be a greeting to the player.
First User Prompt:
It is recommended that the first user prompt should always be Greetings, then letting the model generate a greeting, smiliar to how an NPC would behave in game.
