This repo contains the copy of the original quantized using EXL2. Original: EVA-UNIT-01/EVA-Qwen2.5-14B-v0.1
EVA Qwen2.5 14B 0.1
A RP/storywriting specialist model, full-parameter finetune of Qwen2.5-7B on mixture of synthetic and natural data.
It uses Celeste 70B 0.1 data mixture, greatly expanding it to improve versatility, creativity and "flavor" of the resulting model.
Version 0.1 notes:
Dataset was deduped and cleaned from version 0.0, sequence length was also increased. Resulting model seems to be stabler, and 0.0 problems with handling short inputs and min_p sampling seem to be gone.
This version seems to be more or less optimal for the current data. It (again) started crashing on each checkpoint after some point, but it was less of a problem this time, as eval/loss already flatlined by that time. This is epoch 2.7 checkpoint.
Note: using quantized KV cache with Qwen2.5 is not recommended and can lead to degraded output quality. On the other hand, Qwen's KV cache is already light enough, so using f16 for it shouldn't be problematic.
Prompt format is ChatML.
Recommended sampler values:
- Temperature: 0.87
- Top-P: 0.81
- Min-P: 0.0025
- Repetition Penalty: 1.03
Temperature lower than 1 is still recommended, but this version seems to perform ok with higher temp and Min-P. Problems with Min-P sampling seem to be finally solved.
Recommended SillyTavern presets (via CalamitousFelicitousness):
Training data:
- Celeste 70B 0.1 data mixture minus Opus Instruct subset. See that model's card for details.
- Kalomaze's Opus_Instruct_25k dataset, filtered for refusals.
- A subset (1k rows) of ChatGPT-4o-WritingPrompts by Gryphe
- A subset (2k rows) of Sonnet3.5-Charcards-Roleplay by Gryphe
- A cleaned subset (~3k rows) of shortstories_synthlabels by Auri
- Synthstruct and SynthRP datasets by Epiculous
Training time and hardware:
Model was trained by Kearm and Auri.
Special thanks:
- to Gryphe, Lemmy, Kalomaze, Nopm and Epiculous for the data
- to Alpindale for helping with FFT config for Qwen2.5
- to InfermaticAI's community for their continued support for our endeavors
- and to Allura-org for support and feedback on EVA models.
