This is a experimental llama2 7B qlora made using the VNTL-v2-2k-small dataset.
Unlike version 0.1, the input was masked out of the loss calculation during training.
The objectives of this fine-tune are:
Teaching the model how to translate while respecting the context.
Teaching the model how to translate while following the character's metadata.
Teaching the model how to translate while respecting the translation fidelity.
I can say with certainty that objectives 1 and 2 were completed successfully. However, the objective 3 wasn't, most likely due to the difficulty of the model making such association, so instead, I used the fidelity classification just to exclude the translations with low/medium fidelity from affecting the loss calculation, since these translations are, most of the time, creative translations or straight up mismatched translations.
