Experimental WIP prototype Rank 256/Alpha 256 adapter for Qwen3-4B-Instruct-2507.
This adapter belongs to our Lyrical MT (Machine Translation) series of fine-tuned LLMs and adapters.
Our ultimate aim with the Lyrical MT project is to iteratively foster a translation model capable of adaptively localizing idiomatic, formal/poetic/rhythmic, and performance-catered features of lyrical input texts, whilst retaining adequate accuracy at the level of direct semantic translation.

USES:

Intended scope of effective applicability limited to:
Russian to English translation of song lyrics, poems, scriptures, slogans, etc...
Translation from a Russian-language input text structured in accordance with literary, aesthetic, or/and vocalization-catering compositional devices to an English output text exhibiting cross-lingually rebalanced approximations of source-matched formal features.

Depending on the relative performance, foundations, and the idiosyncracies of a given checkpoint/adapter variant in the Lyrical MT series, the above-suggested applicability scope may plausibly extend to:
Russian to English text-to-text translation in general.
English to Russian translation.

The Lyrical MT models were fine-tuned primarily on single-line (fragment), double-line (couplet), quadruple-line (quatrain), and full-length bilingual textual inputs.

Training Info

The training was conducted on one L4 GPU (w/ 22.5 GB VRAM) via the TRL framework and the ORPO Trainer, leveraged via Unsloth over their 4-bit optimized dynamic quantized variant of the Qwen3-4B-Instruct-2507 model.

Adapter Checkpoint Log

      "epoch": 6.25615763546798,       "grad_norm": 0.14729894697666168,       "learning_rate": 1.379644707441633e-05,       "log_odds_chosen": 8.985773086547852,       "log_odds_ratio": -0.0001251702633453533,       "logits/chosen": -1.50288987159729,       "logits/rejected": -1.4996038675308228,       "logps/chosen": -0.19279448688030243,       "logps/rejected": -7.438176155090332,       "loss": 0.0378,       "nll_loss": 0.037755634635686874,       "rewards/accuracies": 1.0,       "rewards/chosen": -0.019279448315501213,       "rewards/margins": 0.7245382070541382,       "rewards/rejected": -0.7438176274299622,       "step": 12700

Training Data

Fine-tuned for Odds Ratio Preference Optimization (ORPO) on our ORPO-catered Russian-to-English song lyrics translation/localization dataset.

Hyperparameters

Adapter Rank = 256 Adapter Alpha = 256 Learning Rate = 1e-4 Max Sequence Length = 2048 Optimizer = AdamW_8bit Learning Rate Scheduler Type = Cosine Beta/Decay = 0.1

Framework versions

PEFT 0.17.1 transformers 4.55.4

Note:

We would appreciate feedback/reports from anyone else who happens to try out this model, or its other variants (to be released in the near future).

AlekseyCalvin/Lyrical_MT_ru2en_var2b_Qwen3_4b-it_r256_6epochs_adapter

LYRICAL Russian to English Machine Translation Model