AlphaMonarch-daser is a mixture of two techniques that are LaserQlora and Dora. This model is a DPO fine-tuned of
mlabonne/NeuralMonarch-7B using the
argilla/OpenHermes2.5-dpo-binarized-alpha preference dataset. I have fine-tuned this model only on half of the projections, but have achieved better results as compared to the version released
AlphaMonarch-dora. I have trained this model for 1080 steps. Comparison of AlphaMonarch, AlphaMonarch-laser, AlphaMonarch-daser, and AlphaMonarch-dora on the OpenLLM leaderboard are:
On YALL leaderboard: AlphaMonarch-daser > AlphaMonarch-dora > AlphaMonarch > AlphaMonarch-laser
On OpenLLM bench: AlphaMonarch-laser > AlphaMonarch > AlphaMonarch-daser > AlphaMonarch-dora
