Views
No views yet
bigcode/the-stack (v1, non-dedup). It is superseded by the paper's Phase 3 adapter, which was re-trained from scratch on the cleaner bigcode/the-stack-v2-dedup corpus. For paper-grade use, load the Phase 3 adapter from the umbrella repo:PeftModel.from_pretrained(base_model, "legesher/language-decoded-lora", subfolder="tiny-aya-base/condition-1-en-5k-seed42")legesher/language-decoded-lora-condition-1-en-5k; the old URL continues to resolve via a Hugging Face redirect.bigcode/the-stack (v1, non-dedup), 5k subset. Tests whether code fine-tuning improves multilingual reasoning (replicates Aryabumi et al., 2024).For the full adapter inventory across both phases, see the Language Decoded LoRA hub and itsMANIFEST.md.
legesher/language-decoded-data / phase-2-the-stack-v1-condition-1-en-5k — the Phase 2 / The Stack v1 config.1from transformers import AutoModelForCausalLM, AutoTokenizer
2from peft import PeftModel
3
4base_model = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-base")
5tokenizer = AutoTokenizer.from_pretrained("CohereLabs/tiny-aya-base")
6# Preliminary Phase 2 adapter (kept for reproducibility):
7model = PeftModel.from_pretrained(base_model, "legesher/language-decoded-lora-phase-2-the-stack-v1-condition-1-en-5k")1@misc{language-decoded-2026,
2 title={Language, Decoded: Exploring the Impact of Fine-Tuning a Multilingual Model on Native-Language Code},
3 author={Madison Edgar and Saad Ahmed Bazaz and Tom Sherborne and Rashik Shahjahan and Khojasteh Mirza and Sarah Jawaid and Rafay Mustafa and Sohaib Ahmed Bazaz},
4 year={2026},
5 publisher={Hugging Face},
6 url={https://huggingface.co/legesher/language-decoded-lora}
7}