LLaMA-3 7B pretrained on English bilingual data (en1 + en2) with code-switching, 134k steps. Uses the en1 embedding matrix (--extract_vocab 0) from the 2×vocabulary checkpoint. For the en2 embedding matrix version see The-CoLab/llama3-7b-en1-en2-codeswitching-134k.

Final checkpoint: step 133,600

See The-CoLab/llama3-7b-en1-en2-codeswitching-134k for training curves (same checkpoint, different embedding extraction).

Part of the The-CoLab multilingual-transfer collection.