My base pretrain model has undergone full fine-tuning on an additional 350M tokens using portions of Tulu 3 and Nvidia Nemotron instruct sets. It is rough but functionsl, and still needs DPO training to align it with human preferences.

For the base pretrain, see: Nabbers1999/Mini-Llama-3B-Base-0124

** Note: This model has a custom tokenizer. Llama.cpp must be edited before converting from HF to GGUF; however, you may use my BF16 GGUF to create additional quants if desired.