Beta

Explore
Marketplace
Neural Labs
Chat
Wallet
Docs

Beta

Explore Marketplace Neural Labs Wallet Docs

Build

Manage

My assets Manage wallet Organization

Community

Discord X (Twitter)Telegram

Docs Privacy Policy Terms and Conditions

Proud member of

Proud member of

Docs Cookie Policy

Privacy Policy Terms and Conditions

© 2026 AlphaNeural AI. All rights reserved.

bk-moo-llama3-8b-mgda-fineweb – AI Model by Yiran0924 | AlphaNeural AI

You can deploy this model and start earning money today!

Yiran0924/
bk-moo-llama3-8b-mgda-fineweb

like

0

bk-moo-pretrain

continual-pretraining

meta-llama/Meta-Llama-3-8B

Views

No views yet

BK-MOO-Pretrain · mgda · fineweb

Continual pretraining checkpoint of meta-llama/Meta-Llama-3-8B following the BK-MOO-Pretrain recipe (https://github.com/longhp1618/BK-MOO-Pretrain).

Method: mgda
Training stage: fineweb
Base model: meta-llama/Meta-Llama-3-8B
Languages: ar, bn, en, id, it, ko, sw, zh
Training data: FineWeb-2 (and LIMA+S1 for *_limas1 stages)

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "Yiran0924/bk-moo-llama3-8b-mgda-fineweb"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="bfloat16", device_map="auto")

See the BK-MOO-Pretrain repo for the multi-objective gradient surgery details (MGDA / CAGrad / PCGrad / SGD).