Beta

Explore
Marketplace
Neural Labs
Playground
Wallet
Docs

Beta

Explore Marketplace Neural Labs Wallet Docs

Build

Manage

My assets Manage wallet Organization

Community

Discord X (Twitter)Telegram

Docs Privacy Policy Terms and Conditions

Proud member of

Proud member of

Docs Cookie Policy

Privacy Policy Terms and Conditions

© 2026 AlphaNeural AI. All rights reserved.

codellm_pretrain – AI Model by dudeperf3ct | AlphaNeural AI

You can deploy this model and start earning money today!

dudeperf3ct/
codellm_pretrain

like

0

tokyotech-llm/swallow-code-v2

dudeperf3ct/codellm-tokenizer

Views

No views yet

Writeup: https://dudeperf3ct.github.io/projects/train_llm_part2/

Repo: https://github.com/dudeperf3ct/minicode-llm/tree/main/codellm_pretrain/torch_titan

This contains checkpoints every 5k steps for pretraining run 9.8B tokens using

Custom tokenizer: https://dudeperf3ct.github.io/projects/train_llm_part1/
Dataset: tokyotech-llm/swallow-code-v2
Model Architecture: Llama 3.2 1B (1 billion parameter)

The repository contains detailed step on how to run evaluation using PyTorch DCP checkpoints.