You can deploy this model and start earning money today!

epfl-dlab/
candidate-Llaza-3.2-1B-MS2F-4K-FT-1BT-v0.1

Name: candidate-Llaza-3.2-1B-MS2F-4K-FT-1BT-v0.1
Availability: InStock

zip2zip

llaza

adaptive-tokenization

Views

No views yet

candidate-Llaza-3.2-1B-MS2F-4K-FT-1BT-v0.1

Training checkpoint from zip2zip-core. This is a candidate model (not production-ready).

Training Config

Field	Value
model_config	`1B_llama3.2`
init_from	`meta-llama/Llama-3.2-1B-Instruct`
max_subtokens	2
max_codebook_size	4096
seq_len	4096
lr	5e-05
max_tokens	1000000000
step	1908
data	`zip2zip_1b`

Usage

This is a training checkpoint (torchtitan format). To use for inference, export to HuggingFace format first:

bash

python scripts/zip2zip_hf/export_to_zip2zip.py \
    --ckpt_dir <local_path>/step_1908 \
    --output_dir <export_dir> \
    --base_model meta-llama/Llama-3.2-1B-Instruct \
    --model_config 1B_llama3.2

epfl-dlab/candidate-Llaza-3.2-1B-MS2F-4K-FT-1BT-v0.1

candidate-Llaza-3.2-1B-MS2F-4K-FT-1BT-v0.1

Training Config

Usage

epfl-dlab/
candidate-Llaza-3.2-1B-MS2F-4K-FT-1BT-v0.1