You can deploy this model and start earning money today!

mxxsc/
zip2zip-phi35-core-v1

Name: zip2zip-phi35-core-v1
Availability: InStock

zip2zip

llaza

adaptive-tokenization

Views

No views yet

zip2zip-phi35-core-v1

Training checkpoint from zip2zip-core. This is a candidate model (not production-ready).

Training Config

Field	Value
model_config	`Phi3.5-mini`
init_from	`microsoft/Phi-3.5-mini-instruct`
max_subtokens	4
max_codebook_size	2048
seq_len	2048
lr	0.0003
max_tokens	262144000
step	8000
data	`phi-1B-sft-8shards`

Usage

This is a training checkpoint (torchtitan format). To use for inference, export to HuggingFace format first:

bash

python scripts/zip2zip_hf/export_to_zip2zip.py \
    --ckpt_dir <local_path>/step_8000 \
    --output_dir <export_dir> \
    --base_model microsoft/Phi-3.5-mini-instruct \
    --model_config Phi3.5-mini

mxxsc/zip2zip-phi35-core-v1

zip2zip-phi35-core-v1

Training Config

Usage

mxxsc/
zip2zip-phi35-core-v1