You can deploy this model and start earning money today!

locuslab/
ift_gsm-smollm2-1.7b-score0_20p_rephrase123_mild_ref45_metadata_5p-600B-metamix3p-1k-100

Name: ift_gsm-smollm2-1.7b-score0_20p_rephrase123_mild_ref45_metadata_5p-600B-metamix3p-1k-100
Availability: InStock

Views

No views yet

SmolLM2 -score0_20p_rephrase123_mild_ref45_metadata_5p-600B-metamix3p-1k-100 (Version: main)

Model Details

Architecture: SmolLM2
Parameters: 1.7B

Training Configuration

yaml

optimizer:
  class_path: torch.optim.AdamW
  init_args:
    lr: 0.0005
    weight_decay: 0.01
precision: bf16-mixed
seed: 42
train:
  global_batch_size: 1024
  max_seq_length: 2048
  max_tokens: 600000000000
  micro_batch_size: 8

Model Loading and Revision System

This repository hosts multiple revisions of the model. To load a specific revision, use the revision parameter. For example:

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("locuslab/-score0_20p_rephrase123_mild_ref45_metadata_5p-600B-metamix3p-1k-100", revision="final")
tokenizer = AutoTokenizer.from_pretrained("locuslab/-score0_20p_rephrase123_mild_ref45_metadata_5p-600B-metamix3p-1k-100", revision="final")

Replace "final" with the desired revision.

locuslab/ift_gsm-smollm2-1.7b-score0_20p_rephrase123_mild_ref45_metadata_5p-600B-metamix3p-1k-100

SmolLM2 -score0_20p_rephrase123_mild_ref45_metadata_5p-600B-metamix3p-1k-100 (Version: main)

Model Details

Training Configuration

Model Loading and Revision System

locuslab/
ift_gsm-smollm2-1.7b-score0_20p_rephrase123_mild_ref45_metadata_5p-600B-metamix3p-1k-100