You can deploy this model and start earning money today!

Sunny063/
ERAV4-Week13-SmolLLM2-135m

Name: ERAV4-Week13-SmolLLM2-135m
Availability: InStock

Views

No views yet

SmolLM2-135M (ERA V4 Week13) — trained from scratch

This repository contains a SmolLM2-135M compatible checkpoint trained from scratch as part of ERA V4 Week13.

Architecture (exported)

vocab_size=49152
hidden_size=576
intermediate_size=1536
num_layers=30
n_heads=9
n_kv_heads=3
RoPE theta=100000.0
RMSNorm eps=1e-05
max_position_embeddings=8192
tie_word_embeddings=True

Quick usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "<YOUR_HF_USERNAME/YOUR_MODEL_REPO>"
tok = AutoTokenizer.from_pretrained(model_id, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(model_id)

inputs = tok("BIANCA:", return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=80)
print(tok.decode(out[0], skip_special_tokens=True))

Sunny063/ERAV4-Week13-SmolLLM2-135m

SmolLM2-135M (ERA V4 Week13) — trained from scratch

Architecture (exported)

Quick usage

Sunny063/
ERAV4-Week13-SmolLLM2-135m