This model is a pruned version of HuggingFaceTB/SmolLM-135M-Instruct.
The pruning procedure reduced both layers and hidden dimensions, decreasing parameter count from 134M → ~93M (~30.5% reduction).

⚠️ Important Note:
This model has not been fine-tuned after pruning. Since layers and parts of weights were dropped, the model will not produce accurate outputs in its current state. To make it useful, one must apply distillation or fine-tuning.

Developed by: Independent modification (original model: HuggingFaceTB)
Model type: Causal Language Model (decoder-only, LLaMA architecture)
Language(s) (NLP): English (same as original SmolLM training corpus)
License: Inherits license from the original SmolLM-135M-Instruct
Finetuned from model: HuggingFaceTB/SmolLM-135M-Instruct

Model Sources

Repository: Original SmolLM
Paper [optional]: N/A
Demo [optional]: N/A

Uses

Direct Use

⚠️ Not suitable for inference out-of-the-box.
Intended for research in pruning, model compression, and architecture efficiency experiments.

Downstream Use

Can be fine-tuned or distilled on downstream NLP tasks (instruction following, summarization, dialogue, etc.) to regain performance.
Useful as a smaller backbone for constrained environments (edge devices, prototyping).

Out-of-Scope Use

Do not expect reliable outputs without fine-tuning.
Not suitable for production or safety-critical tasks.
Not intended for generating factual, unbiased, or safe text without retraining.

Bias, Risks, and Limitations

Risks: Outputs are nonsensical and misleading in current state.
Biases: Same biases as original SmolLM dataset, but pruning may amplify instability.
Limitations: Lower representational capacity due to fewer layers/hidden units → lower accuracy even after retraining.

Recommendations

Perform knowledge distillation from the original model onto this pruned version.
Apply fine-tuning for task-specific usage.
Do not use for real-world decision-making without retraining and evaluation.

How to Get Started with the Model

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "your-username/SmolLM-135M-Instruct-layer-width-pruned-90M-raw"
tokenizer = AutoTokenizer.from_pretrained("HuggingFaceTB/SmolLM-135M-Instruct")
model = AutoModelForCausalLM.from_pretrained(model_name)

inputs = tokenizer("Hello world!", return_tensors="pt")
outputs = model.generate(**inputs, max_length=50)
print(tokenizer.decode(outputs[0]))

⚠️ The outputs are not meaningful until fine-tuned.

Training Details

Training Data

Same as original SmolLM-135M-Instruct.
No new training performed after pruning.

Training Procedure

Step 1: Layer pruning → kept 25/30 layers.
Step 2: Hidden dimension pruning → hidden size 576 → 504; intermediate size 1536 → 1344.
No fine-tuning yet.

Training Hyperparameters

No training performed. Model is raw after pruning.

Evaluation

Testing Data, Factors & Metrics

No evaluation performed post-pruning.

Results

Model reduced from 134.5M → ~93.4M parameters.
~30.5% reduction in size.
Accuracy and output quality degraded (requires fine-tuning).

Environmental Impact

Minimal, since no retraining has been done yet. Only pruning + saving.

Hardware Type: Single GPU (pruning experiment)
Hours used: <1
Cloud Provider: N/A
Carbon Emitted: Negligible

Technical Specifications

Model Architecture and Objective

Based on LLaMA decoder-only transformer.
Objective: next-token prediction (causal LM).
Modified architecture:
- Layers: 30 → 25
- Hidden size: 576 → 504
- Intermediate size: 1536 → 1344
- Attention heads: 9 (unchanged)
- Key/Value heads: 3 (unchanged)

Compute Infrastructure

Hardware: Single consumer GPU (e.g., RTX series)
Software: PyTorch, Hugging Face Transformers 4.57.0

abhinavv3/SmolLM-135M-Instruct-layer-width-pruned-90000000M-raw

Model Card for SmolLM-135M-Instruct-layer-width-pruned-90M-raw

Model Details

Model Description