Views
No views yet
config.json + model.safetensors) that real loaders accept.| Field | Value |
|---|---|
| Architecture | GPT2LMHeadModel |
| Params | ~43.9K |
| Hidden size | 32 |
| Layers | 2 |
| Heads | 4 |
| Vocab | 256 |
| Context | 64 |
| dtype | float32 |
| Weights file | model.safetensors (~174 KiB) |
1from transformers import AutoModelForCausalLM
2import torch
3
4model = AutoModelForCausalLM.from_pretrained(".")
5out = model(torch.zeros(1, 8, dtype=torch.long))
6print(out.logits.shape) # torch.Size([1, 8, 256])1from safetensors.torch import load_file
2state = load_file("model.safetensors")
3print(len(state), "tensors")torch.manual_seed(0), init range 0.02); LayerNorm/biases use the
conventional ones/zeros init.