Views
No views yet
https://github.com/2sophia/skylar.Honest scope — read before using. This is a 236M domain model, intentionally small and undertrained by Chinchilla (~1.12B tokens ≈ 5× below compute-optimal for this size). It is an Italian specialist (English is not fluent) and it is not a factual oracle — it hallucinates open-domain facts. Its real, measured strength is grounded Italian tasks (answer/extract/classify from provided context). Published as a transparent reference for the Skylar framework, not as a general-purpose or SOTA model.
| Field | Value |
|---|---|
| Params | ~236M |
| Layers | 18 |
| d_model | 1024 |
| Heads (Q/KV) | 16 / 4 (GQA) |
| d_ff | 2816 (SwiGLU) |
| Context | 2048 |
| Vocab | 32,768 (ByteLevel BPE) |
| Pos. enc. | RoPE (θ=1e6) · QK-Norm · RMSNorm |
| License | Apache-2.0 |
| Task | Skylar-236M | random |
|---|---|---|
| XCOPA-it (causal commonsense) | 0.562 | 0.50 |
| HellaSwag-it | 0.292 | 0.25 |
| Belebele-it | 0.267 | 0.25 |
pip install skylar1import skylar # registers the architecture
2from transformers import AutoModelForCausalLM
3from tokenizers import Tokenizer
4from huggingface_hub import hf_hub_download
5import torch
6
7model = AutoModelForCausalLM.from_pretrained("Skyl4r-Ai/Skylar-236M-Base").eval()
8tok = Tokenizer.from_file(hf_hub_download("Skyl4r-Ai/Skylar-236M-Base", "tokenizer.json"))
9
10# base model -> raw text completion
11ids = torch.tensor([[tok.token_to_id("<bos>")] + tok.encode("Il regolamento DORA", add_special_tokens=False).ids])
12out = model.generate(ids, max_new_tokens=40, temperature=0.7, top_p=0.9, eos_token_id=[tok.token_to_id("<eos>")])
13print(tok.decode(out[0].tolist()))