Views
No views yet
| Total Parameters | ~3 Billion |
| Active Parameters | ~1.1 Billion (2 Experts/Token) |
| Architecture | Mixture of Experts (MoE) |
| Number of Experts | 4 |
| Context Length | 8K Tokens |
| Training Data | 40B Tokens (SKT-OMNI-CORPUS-2T) |
pip install transformers accelerate torch peft bitsandbytes1from transformers import AutoModelForCausalLM, AutoTokenizer
2import torch
3
4model_id = "sKT-Ai-Labs/SKT-ST-X-0-3B"
5
6tokenizer = AutoTokenizer.from_pretrained(model_id)
7model = AutoModelForCausalLM.from_pretrained(
8 model_id,
9 device_map="auto",
10 torch_dtype=torch.float16
11)
12
13prompt = "What is Quantum Physics?"
14formatted = f"<|user|>\n{prompt}\n<|assistant|>\n"
15inputs = tokenizer(formatted, return_tensors="pt").to("cuda")
16
17outputs = model.generate(**inputs, max_new_tokens=100)
18response = tokenizer.decode(outputs[0], skip_special_tokens=True)
19print(response.split("<|assistant|>")[-1].strip())1from transformers import BitsAndBytesConfig
2
3quant_config = BitsAndBytesConfig(
4 load_in_4bit=True,
5 bnb_4bit_compute_dtype=torch.float16)
6
7model = AutoModelForCausalLM.from_pretrained(
8 model_id,
9 quantization_config=quant_config,
10 device_map="auto"
11)1@misc{SKT-ST-X-0-3B,
2 author = {SKT AI LABS, India},
3 title = {SKT-ST-X-0-3B: A Compact Mixture of Experts Model},
4 year = {2026},
5 publisher = {Hugging Face},
6 url = {https://huggingface.co/sKT-Ai-Labs/SKT-ST-X-0-3B}
7}