Views
No views yet
Official AI Assistant of Snapgate AI
| Specification | Detail |
|---|---|
| Model Name | Snapgate Surge V3 |
| Parameters | ~7B |
| Fine-tuning Method | LoRA (r=16, alpha=32), 4-bit quantization |
| Target Modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Framework | Unsloth + TRL SFTTrainer |
| Languages | Indonesian & English |
| Feature | Description |
|---|---|
| 💻 Coding Assistant | PHP, JavaScript, Python, HTML/CSS, and other languages |
| 🌐 Web Development | Building, debugging, and optimizing websites |
| 💬 Bilingual Chat | Indonesian and English |
| 🧠 General Knowledge | Answers general questions about technology & AI |
| 🏷️ Consistent Persona | Always identifies itself as Surge from Snapgate AI |
1from unsloth import FastLanguageModel
2
3model, tokenizer = FastLanguageModel.from_pretrained(
4 model_name = "kadalicious22/snapgate-surge-v3-moe",
5 max_seq_length = 2048,
6 load_in_4bit = True,
7)
8FastLanguageModel.for_inference(model)
9
10messages = [
11 {"role": "system", "content": "You are Surge, an AI assistant from Snapgate AI."},
12 {"role": "user", "content": "Who are you?"},
13]
14inputs = tokenizer.apply_chat_template(
15 messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
16).to("cuda")
17
18outputs = model.generate(input_ids=inputs, max_new_tokens=256, temperature=0.7, top_p=0.9)
19print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))| Step | Training Loss |
|---|---|
| 10 | 1.788681 |
| 20 | 0.343310 |
| 30 | 0.151723 |
| 50 | 0.107738 |
| 60 | 0.086470 |
| 70 | 0.037520 |
| 80 | 0.026020 |
| 90 | 0.017798 |
| 100 | 0.015079 |
| 110 | 0.013477 |
| 120 | 0.012742 |
| 130 | 0.011483 |
| 140 | 0.010780 |
| 150 | 0.010536 |
| 160 | 0.010176 |
| 170 | 0.008932 |
| 180 | 0.009087 |
| 190 | 0.008987 |
| 200 | 0.009272 |
| 210 | 0.008632 |
| 220 | 0.008642 |
| 230 | 0.008501 |
| 240 | 0.009492 |
| 250 | 0.009299 |
| 260 | 0.009250 |
| 270 | 0.009292 |
| 280 | 0.009150 |
| 290 | 0.009015 |
| 300 | 0.008798 |