Views
No views yet
| Metric | Value |
|---|---|
| Throughput | ~35 tokens/sec (RTX 3050, 6GB VRAM) |
| Avg Latency | ~4-5s per response |
| Success Rate | 100% (5/5 tasks) |
| Tokens/Response | ~150 avg |
| MMLU (ref) | ~72% |
| GSM8K (ref) | ~58% |
| HumanEval (ref) | ~55% |
| Task | Prompt | Check | Result |
|---|---|---|---|
| Math | "A farmer has 17 sheep. All but 9 run away. How many sheep left?" | 9 | ✅ |
| Coding | "Write a Python function to check if a number is prime." | def | ✅ |
| Knowledge | "What is the capital of Greece?" | athens | ✅ |
| Logic | "If all cats are animals and some animals are pets, then some cats are pets. True or false?" | true | ✅ |
| Translation | "Translate to Greek: Hello, how are you?" | γεια | ✅ |
| Property | Value |
|---|---|
| Base Model | Qwen2.5-7B-Instruct |
| Quantization | GPTQ 4-bit |
| Parameters | 7B |
| Context Length | 32K tokens |
| Size | 5.3GB |
| VRAM Required | ~6GB |
| License | Apache 2.0 |
1from transformers import AutoTokenizer, AutoModelForCausalLM
2
3model_name = "teolm30/Fox-1.5"
4tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
5model = AutoModelForCausalLM.from_pretrained(
6 model_name,
7 torch_dtype=torch.bfloat16,
8 device_map="auto"
9)
10
11messages = [{"role": "user", "content": "Explain quantum entanglement in simple terms"}]
12text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
13inputs = tokenizer(text, return_tensors="pt").to("cuda:0")
14outputs = model.generate(**inputs, max_new_tokens=512)
15print(tokenizer.decode(outputs[0], skip_special_tokens=True))pip install auto-gptq optimumollama run hf.co/teolm30/Fox-1.5