Built on Qwen2.5-3B-Instruct, Ult1.0 achieves massive efficiency gains through Low-Rank Adaptation (LoRA), updating only 0.12% of parameters while preserving the base model's full capability.

GGUF (CPU-Optimized) Inference

The repository includes a Q8_0 quantized GGUF file for ultra-fast CPU inference with llama.cpp, Ollama, LM Studio, or any GGUF-compatible runner:

File	Size	Format	Quality
`Ult1.0-Q8_0.gguf`	3.29 GB	Q8_0 (8-bit)	Near-lossless

llama.cpp

./llama-cli -m Ult1.0-Q8_0.gguf -p "Write a poem about AI" -n 256

Ollama (import from GGUF)

bash

ollama create ult1.0 -f Modelfile
# Modelfile content: FROM ./Ult1.0-Q8_0.gguf
ollama run ult1.0

Python (llama-cpp-python)

python

from llama_cpp import Llama
llm = Llama("Ult1.0-Q8_0.gguf", n_ctx=32768)
output = llm("Write a poem about AI", max_tokens=256)
print(output["choices"][0]["text"])

Transformers (GPU) Inference

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("teolm30/Ult1.0", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("teolm30/Ult1.0")

messages = [{"role": "user", "content": "Explain quantum computing simply"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

1000× Efficiency Benchmark

Metric	Full Fine-Tune	Ult1.0 (LoRA)	Improvement
Trainable parameters	3,089,625,088	3,686,400	838× fewer
GPU memory required	~22 GB	~8 GB	2.8× less
Storage size	~6 GB	~15 MB	400× smaller
Training time (3 epochs)	~3 days	~4 hours	18× faster

Train Your Own (GPU)

Fine-tune on any GPU with ≥8 GB VRAM:

bash

pip install transformers datasets peft accelerate
python train.py

Model Details

Property	Value
Base Model	Qwen/Qwen2.5-3B-Instruct
Total Parameters	3,089,625,088
LoRA Parameters	3,686,400 (0.12%)
LoRA Rank	8
Context Length	32,768 tokens
Architecture	Transformer with RoPE, SwiGLU, Grouped Query Attention

Fox-AI-by-teolm30/Ult1.0

Ult1.0