Vex-Amber-Mini 1.1 is a groundbreaking small language model (SLM) that holds the world record for the most parameter-efficient model with fewer than 1 billion parameters. Meticulously optimized for code generation and general-purpose text tasks, it delivers exceptional performance within a compact 0.6B parameter framework.
To harness the power of Vex-Amber-Mini 1.1, install the required dependencies:
pip install transformers torch
Ensure Python 3.8+ and the latest versions of the required libraries for seamless compatibility.
Usage Example
Experience the model’s elegance with this example of generating a Python function:
python
1from transformers import AutoModelForCausalLM, AutoTokenizer
23# Initialize the tokenizer and model4tokenizer = AutoTokenizer.from_pretrained("Arioron/Vex-Amber-Mini-1.0")5model = AutoModelForCausalLM.from_pretrained("Arioron/Vex-Amber-Mini-1.0")67# Craft the input prompt8prompt ="Write a Python function to compute Fibonacci numbers:"9inputs = tokenizer(prompt, return_tensors="pt")1011# Generate refined output12outputs = model.generate(**inputs, max_length=100, temperature=0.7)1314# Decode and present the result15print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Model Details
Architecture: Transformer-based, derived from Qwen3-0.6B
Training Data: Fine-tuned on a curated dataset optimized for code generation and versatile text tasks
Performance: Achieves a HumanEval Pass@1 score of 20.12%, setting a benchmark for sub-1B models and earning the title of the most parameter-efficient sub-1B model
Use Cases: Ideal for code generation, text completion, and lightweight NLP applications
Context Length: Supports up to 2048 tokens for efficient processing
Performance Metrics
The following table compares the HumanEval performance of Vex-Amber-Mini 1.1 against other code generation models. Note that scores for rival models are approximate, as indicated by "~", based on available benchmarks:
Model
Parameters
HumanEval Pass@1
Notes
Vex-Amber-Mini 1.0
0.6B
20.21%
Compact model optimized for code generation.
Code Llama
7B
~24%
Developed by Meta, optimized for code tasks.
StarCoder
7B
~25%
Developed by Hugging Face and ServiceNow, fine-tuned for code.
CodeGen
6B
~22%
Developed by Salesforce, optimized for code generation.
CodeT5
3B
~20%
Developed by Google, fine-tuned for code tasks.
PolyCoder
12.7B
~28%
Developed by Berkeley, optimized for code generation.
Note: The HumanEval Pass@1 score reflects the model's ability to generate correct code solutions on the first attempt. Vex-Amber-Mini 1.0 achieves competitive performance for its size, outperforming larger models in parameter efficiency.
Limitations
Compact Design: The 0.6B parameter count, while highly efficient, may limit performance on highly complex tasks compared to larger models.
Domain-Specific Fine-tuning: Optimal results may require additional tuning for specialized applications.
Context Constraints: Limited to 2048 tokens, which may impact performance in extended context scenarios.
License
This project is proudly licensed under the Apache 2.0 License, ensuring open and flexible usage.
