You can deploy this model and start earning money today!

xxrickyxx/
ailo-152m

Name: ailo-152m
Availability: InStock

Views

No views yet

AILO-152M: Transformer Language Model

AILO (Artificial Intelligence Language Operator) is a 152M parameter Transformer-based language model trained from scratch.

Model Details

Property	Value
Parameters	151.9M
Architecture	Decoder-only Transformer
Layers	12
Hidden Size	768
Attention Heads	12
Context Length	512 tokens
Vocabulary	50,257 (GPT-2 tokenizer)

Training

Dataset: FineWeb-Edu (100B token sample, streamed)
Training Steps: 182,000+
Final Loss: ~3.0
Training Time: ~64 hours
Optimizer: AdamW with cosine LR schedule + warm restarts

Training Loss Curve

Quick Start

python

# Install dependencies
pip install torch transformers tiktoken huggingface_hub

# Download and use
from huggingface_hub import hf_hub_download
import torch
import sys

# Download model files
repo_id = "xxrickyxx/ailo-152m"
for f in ["config.json", "configuration_ailo.py", "modeling_ailo.py", "pytorch_model.bin"]:
    hf_hub_download(repo_id=repo_id, filename=f, local_dir="ailo_model")

# Load model
sys.path.insert(0, 'ailo_model')
from configuration_ailo import AILOConfig
from modeling_ailo import AILOForCausalLM
import tiktoken

config = AILOConfig.from_pretrained("ailo_model")
model = AILOForCausalLM(config)
state_dict = torch.load("ailo_model/pytorch_model.bin", map_location='cpu')
model.load_state_dict(state_dict, strict=False)
model.eval()

# Generate text
tokenizer = tiktoken.get_encoding("gpt2")
prompt = "What is artificial intelligence?"
tokens = tokenizer.encode(prompt)
input_ids = torch.tensor([tokens])

with torch.no_grad():
    output_ids = model.generate(input_ids, max_new_tokens=100, temperature=0.8)

print(tokenizer.decode(output_ids[0].tolist()))

🏆 AILO vs GPT-2 Arena Comparison

We compared AILO-152M against GPT-2 (124M) on various prompts. Despite similar size, AILO shows better coherence and fewer repetitions.

Example 1: What is artificial intelligence?

Model	Response
AILO ✅	"The term artificial intelligence refers to a range of diverse fields of research that focuses on the ability of human beings to understand complex systems, perform complex tasks, and perform complex operations..."
GPT-2 ❌	"How do you find out? What do you do when you're out in the field? The answer is, you have to do what you know..."

Example 2: Write a short story about a robot

Model	Response
AILO ✅	"But the robot has no control of the robot itself. It uses the robot's hand to drive it. The robot is able to read the information about the robot..."
GPT-2 ❌	"Write a short story about a robot. Write a short story about a robot. Write a short story about a robot..." (infinite repetition)

Example 3: Tell me a joke

Model	Response
AILO ✅	"I think I could have made the joke. But, it's just really bad. I have never made a joke. It's only a joke..."
GPT-2 ❌	"It is not funny. It is not funny. It is not funny. It is not funny. It is not funny..." (infinite repetition)

Summary

Metric	AILO-152M	GPT-2 (124M)
Parameters	151.9M	124.4M
Coherence	✅ Better	⚠️ Often loses track
Repetition	✅ Rare	❌ Frequent
Training Time	64 hours	Weeks

Intended Uses

Text generation
Fine-tuning for specific domains
Educational purposes
Research on small language models

Limitations

Small model size (152M) limits capabilities compared to larger models
May produce repetitive or incoherent text for complex queries
Training data primarily in English

Files

File	Description
`config.json`	Model configuration
`configuration_ailo.py`	Config class
`modeling_ailo.py`	Model architecture
`pytorch_model.bin`	Model weights (607 MB)
`AILO_Demo.ipynb`	Colab notebook

Citation

bibtex

@misc{ailo2026,
  title={AILO-152M: A Small Transformer Language Model},
  author={AILO Team},
  year={2026},
  howpublished={\url{https://huggingface.co/xxrickyxx/ailo-152m}}
}

📜 License This project uses a dual-license model.

🆓 Non-Commercial License Released under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 (CC BY-NC-SA 4.0).

You are free to:

Use the model for research, education, and personal projects Modify and fine-tune the model Redistribute derivatives under the same license You must:

Provide attribution Keep the same license for derivative works Not use the model for commercial purposes 💼 Commercial License Commercial use of AILO-152M is not permitted under the free license. Commercial use includes:

Integration into paid products or services Use in SaaS platforms, APIs, or enterprise systems Any application that generates revenue directly or indirectly For commercial licensing, a separate paid agreement (royalty or license fee) is required. Please contact the author.

📬 Contact For research collaboration or commercial licensing inquiries, contact the project maintainer:

Riccardo Sparacino LinkedIn https://www.linkedin.com/in/riccardo-sparacino-developer-php-javascript-mysql-app-ios-android/it/?skipRedirect=true