You can deploy this model and start earning money today!

VERBAREX/
LuminoLex-1.5B-think

Name: LuminoLex-1.5B-think
Availability: InStock

Views

No views yet

LuminoLex-1.5B-think

LuminoLex-1.5B-think is a compact, high-efficiency causal language model developed by VERBAREX, an artificial intelligence and robotics laboratory.

The model combines Grouped Query Attention (GQA) with a Latent Mixture-of-Experts (MoE) decoder architecture, featuring 1.5B total parameters while activating only 360M parameters per token during inference.

Model Specifications

Specification	Value
Model Name	LuminoLex-1.5B-think
Developer	VERBAREX
Architecture	Multihead Latent MoE Decoder (`luminolex`)
Layers	18
Active Parameters	360,000,000 (360M)
Total Parameters	~1.46 Billion
Attention Mechanism	Fused QKV GQA (24 Query Heads, 8 Key/Value Heads)
Head Dimension	64
Vocabulary Size	32,800
Context Length	1,024 tokens
Default Precision	bfloat16
License	Apache License 2.0

Standard Benchmark Results

Evaluated on industry-standard Open LLM Leaderboard benchmarks using cloud hardware (NVIDIA A10G):

Benchmark	Score	Evaluation Setting	Metric
ARC-Challenge	27.0%	AI2 Reasoning Challenge (Hard subset)	Accuracy
HellaSwag	35.67%	Commonsense Reasoning & Completion	Accuracy (Norm)
MMLU	21.33%	Massive Multitask Language Understanding	5-shot Accuracy
TruthfulQA	24.5%	Truthfulness & Hallucination Resistance	MC1 Accuracy

Thinking & Reasoning Mode

LuminoLex-1.5B-think supports structured chain-of-thought reasoning activated by the <think> token. For multi-step arithmetic, scientific reasoning, or logic puzzles, appending <think> to the assistant prompt engages step-by-step problem decomposition:

python

prompt = "<bos>User: Solve 27 times 14.\nAssistant: <think>"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=160, temperature=0.6)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
# Step breakdown: 27 x 10 = 270. 27 x 4 = 108. 270 + 108 = 378.

Quickstart

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "VERBAREX/LuminoLex-1.5B-think"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

prompt = "<bos>User: Explain why the sky appears blue step by step.\nAssistant: "
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=128, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

License

This project is licensed under the Apache License 2.0. See the LICENSE file for details.