AQ is a 1.26B-parameter academic tutor built completely from scratch by Zyora Labs — proprietary architecture, own training code (pure PyTorch), own tokenizer, own data pipeline, own tutor fine-tune. No fine-tune of any existing model.

This is the tutor (instruct) model: it answers student questions directly with explanations, numbered steps, and worked examples. The pretrained base model is available at zyoralabs/AQ-academic-ai-base.

Training

Pretraining — 20B tokens, knowledge-dense and concept-first: real textbooks, course notes, scientific papers, encyclopedic text, mathematical reasoning — in English, Tamil, and Hindi. Grown progressively 75M → 300M → 1.26B, finished with a quality anneal (LR → 0 on the highest-quality academic text).
Tutor fine-tune — 400M tokens of educator-style instruction data (explanations, step-by-step math, knowledge Q&A, Hindi instructions), loss masked to tutor responses.

Prompt format

### Student:
{your question}

### Tutor:

python

from transformers import AutoModelForCausalLM, AutoTokenizer

# Pin a released revision so the loaded code + weights are immutable
# (trust_remote_code executes this repo's modeling files).
REV = "v2.0"

tok = AutoTokenizer.from_pretrained("zyoralabs/AQ-academic-ai", revision=REV)
model = AutoModelForCausalLM.from_pretrained(
    "zyoralabs/AQ-academic-ai", revision=REV, trust_remote_code=True)

prompt = "### Student:\nWhat is a stack in data structures?\n\n### Tutor:\n"
ids = tok(prompt, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=200, do_sample=True,
                     temperature=0.7, top_k=40, repetition_penalty=1.3)
print(tok.decode(out[0][ids.shape[1]:]))

Tested environment: transformers==4.53.0, torch==2.7.1, tokenizers==0.21, dtype bfloat16, single GPU or CPU.

Architecture (proprietary, from scratch)


Parameters	1.26B
Layers	48
Hidden size	1536
Attention heads	24 (grouped-query, 8 KV heads)
Feed-forward	SwiGLU, 4096
Positional encoding	Rotary (RoPE)
Normalization	RMSNorm
Context length	2048
Vocabulary	32,000 (byte-level BPE, English + Tamil + Hindi)

Benchmarks (0-shot, lm-evaluation-harness)

Benchmark	AQ v2 Tutor	Notes
SciQ	70.0	strong science knowledge for the size/data budget
PIQA	61.9
ARC-easy	45.2
Winogrande	50.1
HellaSwag	29.6
MMLU	25.2	at-chance, like all ~1B-class models
ARC-challenge	20.8

Reproducing these numbers

Scores were produced with lm-evaluation-harness 0.4.8 (pip install lm-eval==0.4.8), 0-shot, default task configs, on a single H100 (bf16):

bash

lm_eval --model hf \
  --model_args pretrained=zyoralabs/AQ-academic-ai,revision=v2.0,trust_remote_code=True,dtype=bfloat16 \
  --tasks mmlu,arc_easy,arc_challenge,hellaswag,piqa,winogrande,sciq \
  --batch_size auto

Environment: transformers==4.53.0, torch==2.7.1 (cu128), datasets==3.2.0, accelerate. The machine-readable harness output is published in this repo at eval/aq_v2_tutor_results.json. Reported metric is acc (see the artifact for acc_norm and per-subtask MMLU results).

For context: models of this size trained on 15×–150× more tokens (e.g. 300B–3T) reach SciQ ~84–89. AQ reaches ~70 on just 20B tokens — the concept-first, knowledge-dense corpus is the point.

Transparency: as a 1.26B model, AQ v1 has real limits — multi-step arithmetic word problems and MMLU-style abstract reasoning are weak (these unlock at larger scale, on our roadmap). In the AQ product, answers are additionally grounded with retrieval over real study material.

Intended use

The tutor layer of the AQ educator stack. Best used with the Student/Tutor prompt format, sampling enabled, and (in production) retrieval grounding over curriculum material.

Team

Name	Role	Affiliation
Vasanth	Chief AI Researcher	Zyora Labs
Adithi Sreedhar	Jr AI Engineer	AI & DS, Arunachala College of Engineering for Women

About

Built in India by Zyora Labs. AQ v1 is the first release of the Academic Quotient model family.