You can deploy this model and start earning money today!

qvx-o/
QED-Base-v2

Name: QED-Base-v2
Availability: InStock

Views

No views yet

QED-Base-v2

QED-Base-v2 is a ~107M parameter causal language model pretrained from scratch by Qarvexium. It is a base model — it has not been instruction-tuned or aligned for chat, and will continue text rather than follow instructions or hold a conversation.

Model Details

Developed by: Qarvexium
Model type: Decoder-only causal language base model
Language: English
License: MIT
Tokenizer: QED-B1 tokenizer (SentencePiece BPE, 48,000 vocabulary)

Architecture

Component	Value
Tokenizer	QED-B1 tokenizer
Vocabulary size	48,000
Model type	Decoder-only Transformer
Parameters	~107M
Hidden size	768
Layers	12
Attention heads	12
KV heads	4
Attention	GQA
Intermediate FFN size	1792
Activation	SwiGLU
Normalization	RMSNorm
Position encoding	RoPE
Context length	2048
RoPE theta	10000

Weight-tied embeddings/LM head.

Uses

Direct Use

As a base model, QED-Base-v2 is intended for:

Text completion / continuation
Research on small-scale language model pretraining
A starting checkpoint for further fine-tuning (instruction tuning, domain adaptation, etc.)

Out-of-Scope Use

This model has not been instruction-tuned, RLHF'd, or safety-aligned. It should not be deployed directly in a chat or assistant product, or in any application where reliable instruction-following or content moderation is required, without further fine-tuning and evaluation.

Bias, Risks, and Limitations

QED-Base-v2 was trained on a large web-crawled corpus and will reflect the biases, inaccuracies, and occasionally toxic content present in that data. As a base model it has no built-in refusal behavior or safety tuning — outputs should be filtered/evaluated before use in any user-facing setting. At ~107M parameters, factual accuracy and reasoning ability are limited compared to larger models.

How to Get Started

python

from infer import load_model, load_tokenizer, run

model = load_model("QED-Base-v2.pt")
tokenizer = load_tokenizer("tok.model")

text = run("Once upon a time", model, tokenizer, max_new_tokens=100)
print(text)

License

MIT License

qvx-o/QED-Base-v2