Violet is a GPT-NeoX language model trained primarily on period texts (1800–1899). This is the completion version of the model, so if you were looking for the Chat version, you should check out Violet 1b4 Chat
It is intended for creative writing, roleplay, period-appropriate correspondence, and Victorian etiquette.
Architecture: GPTNeoXForCausalLM
Parameters: ~1.41B
Context length: 4096
Vocab size: 24014
Tokenizer: PreTrainedTokenizerFast
Intended Use
Good for
Victorian-flavored narrative completions
Not good for
Contemporary factual Q&A
Medical/legal/financial advice
Known Issues / Limitations
Ages and dates can be unreliable (even within 1800–1899).
Because parts of the corpus were derived from OCR, occasional stray modern tokens may appear (e.g., “http”, “Google”, “Internet Archive”).
Training data includes UK and US English from the era.
Notes
Violet is not the first LLM trained on a historical-only pretraining corpus; to the author’s knowledge that distinction belongs to TimeCapsuleLLM. Violet was developed independently, and differs in:
Different (but somewhat overlapping) pretraining corpus and a different range of dates -- Violet focuses specifically on 1800-1899
A custom Victorian tokenizer
Violet was built on a corpus spanning 1800–1899 sourced from Project Gutenberg, the Internet Archive, the British National Library, and other archives.
This project began as an attempt to build a local LLM without relying on copyrighted training sources. The author also values local models that can run on a user’s machine without sending data to the cloud.
Both of these are intended to use WebGPU and run local on your system -- No data is sent to the cloud.
Related repos
Zakarth/violet-1b4 (base/completion)
Zakarth/violet-1b4-chat-onnx (WebGPU INT8)
Prompt Format (Chat)
This model was trained to generate a mood line + assistant tag + response after <|violet_mood|>.
Use this structure:
The morning fog had scarcely lifted when
The model will then generate:
{response...}
Tokenization and Special Tokens
Violet 1b4 was trained on a custom tokenizer specific for Victorian text.
Recommended IDs for generation:
eos_token_id: 0
pad_token_id: 1
Special tokens used during training (typical IDs from training config):
<|system|>: 24000
<|user|>: 24001
<|assistant|>: 24002
<|violet_mood|>: 24005
!! Do not mix tokenizers from other Violet variants (e.g. 160M) with this model.
How to use (Transformers.js)
code
1from transformers import AutoTokenizer, AutoModelForCausalLM
2import torch
34repo = "Zakarth/violet-1b4-chat"
5tok = AutoTokenizer.from_pretrained(repo, use_fast=True)
6model = AutoModelForCausalLM.from_pretrained(repo, device_map="auto")
78prompt = "The morning fog had scarcely lifted when"""
910inputs = tok(prompt, return_tensors="pt", add_special_tokens=False)
11# Filter out token_type_ids if present
12inputs = {k: v.to(model.device) for k, v in inputs.items() if k in ['input_ids', 'attention_mask']}
1314out = model.generate(
15 **inputs,
16 max_new_tokens=180,
17 do_sample=True,
18 temperature=0.8,
19 top_p=0.9,
20 top_k=40,
21 repetition_penalty=1.15,
22 eos_token_id=0,
23 pad_token_id=1,
24)
25print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False))
Sample Outputs
The morning fog had scarcely lifted when
text
1The morning fog had scarcely lifted whenthe first
2light streamed through the window, and before it was quite light the
3flood of rain came on with a suddenness which seemed to scorch up the
4roof. The lightning was as bright as ever, but there were only three or
5four flashes in the sky--a bright flash like a meteor--and the thunder
6was not so loud nor so deafening as usual.
78At last the storm ceased. The storm was over; the stars shone out; the
9thunder rolled away, leaving the clouds behind it in an impenetrable
10haze, which at once became visible, and soon they disappeared. The wind
11blew with fury, driving the snow and ice from off the roofs of the houses
License
Model weights and code in this repository are released under CC0 1.0 (public domain dedication).
