This file is strictly intended for saving the initial weights (checkpoint) of the JiRack GPT model.
The model is "clean": it contains no data and has never undergone any pre-training.
Powered by CMS Manhattan’s cutting-edge Vision-BERT architecture.
It is engineered to be a maximally safe and robust base for training from scratch for specialized, smaller models, such as:
SPAM Detection Systems
FRAUD Detection Models
Background Check (BG Check) Models
A product of CMS Manhattan.
Tokenizer Choices
For English: GPT-2 Hugging Face tokenizer
For multilingual use: BERT tokenizer from the Hugging Face library
Model Architecture Details
GPT-2 Architecture (Classic, Transformer-like)
CustomEmbedding
FrozenSignatureLayer
LearnedPositionalEmbedding
[TransformerBlock]
├── MultiHeadAttention
├── LayerNorm
├── LayerNorm
├── FFN
├── Linear
├── Activation: GELU
└── Linear
LayerNorm
Linear
