This model was trained on exactly 100 million tokens for the BabyLM Challenge.
Key Features
BitNet Quantization: 1.58-bit quantized text encoder/decoder for efficient inference. Vision Processing: Quantized Vision Transformer using pre-computed DiNOv2 features. Episodic Memory: Human Brain inspired memory mechanism for learning, without fine-tuning.
