RubiRLM-1B-Base is a 1B-parameter base language model released by DevHunterAI.
Model size: 1B parameters
Training datasets: FineWeb, UltraChat-200k
Model type: Base / pretrained language model
Important: This release is a base model. It can be used for prompt-based generation and experimental chat-style interaction, but it is not an instruction-tuned chat assistant.
Architecture
RubiRLM 1B Architecture
RubiRLM 1B uses a recursive language modeling architecture with recurrent state flow, Mixture-of-Experts routing, and conditional block execution.
Key Features
1B parameters
Recursive Language Model (RLM) architecture
10 recursive blocks
d_model = 1024
16 attention heads
max sequence length = 2048
6 recursive reasoning steps
Mixture-of-Experts: 32 experts, top-1 routing
Layer skip router for conditional execution
Packed execution support
Tied token embedding and LM head
Training Data
This model was trained using a mixture of:
FineWeb
UltraChat-200k
Intended Usage
This model is intended for:
base language modeling research
continued pretraining
experimental prompt-based generation
architecture experimentation around recursive and MoE-based language models
Not Intended As
This release should not be treated as:
a fully aligned assistant
a safety-tuned production chatbot
an instruction-following model with guaranteed conversational quality
Loading
Because this repository includes custom model code, loading may require trust_remote_code=True depending on your workflow.
Files
pytorch_model.bin: exported RubiRLM weights
training_checkpoint.pt: original training checkpoint
config.json: Hugging Face-facing config
rubirlm_config.json: full RubiRLM architecture config
