Views
No views yet
1python train_ultrathink.py \
2 --dataset c4 --streaming \
3 --hidden_size 768 --num_layers 12 \
4 --enable_moe --enable_dre \
5 --use_amp --gradient_checkpointing| Feature | ULTRATHINK | Others |
|---|---|---|
| Setup Time | ⚡ 5 minutes | 30-120 minutes |
| Lines to Train | 📝 ~10 | 50-100+ |
| MoE Support | ✅ Native | ❌ or Limited |
| Dynamic Reasoning | ✅ Unique | ❌ None |
| Constitutional AI | ✅ Built-in | ❌ None |
| Documentation | 📚 Comprehensive | Varies |
1# Clone repository
2git clone https://github.com/vediyappanm/UltraThinking-LLM-Training.git
3cd UltraThinking-LLM-Training/deep
4
5# Install dependencies
6pip install -r requirements.txt1python train_ultrathink.py \
2 --dataset wikitext \
3 --hidden_size 256 --num_layers 2 --num_heads 4 \
4 --batch_size 2 --max_samples 1000 \
5 --num_epochs 1python train_advanced.py --config configs/train_small.yaml1python train_ultrathink.py \
2 --dataset c4 --streaming \
3 --hidden_size 768 --num_layers 12 --num_heads 12 \
4 --enable_moe --enable_dre --enable_constitutional \
5 --use_amp --gradient_checkpointing \
6 --use_mlflow1# Run Gradio web interface
2docker compose up
3
4# Or build and run manually
5docker build -t ultrathink:latest .
6docker run -p 7860:7860 ultrathink:latest1# Run all tests
2pytest
3
4# Run with coverage
5pytest --cov=src --cov-report=html
6
7# Quick smoke test
8python tests/smoke_test.pydeep/
├── train_ultrathink.py # Main training script
├── train_advanced.py # YAML config-based training
├── app_gradio.py # Web UI for inference
├── src/
│ ├── models/ # UltraThink, MoE, DRE, architecture
│ ├── data/ # Datasets, tokenization, validation
│ ├── training/ # Optimizers, distributed, RLHF
│ ├── monitoring/ # Metrics and system monitoring
│ ├── security/ # Input validation and safety
│ └── evaluation/ # Benchmarks and metrics
├── tests/ # Unit and integration tests
├── configs/ # YAML configuration files
├── scripts/ # Utilities (profiling, inference)
└── docs/ # Documentation and guides1# WikiText-2 (fast iteration)
2python train_ultrathink.py \
3 --dataset wikitext \
4 --hidden_size 512 --num_layers 6 --num_heads 8 \
5 --batch_size 4 --num_epochs 3 \
6 --use_mlflow1# Streaming C4 with all optimizations
2python train_ultrathink.py \
3 --dataset c4 --dataset_subset en --streaming \
4 --hidden_size 768 --num_layers 12 --num_heads 12 \
5 --batch_size 2 --gradient_accumulation_steps 64 \
6 --learning_rate 3e-4 --warmup_steps 5000 \
7 --use_amp --gradient_checkpointing \
8 --max_seq_length 1024 \
9 --output_dir ./outputs/c4_production1# Small model (4-8GB GPU)
2python train_advanced.py --config configs/train_small.yaml
3
4# Medium model (16-32GB GPU)
5python train_advanced.py --config configs/train_medium.yaml
6
7# Large model (40GB+ GPU)
8python train_advanced.py --config configs/train_large.yaml1docker compose up
2# Visit http://localhost:78601docker run -v $(pwd)/outputs:/app/outputs ultrathink:latest \
2 python train_ultrathink.py \
3 --dataset wikitext \
4 --hidden_size 256 --num_layers 2 \
5 --output_dir /app/outputs/my_model1docker run --gpus all \
2 -v $(pwd)/outputs:/app/outputs \
3 ultrathink:latest \
4 python train_ultrathink.py --use_amp| Size | Parameters | Layers | Hidden | Context | Min GPU |
|---|---|---|---|---|---|
| Tiny | 125M | 12 | 768 | 2048 | 6GB |
| Small | 350M | 24 | 1024 | 4096 | 16GB |
| Medium | 760M | 24 | 1536 | 4096 | 24GB |
| Large | 1.3B | 32 | 2048 | 8192 | 40GB |
1@software{ultrathink2025,
2 title={ULTRATHINK: Advanced LLM Training Framework with Mixture-of-Experts and Dynamic Reasoning},
3 author={ULTRATHINK Team},
4 year={2025},
5 url={https://github.com/vediyappanm/UltraThinking-LLM-Training},
6 version={1.0.0}
7}
