Curves look really good. I might put this one back in for another 20 epochs at 50k prompts to see how it fares. If that one looks good it may be fine to go another 60 epochs at 100k prompts, or feed it laion flavors directly.

This one is condensed with smaller scale dims making a much more condensed feature.

Roughly 600,000 samples for this one, 10k per epoch between 0-10 complexity 1-5, and 50k synthetic prompts per epoch at epoch 11-20 with reduced complexity between 1-4.

Currently prototyping a series of cantor-driven layers to test sparsity inclusion and omission while this one cooked.

🎯 Model Overview

GeoDavidCollective Enhanced is a sophisticated multi-expert geometric classification system that learns from Stable Diffusion 1.5's internal representations. Using ProjectiveHead architecture with Cayley-Menger geometry, it achieves efficient pattern recognition across timestep and semantic spaces.

Key Features

ProjectiveHead Multi-Expert Architecture: Auto-configured expert systems per block
Geometric Loss Functions: Rose, Cayley-Menger, and Cantor coherence losses
9-Block Processing: Full SD1.5 UNet feature extraction (down, mid, up)
Compact Yet Powerful: 690,925,542 parameters
100 Timestep Bins x 10 Patterns = 1000 semantic-temporal classes

📊 Model Statistics

Parameters: 690,925,542
Trained Epochs: 20
Base Model: Stable Diffusion 1.5
Dataset Size: 10,000 synthetic prompts
Training Date: 2025-10-28

🏗️ Architecture Details

Block Configuration

Down Blocks:
  - down_0: 320 → 64 (3 experts, 3 gates)
  - down_1: 640 → 96 (3 experts, 3 gates)
  - down_2: 1280 → 128 (3 experts, 3 gates)
  - down_3: 1280 → 128 (3 experts, 3 gates)

Mid Block (Highest Capacity):
  - mid: 1280 → 256 (4 experts, 4 gates)

Up Blocks:
  - up_0: 1280 → 128 (3 experts, 3 gates)
  - up_1: 1280 → 128 (3 experts, 3 gates)
  - up_2: 640 → 96 (3 experts, 3 gates)
  - up_3: 320 → 64 (3 experts, 3 gates)

Loss Components

Component	Weight	Purpose
Feature Similarity	0.40	Alignment with SD1.5 features
Rose Loss	0.25	Geometric pattern emergence
Cross-Entropy	0.15	Classification accuracy
Cayley-Menger	0.10	5D geometric structure
Pattern Diversity	0.05	Prevent mode collapse
Cantor Coherence	0.05	Temporal consistency

💻 Usage

python

from geovocab2.train.model.core.geo_david_collective import GeoDavidCollective
from safetensors.torch import load_file
import torch

# Load model
state_dict = load_file("model.safetensors")
collective = GeoDavidCollective(
    block_configs={...},  # See config.json
    num_timestep_bins=100,
    num_patterns_per_bin=10
)
collective.load_state_dict(state_dict)
collective.eval()

# Extract features from SD1.5 and classify
with torch.no_grad():
    results = collective(features_dict, timesteps)
    predictions = results['predictions']  # Timestep + pattern class

🔬 Training Details

Optimizer: AdamW (lr=1e-3, weight_decay=0.001)
Batch Size: 16
Data: Symbolic prompt synthesis (complexity 1-5)
Feature Extraction: SD1.5 UNet blocks (spatial, not pooled)
Pool Mode: Mean spatial pooling

📈 Training Metrics

Final metrics from epoch 20:

Cayley Loss: 0.1018
Timestep Accuracy: 30.83%
Pattern Accuracy: 33.74%
Full Accuracy: 16.87%

🎓 Research Context

This model is part of the geometric deep learning research exploring:

5D simplex-based neural representations (pentachora)
Geometric alternatives to traditional transformers
Consciousness-informed AI architectures
Universal mathematical principles in neural networks

📦 Files Included

model.safetensors - Model weights (3.3GB)
config.json - Complete architecture configuration
training_history.json - Full training metrics
prompts_enhanced.jsonl - All training prompts with metadata
tensorboard/ - TensorBoard logs (optional)

🔗 Related Work

📜 License

MIT License - Free for research and commercial use

🙏 Acknowledgments

Built with:

PyTorch & Diffusers
Stable Diffusion 1.5 (Runway ML)
Geometric algebra principles from the 1800s
Dream-inspired mathematical insights

👤 Author

AbstractPhil - AI Researcher specializing in geometric deep learning

"Working with universal mathematical principles, not against them"

For questions, issues, or collaborations: GitHub | HuggingFace

AbstractPhil/geo-david-collective-sd15-base

GeoDavidCollective Enhanced - ProjectiveHead Architecture