SOLARIS (PoC)

⚠️ Proof-of-Concept Disclaimer

This repository implements a research-oriented Proof of Concept (PoC) of the SOLARIS pipeline, including when executed on Modal infrastructure.

It is not a complete or production-optimized implementation of the full SOLARIS architecture as described in the paper. In particular, several components may be approximated, simplified, or lack low-level optimizations (e.g., custom kernels, fused operations, or hardware-specific tuning).

As such, all reported results should be interpreted as algorithmic validation signals, not as definitive performance benchmarks.

Do not treat current outputs as representative of final architecture-level performance, efficiency, or scaling behavior.

The purpose of this repository is to validate conceptual correctness, modular integration, and reproducibility, serving as a foundation for future large-scale and fully optimized implementations.

1. environment check
2. short LoRA training
3. evaluation on fixed benchmark slices
4. artifact packaging and manifesting

What This Repo Is

• A working engineering PoC you can run locally or on Modal.
• Focused on pipeline reliability, artifact auditability, and side-by-side model comparisons.
• Built around a small model workflow (TinyLlama/TinyLlama-1.1B-Chat-v1.0 by default).

What This Repo Is Not

• Not a full production 30B training stack.
• Not a benchmark-claim repo for state-of-the-art numbers.
• Not a substitute for the full research paper methodology.

Project Layout

• solaris_poc/modal_system.py: main end-to-end pipeline (env -> train -> evaluate -> bundle)
• solaris_poc/modal_fireproof.py: reproducible wrapper commands (prepare, manifest, pipeline)
• solaris_poc/modal_versus_report.py: generates detailed plots/tables for tinyllama_default vs solaris_system
• solaris_poc/attention/, moe/, compression/, continual/, world/: core PoC modules
• solaris_poc/tests/: tests
• artifacts/: generated run outputs and runbook files

Setup

Quick Start (Local)

• artifacts/modal_runs/<run_id>/

Quick Start (Modal)

• /runs/<run_id>/

Download Artifacts From Modal

Typical Output Files

• environment/: hardware/runtime checks
• train/summary/: train logs, train/eval CSV, metrics JSON, SVG charts
• evaluate/: benchmark JSONL/CSV, summaries, comparison charts
• manifest.json: hash + size registry of recorded files
• artifacts_bundle.tar.gz: packaged run artifacts

• evaluate/tinyllama_default_benchmark.csv
• evaluate/solaris_system_benchmark.csv
• evaluate/benchmark_slices.csv
• evaluate/comparison.csv

Run Detailed Versus Report

• artifacts/modal_runs/<run_id>/evaluate/versus/
• figures (*.png) + tabular summaries (prompt_level_versus.csv, aggregate_versus.csv)

Current Benchmark Snapshot (PoC)

• SOLARIS is currently slower and more verbose on this benchmark.
• Keyword quality is flat versus baseline in this run.

• In low-capacity regimes (around 1B parameters), error is often dominated by approximation limits, not representational inefficiency.
• In that setting, compression/transform pipelines can be approximately information-preserving (ΔE ≈ 0) but still not add quality.
• With short sequence lengths and small workloads, constant overhead dominates wall-clock latency, masking asymptotic gains from KV compression/quantization.

• This PoC result is useful, but it is not the target regime for SOLARIS benefits.
• The next step is evaluation in higher-context and/or higher-capacity setups, where representational redundancy is larger.
• Planned direction: test larger base models and longer-context runs to measure whether quality/efficiency tradeoffs improve in the intended regime.

Useful Environment Variables

• SOLARIS_MODAL_MODEL_ID
• SOLARIS_MODAL_RUN_ID
• SOLARIS_MODAL_TRAINING_MAX_STEPS
• SOLARIS_MODAL_TRAINING_SEQ_LEN
• SOLARIS_MODAL_TRAINING_GPU
• SOLARIS_MODAL_EVAL_GPU
• SOLARIS_MODAL_LORA_RANK
• SOLARIS_MODAL_DATASET_ID
• SOLARIS_MODAL_DATASET_MAX_SAMPLES
• SOLARIS_MODAL_SECRETS

Reliability Notes

• The pipeline writes a manifest plus a tar bundle for reproducibility.
• The manifest records file paths, sizes, and SHA-256 hashes.
• The bundle is the easiest single-file handoff for downstream analysis.

Troubleshooting

1. Confirm you downloaded the full run directory, not only manifest.json.
2. Check <run_id>/manifest.json for expected evaluate/*.csv entries.
3. If needed, pull artifacts_bundle.tar.gz and inspect contents locally.

1. Set SOLARIS_MODAL_SECRETS with your Modal secret name.
2. Re-run pipeline and confirm environment section in output summary.

Tests

Status Summary

• pipeline orchestration
• artifact generation and packaging
• baseline vs adapted evaluation comparisons
• module-level implementation across attention/routing/compression/continual/world helpers