Random-weight tiny dummy of the SolarOpen2 architecture, used for transformers CI integration tests. Not a trained model — outputs are meaningless.

Structure mirrors the public model at small scale: hybrid attention with the default gqa_interval=3-derived pattern ([full_attention, linear_attention x3] x3, 12 layers), NoPE full attention with output sigmoid gate, factored Kimi-Delta-Attention projections (head_dim 128), and a 16-expert top-4 MoE with one shared expert. Tokenizer files are copied from upstage/Solar-Open2-250B. Weights are randomly initialized with a fixed seed (torch.manual_seed(42)) in bfloat16 (617M parameters).