Laguna M.1-base is the pre-trained base checkpoint for Laguna M.1, a 225B total parameter Mixture-of-Experts model with 23B activated parameters per token. This is the base model prior to post-training and reinforcement learning — it is a text-completion model with no instruction-following, reasoning, or tool-calling behavior. For agentic coding and chat use, use the post-trained Laguna M.1.

[!NOTE] For details on how we trained Laguna, check out our release blog post and technical report.

Highlights

Large sparse MoE: Laguna M.1 is a 70-layer MoE transformer with 225B total parameters and 23B activated parameters per token
High-capacity expert routing: After 3 dense SwiGLU layers, Laguna M.1 uses 67 sparse MoE layers with 256 experts, top-k=16 routing and auxiliary-loss-free load balancing
Global attention architecture: Laguna M.1 uses global attention across all layers with 64 Q-heads, 8 KV-heads and softplus attention output gating
Pre-instruct base checkpoint: Trained with pre-training only; intended as a starting point for further training rather than direct chat/agentic use
Apache 2.0 license: Use and modify freely for commercial and non-commercial purposes

Model overview

Stage: pre-training only (no post-training or reinforcement learning)
Number of parameters: 225B total with 23B activated per token
Optimizer: Muon
Layers: 70 layers with global attention
Experts: 256 experts with 1 shared expert; top-k=16 routing
Dense layers: first 3 layers are dense SwiGLU; remaining 67 layers are sparse MoE
Attention: 64 Q-heads, 8 KV-heads, head dimension 128, with softplus attention output gating
Positional encoding: RoPE with YaRN
Modality: text-to-text (completion)
Context window: 262,144 tokens

Usage

Laguna M.1-base is a text-completion model. It has no chat template, reasoning, or tool-calling support — serve it without the reasoning/tool-call parsers and prompt it with raw text.

vLLM

Laguna support is available in vLLM (v0.21.0 and later, vllm-project/vllm#41129).

shell

pip install 'vllm>=0.21.0'

vllm serve \
    --model poolside/Laguna-M.1-base \
    --served-model-name laguna-base

Query the completions endpoint from any OpenAI-compatible client:

python

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

completion = client.completions.create(
    model="laguna-base",
    prompt="def fibonacci(n):\n",
    max_tokens=128,
    temperature=0.7,
)
print(completion.choices[0].text)

SGLang

Laguna M.1-base is supported in SGLang via sgl-project/sglang#28400. As a completion model, serve it without the reasoning/tool-call parsers. A full serving recipe will be added here.

Transformers

Laguna is supported in Transformers v5.7.0 and later (huggingface/transformers#45673).

[!NOTE] Laguna M.1-base is a 225B-parameter model; loading the BF16 checkpoint in Transformers requires substantial multi-GPU memory (device_map="auto" shards across available devices). For single-node serving, vLLM is recommended.

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "poolside/Laguna-M.1-base"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

inputs = tokenizer("def fibonacci(n):\n", return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128, do_sample=True, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

License

This model is licensed under the Apache 2.0 License.

Intended and Responsible Use

Laguna M.1 is designed for software engineering and agentic coding use cases, and you are responsible for confirming that it is appropriate for your intended application. Laguna M.1 is subject to the Apache 2.0 License, and should be used consistently with Poolside's Acceptable Use Policy. We advise against circumventing Laguna M.1 safety guardrails without implementing substantially equivalent mitigations appropriate for your use case.

Please report security vulnerabilities or safety concerns to security@poolside.ai.