You can deploy this model and start earning money today!

jsoul/
geo-perturbation-grouping-qwen3-30b-a3b

Name: geo-perturbation-grouping-qwen3-30b-a3b
Availability: InStock

Views

No views yet

GEO Gene-Perturbation Grouping — Qwen3-30B-A3B (merged, MoE)

A fine-tune of Qwen/Qwen3-30B-A3B-Instruct-2507 that reads the sample metadata of an NCBI GEO gene-expression series and organizes its samples into valid genetic-perturbation case/control experimental groups (methods KD / KO / OE, ≥2 controls and ≥2 cases, single wild-type HGNC target gene, one cell line, matched time points), or reports No valid groups found.

This is the larger Mixture-of-Experts sibling of jsoul/geo-perturbation-grouping-qwen3-4b. The LoRA adapter has been merged into the base weights, so this is a standalone model (bfloat16, safetensors, 16 shards) usable directly with transformers, vLLM, TGI, etc.

Model details


Base model	`Qwen/Qwen3-30B-A3B-Instruct-2507` (Qwen3 MoE)
Architecture	`Qwen3MoeForCausalLM` — 48 layers, hidden 2048, 128 experts, 8 active per token
Parameters	~30B total / ~3B active per token
Attention	GQA, 32 query / 4 KV heads; 262k context
Precision	bfloat16
Format	merged full weights, `safetensors` (16 shards, ~57 GB)

Training


Method	LoRA (merged), via Unsloth 2025.8.5 (`transformers` 4.55.1)
LoRA rank / alpha	16 / 16
Learning rate	2e-4
Epochs	2
Seed	3407
Training data	`jsoul/geo-perturbation-grouping-train` — `train` split only (2,400 examples)

The companion validation split (600 examples) and the held-out jsoul/geo-perturbation-grouping-test set (300 examples) were not used for training. GEO accessions are disjoint across train / validation / test, so the test set is a clean leakage-free benchmark.

Prompt format

The model was trained on single-turn chats: a fixed system rulebook, a user message containing the study metadata, and an assistant target. For best results, use the exact system prompt from the training dataset (ds["train"][0]["messages"][0]["content"]) and supply the study metadata as the user turn.

Target output format:

Group <n>:
  Cell line: <cell line>
  Perturbation method: <KD | KO | OE>
  Target gene: <HGNC symbol>
  Control: <comma-separated GSM IDs>
  Case: <comma-separated GSM IDs>

Usage

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "jsoul/geo-perturbation-grouping-qwen3-30b-a3b"  # private; requires a token
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")

system = "<the fixed rulebook — see the training dataset's messages[0]>"
user   = "<GEO study title/summary/design + the per-sample GSM table>"

messages = [{"role": "system", "content": system},
            {"role": "user", "content": user}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=1024, do_sample=False)
print(tok.decode(out[0, inputs.shape[1]:], skip_special_tokens=True))

Greedy decoding (do_sample=False) is recommended for this structured extraction task; the packaged generation_config.json otherwise defaults to sampling (temperature=0.7, top_p=0.8, top_k=20), inherited from the base model. For serving, vLLM/TGI with expert parallelism is recommended given the MoE size (~57 GB in bf16).

Intended use & limitations

Intended: curation assistance for identifying perturbation case/control groups from GEO transcriptomic studies (RNA-seq / microarray). Outputs should be reviewed by a human.
Out of scope: non-transcriptomic assays, non-human samples, multi-gene or mutant constructs, and drug/stimulus-confounded designs — the rulebook instructs the model to skip these, but errors are possible.
Limitations: may hallucinate GSM IDs or gene symbols, mis-assign controls, or mishandle very long sample tables. Always validate GSM membership and HGNC symbols against the source GEO record before use.

License

Inherits the base model's license (Apache 2.0, from Qwen/Qwen3-30B-A3B-Instruct-2507). Training data derives from public NCBI GEO metadata; see the dataset cards for provenance.

Citation

Please cite this repository, the base model (Qwen/Qwen3-30B-A3B-Instruct-2507), and acknowledge NCBI GEO as the metadata source.