We introduce Intern-MemDec-4B, a memory decoder designed to extend Intern-S2 backbone models with specialized domain knowledge and capabilities. It is an auxiliary model component rather than a standalone language model: during inference, an Intern-S2 backbone and the memory decoder process the same context in parallel, while a lightweight token-level router dynamically combines their predictions.
Scientific knowledge is long-tailed and continuously evolving. A fixed post-trained foundation model cannot fully cover every specialized subfield, task protocol, or newly emerging discovery. Updating the entire backbone for each domain can also disturb its general reasoning, instruction-following, multimodal, and agentic capabilities. Intern-MemDec-4B instead provides a modular domain-extension mechanism. The Intern-S2 backbone remains intact and serves as the primary reasoning engine, while the attached memory supplies additional biological knowledge and recurring task patterns when they are useful.
Intern-MemDec-4B is trained by compressing retrieval-based evidence from domain data into a reusable parametric module. At inference time, it does not require access to the original retrieval datastore.
Important: Intern-MemDec-4B is not intended to be loaded or used as a standalone chat model. It must be deployed together with a compatible Intern-S2 backbone and the corresponding fusion configuration or router.
Features
Modular Domain Extension. Intern-MemDec-4B adds specialized domain knowledge and task capabilities without updating the Intern-S2 backbone parameters. By turning domain adaptation into memory attachment rather than backbone rewriting, it provides a practical way to extend a general scientific foundation model with expertise in DNA, RNA, proteins, and biomolecular interactions while keeping the original backbone unchanged.
Adaptive Fusion While Preserving General Capabilities. The Intern-S2 backbone and the biological memory process the same decoding context in parallel, and a lightweight token-level router dynamically determines their contributions to each next-token prediction. The memory branch can strengthen specialized predictions on biology-related inputs, while the backbone remains dominant for general reasoning, instruction following, factual knowledge, multimodal understanding, and agentic tasks, reducing interference with the model's original capabilities.
Reusable and Extensible Parametric Memory. Domain evidence is compressed into a reusable parametric module, so online inference does not require the original retrieval datastore. Once trained, the memory can be attached to compatible Intern-S2 backbones using a matching router or fusion configuration. The same approach can also be applied to up-to-date data from other target domains, enabling independently trained memories to continually introduce new knowledge and capabilities as scientific fields evolve.
How It Works
For every decoding step, the Intern-S2 backbone and Intern-MemDec-4B receive the same input context and independently produce next-token predictions. A lightweight router examines signals from both branches and determines how much each should contribute to the final prediction.
On biology-related inputs, the router can increase the contribution of the memory branch to strengthen specialized predictions. On general or unrelated inputs, it can keep the backbone dominant so that the original reasoning, instruction-following, factual, multimodal, and agentic behavior is retained. New domain capabilities can therefore be added as independently trained, plug-and-play memories rather than through repeated full-model fine-tuning.
Memory Decoder architecture
Performance
We evaluate Intern-MemDec-4B with Intern-S2-Preview-397B on all 21 tasks in Biology-Instructions. The attached biological memory improves the average score from 56.92 to 60.32 (+3.40 points). The evaluation in the Technical Report also shows a broadly comparable capability profile on general and multimodal benchmarks, supporting the intended use of Memory Decoder as a domain-extension interface that preserves the foundation model's general behavior.
Biology-Instructions and cross-domain capability radar plots
(a) Biology-Instructions category performance
(b) Cross-domain capability profile
Biology-Instructions
All scores are reported on a 0–100 scale.
Biology-Instructions task
Intern-S2-Preview-397B
+ Intern-MemDec-4B
DNA-cpd
63.11
72.57
DNA-emp
19.95
27.25
DNA-enhancer activity
53.68
60.71
DNA-pd
84.40
89.12
DNA-tf-h
56.57
55.99
DNA-tf-m
56.96
67.09
Multi-sequence antibody-antigen
40.24
36.44
Multi-sequence promoter-enhancer interaction
22.46
38.47
Multi-sequence RNA-protein interaction
84.74
87.34
Multi-sequence siRNA efficiency
63.05
60.63
Protein-Fluorescence
70.48
72.23
Protein-FunctionEC
61.88
60.10
Protein-Solubility
68.60
68.00
Protein-Stability
69.67
67.80
Protein-Thermostability
58.44
53.97
RNA-CRISPROnTarget
6.61
17.18
RNA-Isoform
82.65
84.81
RNA-MeanRibosomeLoading
56.20
59.71
RNA-Modification
59.64
60.48
RNA-NoncodingRNAFamily
78.80
85.70
RNA-ProgrammableRNASwitches
37.13
41.23
Average score
56.92
60.32
Domain Coverage
This release covers the biology domain only. Its training and evaluation focus on biological sequence understanding and prediction tasks involving DNA, RNA, proteins, and biomolecular interactions. Representative task families include:
DNA property, enhancer activity, and transcription-factor prediction;
RNA modification, isoform, regulatory, and interaction prediction;
protein function, solubility, stability, fluorescence, and thermostability prediction;
antibody-antigen and other multi-sequence biomolecular interaction tasks.
Intern-MemDec-4B should not be interpreted as a general memory covering every scientific discipline. To extend Intern-S2 to another domain, a separate memory decoder can be trained on up-to-date, high-quality data from that target domain. The resulting memory can inject new domain knowledge and specialized capabilities into the backbone without rewriting the backbone parameters. This makes the memory interface suitable for continual domain extension as scientific data and task requirements evolve.
The service exposes OpenAI-compatible APIs at http://0.0.0.0:23333 by default. Query it with:
shell
1curl http://localhost:23333/v1/chat/completions \2 -H "Content-Type: application/json"\3 -d '{
4 "model": "internlm/Intern-S2-Preview-397B-FP8",
5 "messages": [{"role": "user", "content": "<PROTEIN> GSSGSSGPSKFIEGLRNEEATEGDTATLWCELSKAAPVEWRKGHETLRDGDRHSLRQDGSRCELQIRGLAVVDAGEYSCVCGQERTSATLTVRALPARFIESGPSSG </PROTEIN> What is the EC number associated with the enzymatic function of this protein? Please put the final enzyme within \\boxed{} using an EC number such as ECx.x.x.x, and separate multiple entries with commas."}]
6 }'
For this example, the model returns the predicted enzyme classes:
\boxed{EC2.7.11.-,EC2.7.11.1}
Compatibility
Intern-MemDec-4B must be used with a supported Intern-S2 backbone through a Memory Decoder-enabled inference runtime. Currently, Intern-S2-Preview-397B is the only supported backbone.
Although the memory checkpoint is designed to be reusable across compatible backbone scales, adaptive routers depend on the selected backbone-memory pair. Use only a router or fusion configuration explicitly released for that pair.
Citation
bibtex
1@misc{wang2026memsftmitigatingalignmenttax,
2 title={MemSFT: Mitigating Alignment Tax with an External Parametric Memory},
3 author={Jiarui Wang and Xiang Shi and Jiaqi Cao and Rubin Wei and Xiquan Wang and Hao Sun and Jingzhi Wang and Zhiqi Yang and Qipeng Guo and Bowen Zhou and Zhouhan Lin},
4 year={2026},
5 eprint={2607.25614},
6 archivePrefix={arXiv},
7 primaryClass={cs.LG},
8 url={https://arxiv.org/abs/2607.25614},
9}
10@article{cao2026memory,
11 title={Memory decoder: A pretrained, plug-and-play memory for large language models},
12 author={Cao, Jiaqi and Wang, Jiarui and Wei, Rubin and Guo, Qipeng and Chen, Kai and Zhou, Bowen and Lin, Zhouhan},
13 journal={Advances in Neural Information Processing Systems},
14 volume={38},
15 pages={115487--115510},
16 year={2026}
17}
