ChemLink is a LoRA fine-tune of tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.3 for extracting chemical measurement values (MW, IC50, EC50, Yield) from scientific literature, with compound-name linkage for PubChem grounding and Graph RAG integration.

Background and Motivation

Target environment: CPU-only local hardware, no GPU required.

Chemical and pharmaceutical researchers frequently operate under security policies that prohibit cloud API usage. This model is designed to run on a standard CPU workstation (e.g., Core i7 / 24 GB RAM) via Ollama in GGUF format (q5_K_M, ~5 GB), suitable for overnight batch processing in network-restricted or air-gapped environments without any cloud dependency.

A critical requirement in this setting is compound-name linkage: downstream pipelines (PubChem grounding, Graph RAG, compound databases) need to know not just the measurement value, but which chemical compound it belongs to. This requires the model to output a compound_name field alongside each extracted value.

Two prompt conditions were evaluated:

Condition A (no instruction): prompt requests only type / value / unit; compound_name is not mentioned.
Condition B (with instruction): prompt explicitly requests compound_name in addition to type / value / unit.

In the Colab GPU evaluation, ChemLink outputs compound_name under both conditions. In the same Colab evaluation, all comparison models (Swallow-base, Mistral-7B) output 0% compound_name without explicit instruction (Condition A). In the local Ollama evaluation, Swallow-base also produced compound_name under Condition A; this behavior is environment- and template-dependent (see Evaluation and Limitations).

Key Capability

ChemLink retained compound_name output when the evaluation prompt did not explicitly request the field, across the evaluated Colab GPU and local CPU environments.

json

{
  "chemical_entities": [
    {
      "compound_name": "linezolid",
      "measurements": [
        {"type": "Molecular Weight", "value": 337.35, "unit": "g/mol"}
      ]
    }
  ]
}

Note: compound_name reflects the name as it appears in the source text. It is not normalized or verified against any database at inference time. Across the evaluated MW records, approximately 59–65% of ChemLink-generated compound names were directly resolvable through the PubChem REST API (see Evaluation).

This stability reduces the risk of pipeline failures where a measurement value is extracted but cannot be linked to its source compound — a risk that depends on prompt design when using baseline models.

Model Overview

Item	Detail
Developer	MitzMitz / Ingenta AI
Base model	tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.3
Published	LoRA adapter (168 MB) + tokenizer; base model auto-loaded from HuggingFace
Training tool	unsloth + TRL (SFTTrainer)
Quantization	4-bit NF4 (QLoRA, training); q5_K_M GGUF (local CPU deployment)
LoRA config	r=16, alpha=32, dropout=0, bias=none
Max seq length	2048
Local deployment	Ollama (GGUF q5_K_M) — CPU only, no GPU required
Supported languages	Japanese, English
License	Llama 3.1 Community License

Usage

Local CPU Inference (Ollama — Primary Use Case)

bash

ollama create llama-chemlink-parser-8b-mtys -f Modelfile
ollama run llama-chemlink-parser-8b-mtys

Modelfile example (replace /path/to/ with your actual GGUF file path):

FROM /path/to/Llama-3.1-Swallow-8B-Instruct-v0.3.Q5_K_M.gguf

TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"""

PARAMETER temperature 0
PARAMETER num_ctx 2048
PARAMETER num_predict 256
PARAMETER stop "<|eot_id|>"

Note: num_predict 256 is required. The default (128) causes truncation of structured JSON output.

Inference (Colab / GPU)

This repository publishes the LoRA adapter only. The base model (tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.3) is loaded automatically from HuggingFace.

python

import torch, json, re
from unsloth import FastLanguageModel
from google.colab import userdata

HF_TOKEN = userdata.get('HF_TOKEN')

SYSTEM_PROMPT = (
    "You are a chemical data extraction assistant. "
    "Extract measurements from the given text and return a JSON object. "
    "The object must have a 'chemical_entities' array. "
    "Each element must have: compound_name (string), "
    "measurements (array of objects with type/value/unit). "
    "If no target measurement is found, return {\"chemical_entities\": []}. "
    "Output only the JSON object, no explanation."
)

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name     = "MitzMitz/Llama-ChemLink-Parser-8B-MTYS",
    max_seq_length = 2048,
    dtype          = None,
    load_in_4bit   = True,
    token          = HF_TOKEN,
)
FastLanguageModel.for_inference(model)

def extract(text):
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user",   "content": text},
    ]
    input_ids = tokenizer.apply_chat_template(
        messages, tokenize=True,
        add_generation_prompt=True, return_tensors="pt"
    ).to("cuda")
    with torch.no_grad():
        output = model.generate(
            input_ids, max_new_tokens=256,
            temperature=0.0, do_sample=False,
            pad_token_id=tokenizer.eos_token_id,
        )
    return tokenizer.decode(
        output[0][input_ids.shape[1]:], skip_special_tokens=True
    ).strip()

print(extract("The compound linezolid has a molecular weight of 337.35 g/mol."))

Training Configuration

Parameter	Value
per_device_train_batch_size	1
gradient_accumulation_steps	16
num_train_epochs	2
learning_rate	2e-4
warmup_steps	10
lr_scheduler_type	cosine
fp16 / bf16	auto-detected
optimizer	adamw_8bit (unsloth default)
save_strategy	steps (save_steps=20)

Training Data

File	Total	MW	IC50	EC50	Yield	Negative	Source
phase6_train_mix	3,763	2,283	717	0	44	719	PubChem / ChEMBL / ORD
additional_ec50_yield	2,534	0	0	1,000	1,000	534	ChEMBL / ORD
additional_yield_table	621	0	0	0	500	121	ORD
additional_mw_unit_fix	120	84	16	0	0	20	PubChem
additional_phase5	740	17	115	22	425	161	ChEMBL / ORD / PubChem
Total	7,778	2,384	848	1,022	1,969	1,555

Negative samples (1,555 records, 20.0%) contain [] as output.

Data licenses:

ORD: CC-BY-SA 4.0
ChEMBL: CC-BY-SA 3.0 (EMBL-EBI)
PubChem: Public Domain (NCBI/NIH)

Evaluation

Dataset

Source: true_eval_all_pmid_clean.jsonl (2,963 records total; PMID-verified; no overlap was found between the fine-tuning dataset and the evaluation dataset using PMID-based matching).

This evaluation uses a stratified 500-sample subset (125 per indicator: MW / Yield / IC50 / EC50), RANDOM_SEED=42. The full 2,963-sample dataset was used to construct the source file; the 500-sample subset is drawn from it without replacement.

The tables below report results for MW only. IC50 and EC50 accuracy was not evaluated under this protocol because the structured-output format suppressed IC50/EC50 responses across the evaluated models. Yield records were included in the sampled dataset, but Yield accuracy was not evaluated because ground-truth Yield values were not extracted for scoring. PubChem-based validation is also not applicable to Yield because reaction yield is not an intrinsic molecular property.

Yield evaluation: Yield records were included in the stratified evaluation sample, but Yield performance is not reported in the present evaluation because ground-truth Yield values were not extracted and accuracy scoring was not performed. Unlike molecular weight, reaction yield is specific to the reaction, conditions, and reported yield definition, and therefore cannot be independently validated through PubChem molecular-property lookup. A separate Yield-specific evaluation dataset and scoring protocol are required before reporting Yield accuracy.

Evaluation Prompts

The following system prompts were used verbatim in all evaluations. These are identical to SYSTEM_PROMPT_NO_COMPOUND and SYSTEM_PROMPT_WITH_COMPOUND in common_chemfmt.py.

Condition A — no compound_name instruction:

You are a chemical data extraction assistant. Extract measurements from the given text and return a JSON array. Each element must have: type (IC50/EC50/MW/Yield), value (number), unit (string). If no target measurement is found, return []. Output only the JSON array, no explanation.

Condition B — with compound_name instruction:

You are a chemical data extraction assistant. Extract measurements from the given text and return a JSON object. The object must have a 'chemical_entities' array. Each element must have: compound_name (string), measurements (array of objects with type/value/unit). If no target measurement is found, return {"chemical_entities": []}. Output only the JSON object, no explanation.

The Usage section (Colab inference code) uses Condition B as the example system prompt. Condition A cannot be reproduced from the Usage section; use the prompt above.

Column Definitions

All values are computed via the PubChem REST API (queried by compound name, https://pubchem.ncbi.nlm.nih.gov/rest/pug).

JSONL field mapping (source file: mw_pubchem_eval_results.jsonl):

Table column	JSONL field	Denominator
MW output coverage	n / 125	125 source MW records
compound_name	compound_name_present / n	n
PubChem resolved	pubchem_found / compound_name_present	compound_name_present
MW accuracy	truth_match / n	n
PubChem MW match	pubchem_match / compound_name_present	compound_name_present

Denominator hierarchy:

125 source MW records
  └─ n: correctly typed MW outputs
       └─ compound_name-present records
            └─ PubChem-resolved records
                 └─ PubChem MW-consistent records

MW output coverage = n / 125. Records with no parseable output, malformed JSON, no MW measurement, or non-matching type labels are excluded from n. MW accuracy is a conditional value accuracy calculated only among correctly typed MW outputs; it is not the end-to-end success rate over all 125 records.

n differs between conditions and models because different records fail to produce a correctly-typed MW field under each prompt format and model. The source pool of 125 MW records is identical across all conditions.

Of the 81 PubChem-resolved names (ChemLink q5_K_M, with instruction), 80 had a molecular weight consistent with the extracted value within ±1% (80/81 = 98.8%), indicating that the resolved PubChem record is consistent with the intended compound. This differs from the PubChem MW match column value (80/125 = 64.0%), which uses compound_name-present records as denominator.

Colab GPU / NF4 — MW (n per source)

Model	Condition	n	MW output coverage	compound_name	PubChem resolved	MW accuracy	PubChem MW match
ChemLink NF4	with instruction	120	120/125 (96.0%)	120/120 (100.0%)	76/120 (63.3%)	120/120 (100.0%)	75/120 (62.5%)
ChemLink NF4	no instruction	123	123/125 (98.4%)	123/123 (100.0%)	73/123 (59.3%)	123/123 (100.0%)	69/123 (56.1%)
Swallow-base	with instruction	124	124/125 (99.2%)	124/124 (100.0%)	80/124 (64.5%)	124/124 (100.0%)	79/124 (63.7%)
Swallow-base	no instruction	123	123/125 (98.4%)	0/123 (0.0%)	—	123/123 (100.0%)	—
Mistral-7B	with instruction	32	32/125 (25.6%)	32/32 (100.0%)	22/32 (68.8%)	32/32 (100.0%)	22/32 (68.8%)
Mistral-7B	no instruction	112	112/125 (89.6%)	0/112 (0.0%)	—	112/112 (100.0%)	—

Colab GPU parameters: temperature=0.0, max_new_tokens=256, apply_chat_template. These results are provided for reference only and do not represent the local CPU deployment scenario this model targets.

Mistral-7B with instruction n=32: Only 32 of 125 MW records contained a correctly-typed MW field. Other records produced output in chemical_entities format with incorrect type labels. This is a type-label inconsistency, not truncation.

MW accuracy is conditional on correctly typed MW output and should be interpreted together with MW output coverage.

Local CPU / Ollama q5_K_M — MW (n per source)

Model	Condition	n	MW output coverage	compound_name	PubChem resolved	MW accuracy	PubChem MW match
ChemLink q5_K_M	with instruction	125	125/125 (100.0%)	125/125 (100.0%)	81/125 (64.8%)	125/125 (100.0%)	80/125 (64.0%)
ChemLink q5_K_M	no instruction	124	124/125 (99.2%)	124/124 (100.0%)	75/124 (60.5%)	124/124 (100.0%)	75/124 (60.5%)
Swallow-base q5_K_M	with instruction	125	125/125 (100.0%)	125/125 (100.0%)	80/125 (64.0%)	125/125 (100.0%)	79/125 (63.2%)
Swallow-base q5_K_M	no instruction	124	124/125 (99.2%)	124/124 (100.0%)	75/124 (60.5%)	124/124 (100.0%)	75/124 (60.5%)
Mistral-7B q5_K_M	with instruction	67	67/125 (53.6%)	67/67 (100.0%)	32/67 (47.8%)	67/67 (100.0%)	32/67 (47.8%)
Mistral-7B q5_K_M	no instruction	123	123/125 (98.4%)	0/123 (0.0%)	—	123/123 (100.0%)	—

Local CPU parameters: temperature=0.0, num_predict=256, num_ctx=2048, Ollama Modelfile TEMPLATE.

Mistral-7B q5_K_M with instruction n=67: Only 67 of 125 MW records contained a correctly-typed MW field. Same type-label inconsistency as Colab (less severe locally).

MW accuracy is conditional on correctly typed MW output and should be interpreted together with MW output coverage.

† Swallow-base q5_K_M no instruction: compound_name output under no-instruction condition via Ollama differs from the Colab GPU result (0%) for the same base model. The discrepancy may result from differences in prompt serialization, chat templates, quantization, or inference runtimes. A controlled ablation was not performed. This result should not be interpreted as an intrinsic model capability.

Limitations

compound_name reflects source text only: The model copies the compound name as written in the source document. It is not normalized or verified at inference time. Generic codes ("compound 3", "2b") common in real PubMed abstracts will be output as-is and typically fail PubChem resolution.

Mistral-7B type-label inconsistency under chemical_entities schema: Mistral-7B-Instruct-v0.2 frequently returned chemical_entities JSON, but the measurement type labels did not match the accepted MW labels (67/125 correctly typed locally; 32/125 on Colab GPU). The JSON structure itself was produced; type vocabulary was inconsistent with the schema.

Swallow-base no-instruction Ollama artifact: Swallow-base q5_K_M showed compound_name output under no-instruction condition via Ollama, not observed in Colab GPU evaluation of the same base model (0%). The discrepancy may result from differences in prompt serialization, chat templates, quantization, or inference runtimes. A controlled ablation was not performed.

IC50 / EC50: IC50/EC50 accuracy was not evaluated under this protocol. Not suitable for cross-model comparison.

Inference environment differences: Colab GPU: temperature=0.0, max_new_tokens=256, apply_chat_template. Local Ollama: temperature=0.0, num_predict=256, Modelfile TEMPLATE. Cross-environment comparisons should account for these differences.

LoRA adapter only: This repository publishes the LoRA adapter (168 MB) and tokenizer files. The base model (~16 GB) is loaded from HuggingFace at inference time. For local CPU deployment, a pre-merged GGUF file is required.

Intended Use

Automated extraction of MW / Yield from chemical literature in network-restricted, CPU-only local environments
Compound-name to measurement-value association for PubChem grounding and Graph RAG pipelines
Overnight batch processing on CPU-only hardware without cloud API dependency

Out-of-Scope Use

Medical diagnosis or legal judgment
Domains outside chemistry and chemical biology
IC50 / EC50 extraction (see Limitations)

Base Model Reference

Model	License
tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.3	Llama 3.1 Community License
meta-llama/Llama-3.1-8B-Instruct	Llama 3.1 Community License

License

Framework Versions

Library	Version
unsloth	2026.5.2
PEFT	0.19.1
Transformers	5.5.0
PyTorch	2.10.0
TRL	0.24.0
Datasets	4.3.0