You can deploy this model and start earning money today!

andreasmartin/
apertus-v1.1-swiss-embed-0.4b-bidir

Name: apertus-v1.1-swiss-embed-0.4b-bidir
Availability: InStock

Views

No views yet

Apertus v1.1 Swiss multilingual retrieval embeddings

A standalone Sentence Transformers retrieval model derived from swiss-ai/Apertus-v1.1-0.5B. The model targets multilingual retrieval for German, English, French, Italian, Romansh, and Alemannic / Swiss German.

Model

Property	Value
Parameters	439,397,928 (0.439B)
Source	`swiss-ai/Apertus-v1.1-0.5B`
Apertus lineage	`swiss-ai/Apertus-8B-2509`
Transformer layers	20 / 20
Attention	bidirectional
Pooling	mean
Projection	1024 → 1024
Native embedding dimension	1024
Matryoshka dimensions	`[1024, 768, 512, 256]`
Maximum inference length	1024 tokens

LoRA is used during retrieval training and fully merged before publication. The released repository is therefore a standalone model without a PEFT adapter dependency.

Training

Training uses MultipleNegativesRankingLoss with cross-device negatives and MatryoshkaLoss.

All datasets were loaded from Hugging Face at exact immutable revisions. The released data_manifest.json records source URLs, dataset commit SHAs, the recipe signature and SHA256 hashes of the processed data files.

Training data:

wikimedia/wikipedia: monolingual Swiss-core retrieval pairs plus a small multilingual preservation replay.
eljuanina/VotingBooklets-v1: DE/FR/IT/RM cross-lingual alignment.
ZurichNLP/SwissGov-RSD: EN↔DE/FR/IT cross-lingual alignment.

Actual final training set: 31,483 rows (23,149 Swiss monolingual, 6,884 cross-lingual, 1,450 multilingual preservation).

Training profile: quality. Configuration: 2000 steps, batch/GPU 16, learning rate 5e-05, training length 512 tokens, gradient checkpointing True, LoRA r=8, alpha=16.

Evaluation

Development evaluation uses held-out Wikipedia pairs, eljuanina/VotingBooklets-Diamond-v1, and a held-out SwissGov-RSD split. Diamond is evaluation-only and is therefore not listed in the datasets training metadata above.

Diagnostic	Dim	Accuracy@1	Recall@10	nDCG@10	MRR@10
Swiss monolingual	1024	0.8042	0.9271	0.8638	0.8439
Swiss monolingual	512	0.7958	0.9250	0.8594	0.8386
Swiss cross-lingual	1024	0.2812	0.9437	0.6346	0.5326
Swiss cross-lingual	512	0.2667	0.9396	0.6242	0.5203

Monolingual development queries: 480
Cross-lingual development queries: 480

These are internal development diagnostics, not MTEB/MMTEB benchmark claims.

Usage

python

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("andreasmartin/apertus-v1.1-swiss-embed-0.4b-bidir")

queries = model.encode_query([
    "Welche Aufgaben hat der Bundesrat?"
])

documents = model.encode_document([
    "Le Conseil fédéral est l'autorité exécutive suprême de la Confédération suisse.",
    "Der Nationalrat ist die grosse Kammer der Bundesversammlung.",
])

scores = model.similarity(queries, documents)
print(scores)

For compact Matryoshka embeddings:

embeddings = model.encode_query(texts, truncate_dim=512)

Limitations

Wikipedia title→paragraph pairs are pseudo-retrieval supervision.
Parallel passages provide semantic alignment rather than natural search queries.
Alemannic / Swiss German has monolingual but no explicit parallel supervision.
External MTEB/MMTEB/MIRACL and Swiss/domain-specific evaluation is required for comparative quality claims.

License and attribution

Apertus is developed by the Swiss AI Initiative. This is an independent retrieval adaptation and not an official Swiss AI Initiative embedding release.

The direct source model is swiss-ai/Apertus-v1.1-0.5B. Available license and usage-policy files from the source repository are copied into this model repository. See the linked Hugging Face dataset cards for dataset licensing and attribution.