Views
No views yet
Package.swift:1.package(url: "https://github.com/john-rocky/CoreML-LLM", branch: "main"),
2
3// In your target:
4.product(name: "CoreMLLLM", package: "CoreML-LLM"),1import CoreMLLLM
2
3let modelsDir = try FileManager.default.url(
4 for: .applicationSupportDirectory, in: .userDomainMask,
5 appropriateFor: nil, create: true)
6
7let eg = try await EmbeddingGemma.downloadAndLoad(modelsDir: modelsDir)
8
9// 768-dim L2-normalised embedding
10let v = try eg.encode(text: "How do I list files in Swift?")
11// Matryoshka: cheap-to-truncate dims (768 / 512 / 256 / 128)
12let v256 = try eg.encode(text: "How do I list files in Swift?",
13 dim: 256)
14
15// Task-prefixed (RAG document vs. query)
16let q = try eg.encode(text: "list files",
17 task: .retrievalQuery)
18let d = try eg.encode(text: "Use FileManager.contentsOfDirectory(...)",
19 task: .retrievalDocument)Gemma3EmbeddingGemma.swift
for task prefixes and dim list.google/embeddinggemma-300m produced with the
CoreML-LLM pipeline. Targets
iOS 26 / macOS 26.| File | Notes |
|---|---|
encoder.mlmodelc/ | Compiled stateless bidirectional encoder (INT8, ~295 MB) |
encoder.mlpackage/ | Complete generated Core ML source package (INT8, ~295 MB) |
model_config.json | I/O contract, Matryoshka dims, task prefixes |
hf_model/ | Tokenizer files |
MLComputePlan on macOS 26). Achieved by:sum(x²) bounded)1import CoreMLLLM
2let bundleURL = try await Gemma3BundleDownloader.download(
3 .embeddingGemma300m, into: appSupportDir)
4let eg = try await EmbeddingGemma.load(bundleURL: bundleURL)
5let vec = try eg.encode(text: "On-device embeddings",
6 task: .retrievalQuery,
7 dim: 768) // or 512 / 256 / 128 (Matryoshka)input_ids (1, 256) int32, attention_mask (1, 256) fp16 (1.0 valid, 0.0 pad)embedding (1, 768) fp16 — L2 unit norm; truncate the trailing dim and
re-normalize for Matryoshka 512 / 256 / 128max_seq_len=256. To choose a different
fixed input length, re-run
python conversion/build_embeddinggemma_bundle.py --max-seq-len <length> --quantize int8.cosine("cat sat on mat", "feline rested on rug") = 0.7345 (high — similar)
cosine("cat sat on mat", "quantum mechanics") = 0.4650 (low — different)