Gemma 4 E2B — OceanGuard Marine Debris LoRA Adapter

Headline result. On the held-out evaluation split (n = 200 stratified samples, real_test_holdout.jsonl) the LoRA fine-tune reaches mAP@0.5 = 0.3256, against 0.1067 for the unmodified base model (Δ = +0.2189, a +205 % relative improvement). The winner is exp12_vision_lora, which unfreezes the SigLIP2 vision encoder via LoRA on top of the language-only adapter. Canonical metrics are tracked in finetune/experiments/results/best.json and results.md of the source repository.

Files in This Repository

Model Description

Intended Use

• Mobile and edge marine-debris detection for citizen science, NGO field operations, and coastal monitoring campaigns.
• Research baselines for open-vocabulary detection over specialised environmental taxonomies.
• A grounding component for tool-augmented reporting pipelines (see the OceanGuard single-conversation agentic tool-calling architecture).

• Autonomous decision-making (cleanup dispatch, regulatory enforcement, environmental fines). This model is a triage and reporting aid; final decisions must remain with a qualified human.
• Real-time safety-critical detection (collision avoidance, navigation). End-to-end latency on the reference device is on the order of seconds per image.
• General-purpose object detection outside the marine-debris domain — the adapter narrows the visual prior towards marine debris and may degrade on unrelated tasks.

Training Data

Training Procedure

Configuration (best: exp12_vision_lora)

Why training stopped at step 1 150

Convergence

Hardware and wall-clock

• 1× NVIDIA RTX 5090 (32 GiB GDDR7), Windows 11, CUDA 12.8, PyTorch 2.7
• Training wall-clock: ≈ 5 h 30 min end-to-end (initial run to step 1 097 + recovery cycle to step 1 150 + final eval)
• Deployment target: Samsung Galaxy S22 Ultra (Exynos 2200) via LiteRT-LM 0.11.0

Software

Evaluation

Aggregate

Per-class mAP@0.5

Per-source mAP@0.5 (stratified breakdown)

• The values are computed with the same VOC 11-point AP fallback used by the project's eval_grid.manual_map_eval (matches the global card row within ≤ 0.002 rounding versus the pycocotools COCO-standard pass that produced 0.3256).
• CleanSea (n = 19) is the smallest subset and therefore the 6.54× ratio has wider confidence; treat it as a strong directional signal rather than a tight point estimate.
• The eval split is not contaminated: the 200 ids are a subset of real_test_holdout.jsonl, none of which appears in any experiments/splits/exp*/train.jsonl. The companion eval dataset on Hugging Face, asferrer/oceanguard-marine-debris-eval-1000, ships the full annotation set plus compute_per_source_map.py for independent verification.

Deployment

1. GGUF + llama.cpp (available now): the merged model is quantized to Q4_K_M and served via the llama.cpp engine already integrated in the OceanGuard Android app.
2. LiteRT-LM (pending): the .litertlm export is blocked on upstream converter support for the Gemma 4 / Gemma3n architecture.

GGUF Download

In-App Selection

• Decode throughput: 5.1 tok/s (BASE LiteRT-LM); GGUF expected ~4-5 tok/s CPU-only
• Time to first token (warm / cold first load): 2.33 s / 17.34 s
• Prefill throughput: 48.7 tok/s
• Single-shot detection, ~100-token JSON output: ≈ 22 s end-to-end

Limitations

• Fixed-with-fallback taxonomy. The fine-tune adheres to an 8-class core extended to 50 fine-grained classes. Out-of-distribution debris (polar microplastic, deep-sea anthropogenic objects) is not represented in the training data and will be mapped to the generic catch-all classes.
• CPU + Vulkan-only inference on the reference device. The Exynos 2200 GPU pass via Vulkan improves correctness but not throughput; on this hardware Gemma 4 E2B is reserved for single-shot deep analysis (≈ 20 s end-to-end per image), not for continuous live-camera operation.
• Geographic bias. The training imagery is predominantly sourced from coastal waters around Japan and the broader north-western Pacific. Performance on the Mediterranean, tropical reefs, polar seas, the south Atlantic and freshwater debris surveys is expected to be lower and is not yet measured. Users surveying outside Japan-Pacific waters should treat the model as a baseline and validate against a local hold-out set before drawing conclusions.
• Lighting and capture bias. Synthetic-augmentation passes can inject lighting biases that may degrade under poor underwater visibility.
• Metal_Debris remains unlearned. Despite unfreezing the vision encoder via LoRA, Metal_Debris per-class mAP@0.5 stays at 0.000, identical to the language-only baseline. This rules out visual representation as the bottleneck and points to either label ambiguity in the test split (overlap with Tire, Plastic_Debris, Other_metal_objects) or insufficient ground-truth instances. A class-balanced data campaign and a per-class label audit are planned.
• Can improves but remains weak. mAP@0.5 = 0.091. Operators should expect silent misses on this class and gate critical decisions accordingly.
• JSON validity is 88.5 %, not 100 %. Roughly 1 in 9 detections in the eval set emerge in malformed JSON and are dropped by the parser. The Android app handles these gracefully by surfacing an empty detection set to the user rather than fabricating boxes. The drop from the language-only baseline (94.5 %) is the cost the vision LoRA pays for the higher recall on texture-rich classes — it is the dominant remaining failure mode.
• Trained to step 1 150 of a 2 292-step plan. The published checkpoint reaches the ablation-grid winner mAP but was halted before full 3-epoch convergence (see §Training Procedure). Further training on a workstation with a larger pagefile or with a lazy-loading collator is expected to refine these numbers modestly, particularly on Plastic_Debris, Bottle and Tire which regressed slightly versus the language-only winner.
• Not certified. This model is not certified for legal, regulatory or environmental enforcement reporting. The OceanGuard app produces draft reports that a human reviews before any official use.
• Function-calling head untouched. The fine-tune is intentionally scoped to the detection head and does not rewrite tool-calling syntax, to avoid regressing the single-conversation agentic tool-calling pipeline that the Android app depends on for grounded report generation.

Bias, Risks and Safety

• Spurious-correlation risk. Marine-debris classes co-occur with characteristic backgrounds (e.g. fishing nets with rocky shorelines). The model may exploit context cues that do not generalise across geographies.
• Sensitive categories. The syringe / medical waste class is rare and high-stakes; positive detections should trigger human handling protocols rather than autonomous action.
• Citizen-science abuse vectors. As with any open-vocabulary detector, the model can be prompted to label arbitrary objects; downstream applications should constrain inputs to the marine-debris domain and filter implausible outputs (the OceanGuard app does this via LABEL_TO_TYPE and the canonical 11-class projection in EnvironmentalImpact.IMPACT_MAP).
• Privacy. Photos analysed on-device in OceanGuard never leave the phone; this adapter itself processes no user data. When used outside the app, downstream operators are responsible for compliance with applicable privacy regulation.

How to Use

A. Research / training pipeline — PEFT + Unsloth

B. On-device deployment — GGUF Q4_K_M via llama.cpp (available now)

C. On-device deployment — direct .litertlm download (pending upstream)

License and Citation

• LoRA adapter — Apache 2.0
• Base model — google/gemma-4-E2B-it, subject to the Gemma Terms of Use
• Training datasets — source-dataset licenses apply (CC-BY or equivalent; see dataset cards)

Related Publications by the Author

Peer-reviewed papers

1. Sánchez-Ferrer, A., Valero-Mas, J. J., Gallego, A. J., & Calvo-Zaragoza, J. (2023). An experimental study on marine debris location and recognition using object detection. Pattern Recognition Letters, 168, 154–161. https://doi.org/10.1016/j.patrec.2022.12.019
2. Sánchez-Ferrer, A., Gallego, A. J., Valero-Mas, J. J., & Calvo-Zaragoza, J. (2022). The CleanSea Set: A Benchmark Corpus for Underwater Debris Detection and Recognition. In Iberian Conference on Pattern Recognition and Image Analysis (IbPRIA 2022), Lecture Notes in Computer Science, Springer. doi.org/10.1007/978-3-031-04881-4_49

Theses

1. Sánchez-Ferrer, A. (2024). Modelos de difusión aplicados a la detección de objetos en el fondo marino. Master's Thesis, Universidad de Alicante. rua.ua.es/…/88244474
2. Sánchez-Ferrer, A. (2021). Deep Learning aplicado a la detección de residuos en el fondo marino. Bachelor's Thesis, Universidad de Alicante. rua.ua.es/…/92c34588

Applied research

1. Sánchez-Ferrer, A. (2025). OceanGuard AI: Mapping and Mitigating Marine Pollution. Kaggle hackathon write-up. Kaggle write-up

Acknowledgments

• Google DeepMind for releasing Gemma 4 under terms that enable open research and on-device deployment.
• The Unsloth team for the FastVisionModel implementation that makes a Gemma 4 vision-language fine-tune tractable on a single workstation GPU.
• CleanSea, Ocean_garbage and Neural_Ocean dataset authors for releasing marine-debris annotations under permissive terms.
• Google AI Edge / LiteRT-LM team for the on-device runtime that hosts the base model in the OceanGuard Android app.
• Pattern Recognition and Artificial Intelligence (PRAI) group, Universidad de Alicante — research group hosting the doctoral programme under which this work is carried out, and co-authors of the prior publications cited above.