🐚 OneOcean : 多模态深海大模型
We have released OneOcean, a multimodal large model specialized for the marine domain, to help researchers and engineers explore intelligent understanding of deep-sea imagery.
If you are interested in extending OneOcean for specific marine applications, feel free to contact us for collaboration.
我们已发布 OneOcean 多模态海洋领域大模型，旨在支持科研人员与工程师探索深海图像的智能理解。
如果您希望将 OneOcean 应用于特定的海洋研究或工程场景，欢迎与我们联系合作。
🔔 Important
We will continue to update and optimize the OneOcean multimodal large model.
The model’s capabilities may vary across versions, and your feedback is highly appreciated to advance the use of multimodal LLMs in marine science.
🔔 重要提示
我们将持续更新和优化OneOcean多模态大模型。
模型的性能可能随版本变化而不同，欢迎您提出宝贵反馈，共同推动多模态大模型在海洋科学领域的应用发展。
📘 Table of Contents / 目录
ℹ️ About the model / 关于模型
We fine-tuned the Qwen2.5-VL-72B-Instruct foundation model to create OneOcean, a multimodal model specialized for the marine domain. OneOcean supports functionalities such as marine organism detection and geological feature recognition.
基于 Qwen2.5-VL-72B 大模型进行微调，得到 OneOcean 多模态海洋领域大模型，支持 生物检测 和 地质识别 等功能。
✨ Features / 功能
- Marine organism detection (生物检测)
- Geological feature recognition (地质识别)
- Multimodal understanding of underwater imagery (多模态海底图像理解)
📊 Training summary / 训练摘要
- Base model: Qwen2.5-VL-72B-Instruct
- Fine-tuning method: LoRA via LLaMA-Factory
- Final training loss: 0.3548
| Training Loss | Validation Loss |
|---|
| |
🗂️ Training datasets / 训练数据
- ~300K+ image-text pairs for marine organisms (30多万条生物图文对)
- ~100K+ image-text pairs for geological features (10多万条地质图文对)
- LoRA fine-tuning via LLaMA-Factory
⚙️ Framework versions / 框架版本
- PyTorch 2.4.1+cu121
- Transformers 4.49.0.dev0
- PEFT 0.15.2
- Datasets 3.6.0
- Tokenizers 0.21.1
🚀 Quickstart / 快速使用
1conda create -n py3.11 python=3.11
2conda activate py3.11
3pip install -r requirements.txt
Download the model
Download from HuggingFace
1# use git lfs
2git lfs install
3git clone https://huggingface.co/Jady-Zhao/OneOcean
🔍 Inference / 推理
🌋 Geological inference / 地质推理
1from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
2from PIL import Image
3import torch
4
5model_name = "Jady-Zhao/OneOcean"
6
7# Load model and processor
8model = Qwen2_5_VLForConditionalGeneration.from_pretrained(model_name, device_map="auto", trust_remote_code=True)
9processor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)
10
11image = Image.open("example_underwater_image.jpg")
12question = "请判断这张图的地质类型。" # Please determine the geological type of this image
13
14inputs = processor(
15 text=[question],
16 images=[image],
17 padding=True,
18 return_tensors="pt"
19).to(model.device)
20
21outputs = model.generate(**inputs, max_new_tokens=50)
22prediction = processor.batch_decode(outputs, skip_special_tokens=True)[0]
23
24print("Predicted geological type:", prediction)
🦑 Biological inference / 生物推理
1from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
2from PIL import Image
3import torch
4
5model_name = "Jady-Zhao/OneOcean"
6
7# Load model and processor
8model = Qwen2_5_VLForConditionalGeneration.from_pretrained(model_name, device_map="auto", trust_remote_code=True)
9processor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)
10
11image = Image.open("example_underwater_image.jpg")
12question = "请输出这张图中的深海生物类型。" # Please identify the deep-sea organism in the image
13
14inputs = processor(
15 text=[question],
16 images=[image],
17 padding=True,
18 return_tensors="pt"
19).to(model.device)
20
21outputs = model.generate(**inputs, max_new_tokens=50)
22prediction = processor.batch_decode(outputs, skip_special_tokens=True)[0]
23
24print("Predicted biological type:", prediction)
🌻 Acknowledgement / 致谢
We would like to thank:
谨向以下人员致谢：
- The developers of Qwen2.5-VL-72B for providing the foundation model;
- Qwen2.5-VL-72B 基础模型的开发者;
- The creators of LLaMA-Factory / PEFT / Transformers / BitsAndBytes for their excellent open-source libraries;
- LLaMA-Factory / PEFT / Transformers / BitsAndBytes 等优秀开源库的贡献者；
- The contributors of the Oceanus_Bio and Oceanus_Geo datasets;
- Oceanus_Bio 与 Oceanus_Geo 数据集的提供者;
⚠️ Limitations / 模型限制
- The model is specialized for marine organisms and geological features and may not generalize to other domains;
- 模型专注于 海洋生物和地质特征，可能无法推广到其他领域;
- Predictions may be inaccurate for low-quality, noisy, or unusual images.
- 对低质量、噪声或特殊图像的预测可能不准确;
- Outputs can vary slightly depending on prompt wording.
- 输出结果可能受 提示语 影响而略有差异;
- The model is not optimized for real-time processing of large video streams.
- 模型尚未针对大规模视频流进行实时优化
🚩 Citation & Contact / 引用与联系
This model (OneOcean) has not been formally published yet.
本模型 (OneOcean) 尚未正式发布。
If you would like to use it in your work, please contact the author via email:
如果希望在研究或工作中使用，请 发送邮件联系作者:
1@misc{oneocean2025,
2 title = {OneOcean: A Multimodal Marine Foundation Model},
3 author = {Zhejianglab},
4 year = {2025},
5 howpublished = {\url{https://huggingface.co/Jady-Zhao/OneOcean}},
6 note = {Accessed: 2025-11-03}
7}
🏆 Contributors
COMPUTATIONAL SENSING RESEARCH CENTER, ZHEJIANG LAB
