You can deploy this model and start earning money today!

AL-GR/
Forge-EMB-mmclip

Name: Forge-EMB-mmclip
Availability: InStock

Views

No views yet

Forge-EMB-mmclip

This repository contains a multimodal embedding model based on CLIP, designed for the FORGE framework tasks. It allows for joint embedding of images and text.

The model was introduced in the paper FORGE: Forming Semantic Identifiers for Generative Retrieval in Industrial Datasets.

Code: selous123/al_sid
Framework: AL-GR/FORGE

📚 Reference: This implementation is based on the mm_clip project from the al_sid repository.

🚀 Quick Start

The easiest way to use this model is by running the provided Jupyter Notebook.

1. Clone the Repository

bash

git clone https://huggingface.co/AL-GR/Forge-EMB-mmclip
cd Forge-EMB-mmclip

⚠️ Important Note on Large Files: The file clip.pth is stored using Git LFS. If you see that clip.pth is only ~134 Bytes on your local machine, it means the actual weights were not downloaded. Please ensure you have git-lfs installed and run:

bash
1git lfs install 2git lfs pull

Or download the files directly via the "Download" button on the website.

2. Install Dependencies

Ensure you have the necessary libraries installed (e.g., torch, transformers, jupyter).

pip install torch torchvision transformers matplotlib pillow

3. Run the Demo

Simply open and run the notebook:

jupyter notebook CLIP_demo.ipynb

Inside the notebook, you will find examples of how to load the model from clip.pth and process images like test.jpg.

📂 File Structure

File/Folder	Description
`CLIP_demo.ipynb`	The main entry point. Contains inference code and visualization examples.
`clip.pth`	The pre-trained model weights (PyTorch checkpoint).
`bert/`	Configuration or tokenizer files related to the text encoder.
`utils/`	Helper functions used by the demo.
`test.jpg`	A sample image for testing the model.

🔗 Related Resources

Original Implementation: selous123/al_sid/mm_clip
FORGE Framework: AL-GR/FORGE

📄 Citation

If you find this work helpful, please cite:

bibtex

@article{fu2025forge,
  title={FORGE: Forming Semantic Identifiers for Generative Retrieval in Industrial Datasets},
  author={Fu, Kairui and Zhang, Tao and Xiao, Shuwen and Wang, Ziyang and Zhang, Xinming and Zhang, Chenchi and Yan, Yuliang and Zheng, Junjun and others},
  journal={arXiv preprint arXiv:2509.20904},
  year={2025}
}

📄 License

This project is licensed under the Apache License 2.0.