Model Card for Model ID
This is an adapter for the Salesforce BLIP2 2.7B model (more information on the model
here). It was fine-tuned for generating product descriptions based on images using the
H&M dataset from the kaggle challenge 2022.
How to Get Started with the Model
Use the code below to get started with the model. Make sure to replace the path with a local path to an image.
1from transformers import Blip2Processor, Blip2ForConditionalGeneration
2from PIL import Image
3import torch
4
5torch_dtype = torch.bfloat16
6device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
7
8base_checkpoint = "Salesforce/blip2-opt-2.7b"
9base_model = Blip2ForConditionalGeneration.from_pretrained(base_checkpoint, torch_dtype=torch_dtype)
10
11adapter_checkpoint = "CDL-RecSys/blip2-opt-2.7b-hm"
12model = PeftModel.from_pretrained(base_model, model_id=adapter_checkpoint)
13
14processor = Blip2Processor.from_pretrained(base_checkpoint)
15tokenizer = processor.tokenizer
16
17image = Image.open("path/to/your/image.jpg")
18inputs = processor(image, return_tensors="pt").to(device, torch_dtype)
19generated_ids = model.generate(**inputs, max_length=max_length)
20generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)
21generated_text
Training Details
Training Data
[More Information Needed]
Training Procedure
Preprocessing [optional]
[More Information Needed]
Training Hyperparameters
- Training regime: [More Information Needed]
Speeds, Sizes, Times [optional]
[More Information Needed]
Evaluation
Testing Data, Factors & Metrics
Testing Data
[More Information Needed]
Factors
[More Information Needed]
Metrics
[More Information Needed]
Results
[More Information Needed]
Summary
Model Examination [optional]
[More Information Needed]
Environmental Impact
Carbon emissions can be estimated using the
Machine Learning Impact calculator presented in
Lacoste et al. (2019).
- Hardware Type: [More Information Needed]
- Hours used: [More Information Needed]
- Cloud Provider: [More Information Needed]
- Compute Region: [More Information Needed]
- Carbon Emitted: [More Information Needed]
Technical Specifications [optional]
Model Architecture and Objective
[More Information Needed]
Compute Infrastructure
[More Information Needed]
Hardware
[More Information Needed]
Software
[More Information Needed]
Citation [optional]
BibTeX:
[More Information Needed]
APA:
[More Information Needed]
Glossary [optional]
[More Information Needed]
More Information [optional]
[More Information Needed]
Model Card Authors [optional]
[More Information Needed]
Model Card Contact
[More Information Needed]
Framework versions
