Model Card for Model ID

Model Details

Model Description

• Developed by: [More Information Needed]
• Funded by [optional]: [More Information Needed]
• Shared by [optional]: [More Information Needed]
• Model type: [More Information Needed]
• Language(s) (NLP): [More Information Needed]
• License: [More Information Needed]
• Finetuned from model [optional]: [More Information Needed]

Model Sources [optional]

• Repository: [More Information Needed]
• Paper [optional]: [More Information Needed]
• Demo [optional]: [More Information Needed]

Uses

Inference

model_path = "/path/to/MiniCPM-V-2"
peft_path = "/path/to/checkpoint-xxx"

model = AutoModel.from_pretrained(model_path, trust_remote_code=True).to(dtype=torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
model.load_adapter(peft_path)
model.eval().cuda()

``

Then, inference
``python
import io
import json
import base64
from PIL import Image

def img2base64(file_name):
   with open(file_name, 'rb') as f:
      encoded_string = base64.b64encode(f.read())
              return encoded_string

      im_64 = img2base64('/mnt/workspace/xray/data/images/99_2.png')
      question = [{"role": "user", "content": "描述一下"}]
      inputs = {"image": im_64, "question": json.dumps(question)}

      image = Image.open(io.BytesIO(base64.b64decode(inputs['image']))).convert('RGB')
      msgs = json.loads(inputs['question'])

      answer, context, _ = model.chat(
           image=image,
           msgs=msgs,
           context=None,
           tokenizer=tokenizer,
           sampling=True,
           temperature=0.7)

print(msgs[-1]["content"]+'\n', answer)
```

Direct Use

Downstream Use [optional]

Out-of-Scope Use

Bias, Risks, and Limitations

Recommendations

How to Get Started with the Model

Training Details

Training Data

Training Procedure

Preprocessing [optional]

Training Hyperparameters

• Training regime: [More Information Needed]

Speeds, Sizes, Times [optional]

Evaluation

Testing Data, Factors & Metrics

Testing Data

Factors

Metrics

Results

Summary

Model Examination [optional]

Environmental Impact

• Hardware Type: [More Information Needed]
• Hours used: [More Information Needed]
• Cloud Provider: [More Information Needed]
• Compute Region: [More Information Needed]
• Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

Compute Infrastructure

Hardware

Software

Citation [optional]

Glossary [optional]

More Information [optional]

Model Card Authors [optional]

Model Card Contact

Framework versions

• PEFT 0.10.0