This is a transformers-compatible version of the LaViDa-LLaDa checkpoint. It allows direct loading via Huggingface transformers APIs for easier inference and integration.

Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained('./lavida-llada-v1.0-instruct/')
model = AutoModelForCausalLM.from_pretrained('./lavida-llada-v1.0-instruct/', torch_dtype=torch.bfloat16)
image_processor = model.get_vision_tower().image_processor

model.resize_token_embeddings(len(tokenizer))
model.tie_weights()

License

Apache 2.0

KonstantinosKK/lavida-llada-v1.0-instruct-hf-transformers

LaViDa-LLaDa v1.0 Instruct (Transformers-Compatible)

Usage

License

KonstantinosKK/
lavida-llada-v1.0-instruct-hf-transformers