import openvino_genai as ov_genai

# Load the model
pipe = ov_genai.VLMPipeline("Big-Tiger-Gemma-27B-v3-int4_asym-ov", "CPU")

# Generate text
config = ov_genai.GenerationConfig()
config.max_new_tokens = 100

response = pipe.generate("What is the capital of France?", generation_config=config)
print(response)

Requirements

pip install openvino-genai openvino-tokenizers

Performance

This INT4 quantized model provides significant memory savings compared to the original FP16/FP32 model while maintaining good inference quality. Optimized for CPU inference with OpenVINO.

License

Follows the same license as the original model: TheDrummer/Big-Tiger-Gemma-27B-v3

DudePls/Big-Tiger-Gemma-27B-v3-int4-asym-ov

Big-Tiger-Gemma-27B-v3 OpenVINO INT4

Model Details

Model Components

Usage

Requirements

Performance

License

DudePls/
Big-Tiger-Gemma-27B-v3-int4-asym-ov