MIXdevAI-gemma3-4B is an experimental merged model based on Google Gemma-3-4B, combining the best qualities of several fine-tuned versions. The model features:

Improved reasoning capabilities (enhanced "thinking")
Strong vision understanding (fully multimodal)
Natural Russian language support
Compact 4B parameter size

This model was created using weight merging with mergekit.

Key Features

Vision support: Works as a full Vision-Language model.
Russian language: Trained on Russian data and prompts.
Improved reasoning: Demonstrates chain-of-thought and analytical abilities.
Compatibility: Fully compatible with transformers and the Gemma-3 format.

Merge Details

Merge Method

The model was assembled using the Linear Merge method (weighted average) with google/gemma-3-4b-it as the base.

Models Merged

The merge includes:

google/gemma-3-4b-it (base multimodal model)
Thinking (fine-tuned version with improved reasoning and Russian language support)

Configuration

yaml

# gemma-3-4b-heretic-merge.yml
merge_method: linear
name: MIXdevAI-gemma3-4B
base_model: google/gemma-3-4b-it

models:
  - model: google/gemma-3-4b-it
    parameters:
      weight: 0.5
  - model: Thinking
    parameters:
      weight: 0.5

dtype: bfloat16
tokenizer_source: union
chat_template: auto

Kolyadual/MIXdevAI-gemma3-4B

MIXdevAI-gemma3-4B

Key Features

Merge Details

Merge Method

Models Merged

Configuration

Kolyadual/
MIXdevAI-gemma3-4B