This is a highly specialized, local-first Vision-Language Model (VLM) designed to estimate nutritional macros and physical weight from images of food. The model is fine-tuned to output strict, minified JSON payloads containing calories, macros (protein, fat, carbohydrates), and estimated dry/physical weight in grams.

It has been natively converted to Apple's MLX format and quantized to 4-bit (INT4), reducing its memory footprint to roughly ~1.5GB. This allows for lightning-fast, entirely offline, on-device inference on Apple Silicon hardware (macOS, iOS, iPadOS) using the Neural Engine.

Base Architecture: Qwen3.5-2B (Omni/Unified Vision-Language architecture)
Framework: MLX (mlx-vlm)
Quantization: INT4 (4-bit)
Fine-Tuning Method: LoRA (via Unsloth), merged to 16-bit prior to MLX conversion.
Primary Task: Multimodal Image-to-JSON parsing.

Intended Use

Primary Use Case: Seamless integration into native Swift/Apple ecosystem applications requiring on-device visual food analysis without relying on cloud APIs.
Input: A user prompt (e.g. food description) + a standard RGB food image.
Output: A standardized JSON schema containing absolute nutritional values.

Training Data & Credit

This model was fine-tuned on a custom dataset synthesized from a massive corpus of recipe data and user-uploaded images.

Dataset Source: Modified from the Food.com Recipes and Reviews dataset. The text macros were mathematically processed to derive estimated physical weights, combined with dynamically downloaded web images, and formatted into an OpenAI-style JSONL conversation structure.
Dataset License: Public Domain (CC0)

Licensing & Attribution

Model License: Apache License 2.0 (Inherited from the Qwen 3.5 2B base model).
Attribution: All base architectural credit belongs to the Qwen team at Alibaba Cloud.

Usage Example (MLX)

python

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import get_message_profile
from mlx_vlm.utils import load_image

# Load the INT4 MLX model
model, processor = load("alexsofonea/Qwen3.5_2B_MLX_Nutrition")

# Format input
image = load_image("path/to/food.jpg")
prompt = "Output JSON.\nEstimate."

# Generate JSON
output = generate(model, processor, prompt, image, max_tokens=150, temperature=0.0)
print(output)

About Developer

JSON

{
  "developer": {
    "name": "Alex",
    "age": 19,
    "location": "Munchen, Germany",
    "company": "Tecky",
    "role": "Founder",
    "background": [
      "Developer since age 7",
      "Filmmaker"
    ]
  },
  "mission": "Building local-first AI agents and integrating modern cinematic design into native software."
}

Contact: