# Install required tools
pip install huggingface_hub

# Install Git LFS (Large File Support)
# macOS with Homebrew:
brew install git-lfs

# Initialize Git LFS
git lfs install

# Clone the repository with model files
git clone https://huggingface.co/dungnvt/Qwen3-4B-Thinking-2507-LUT444-ctx4096_0.3.4-anemll

Extract model files:

bash

# Navigate to cloned directory
cd Qwen3-4B-Thinking-2507-LUT444-ctx4096_0.3.4-anemll

# Pull LFS files (model weights)
git lfs pull

# Extract CoreML model files
find . -type f -name "*.zip" -exec unzip {} \;

Install dependencies:

pip install coremltools transformers

Coremltools:

See coremltools installation guide at https://coremltools.readme.io/v4.0/docs/installation

How to Run

Basic chat interface:

python chat.py --meta ./meta.yaml

Full conversation mode with history:

python chat_full.py --meta ./meta.yaml

Note: The first time the model loads, macOS will take some time to place it on the device. Subsequent loads will be instantaneous. Use Ctrl-D to exit, Ctrl-C to interrupt inference.

Quick Start

Test in iOS/macOS App

Try our sample Chat-Bot app on TestFlight:

Install TestFlight from App Store
Join beta test: TestFlight Link
App includes a small demo model pre-installed
You can add custom models via HuggingFace URLs

[!Note]

The TestFlight app works on both iOS and macOS

Demonstrates proper model integration and provides a reference implementation

iOS requires unzipped MLMODELC files and config.json for offline tokenizer

macOS supports both zipped and unzipped model formats