Rank (r): 16
Scale: 20.0
Dropout: 0.1
Target Modules: self_attn.q_proj, self_attn.k_proj, self_attn.v_proj, self_attn.o_proj
Number of Layers: 32 (out of 32 total)

Usage

Installation

pip install mlx-lm

Loading the Adapters

Option 1: Load from HuggingFace Hub

python

from mlx_lm import load, generate
from mlx_lm.tuner import linear_to_lora_layers
from huggingface_hub import snapshot_download
import json

# Download adapters from HuggingFace
adapter_path = snapshot_download(repo_id="didierlopes/phi-3-mini-4k-instruct-ft-on-my-blog")

# Load base model
model, tokenizer = load("microsoft/Phi-3-mini-4k-instruct")

# Load adapter config
with open(f"{adapter_path}/adapter_config.json", "r") as f:
    adapter_config = json.load(f)

# Freeze base model and apply LoRA layers
model.freeze()
linear_to_lora_layers(
    model, 
    adapter_config["lora_layers"],
    adapter_config["lora_parameters"]
)

# Load the LoRA weights
model.load_weights(f"{adapter_path}/adapters.safetensors", strict=False)

# Generate text
prompt = "<|system|>\nYou are a helpful assistant.<|end|>\n<|user|>\nHello!<|end|>\n<|assistant|>"
response = generate(model, tokenizer, prompt, max_tokens=200)
print(response)

Option 2: Clone and Load Locally

bash

git clone https://huggingface.co/didierlopes/phi-3-mini-4k-instruct-ft-on-my-blog
cd phi-3-mini-4k-instruct-ft-on-my-blog

Then use the same Python code above, replacing adapter_path with your local directory path.

Training Details

These adapters were trained using:

Framework: MLX with LoRA fine-tuning
Hardware: Apple Silicon
Training approach: Parameter-efficient fine-tuning with gradient checkpointing

Files

adapters.safetensors: Final adapter weights
adapter_config.json: LoRA configuration
config.json: Training and model metadata
*.safetensors: Training checkpoint files (optional)

License

These adapters are released under the MIT License. The base model may have its own license requirements.