You can deploy this model and start earning money today!

xn6o/
lora-vit-large-patch16-224-in21k-r64-imagenet1k

Name: lora-vit-large-patch16-224-in21k-r64-imagenet1k
Availability: InStock

safetensors

Views

No views yet

🚀 ViT-B/16 LoRA (r=64) — ImageNet-1K Fine-Tuned Model

🚀 ViT-B/16 LoRA (r=64) — ImageNet-1K 微调模型

📌 Overview | 概述

This repository provides a LoRA fine-tuned version of google/vit-large-patch16-224-in21k on ImageNet-1K. 本仓库提供了一个基于 google/vit-large-patch16-224-in21k 模型，使用 LoRA 技术在 ImageNet-1K 数据集上微调后的版本。

The model is optimized with low-rank adaptation (LoRA) while updating only ~2% of total parameters, achieving strong accuracy comparable to full fine-tuning. 模型使用 低秩适配 LoRA 技术进行高效训练，仅更新约 8% 参数量，并取得了接近甚至超越全量微调的效果。

🏆 Results | 模型性能

Metric 指标	Score 分数
Top-1 Accuracy	82.66%
Top-5 Accuracy	93.10%

✔️ Achieved with early stopping ✔️ Training stability verified ✔️ Excellent compute-to-performance ratio

📂 Training Configuration | 训练配置

Item 项目	Value 配置
Base Model	`google/vit-large-patch16-224-in21k`
Method	LoRA
LoRA Rank (r)	64
Trainable Params	28,311,552 （trainable%: 8.5112）
Dataset	ImageNet-1K
Epochs	20 (Early Stop at 16)
Optimizer	AdamW
Scheduler	CosineAnnealingWarmRestarts
Precision	AMP (FP16/BF16)
Input Resolution	224×224

📥 How to Load | 模型加载方式

🔹 1️⃣ Load base model + LoRA weights

🔹 1️⃣ 加载基础模型与 LoRA 权重

python

from transformers import ViTForImageClassification
from peft import PeftModel
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"

base = ViTForImageClassification.from_pretrained(
    "google/vit-large-patch16-224-in21k",
    num_labels=1000
)

model = PeftModel.from_pretrained(base, "xn6o/lora-vit-large-patch16-224-in21k-r64-imagenet1k")
model.to(device).eval()

🔹 2️⃣ Load classifier head (if provided)

🔹 2️⃣ 加载自定义分类头（如果提供）

python

state_dict = torch.load("vit_classifier_best.pt", map_location=device)
model.base_model.classifier.load_state_dict(state_dict)

🔧 Export merged inference model (optional)

🔧 导出合并后的推理模型（可选）

This step merges LoRA weights into the base model so no adapter is required at inference time. 此步骤将 LoRA 权重 永久合并，推理时无需 LoRA 适配器模块。

python

merged = model.merge_and_unload()
merged.save_pretrained("vit_lora_merged_best")

🖼 Inference Example | 推理示例

python

from PIL import Image
from torchvision import transforms
import torch.nn.functional as F

img = Image.open("your_image.jpg")

tfm = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406],
                         [0.229, 0.224, 0.225]),
])

pixel_values = tfm(img).unsqueeze(0).to(device)

with torch.no_grad():
    logits = model(pixel_values=pixel_values).logits
    probs = F.softmax(logits, dim=1)

top5 = probs.topk(5)
print("Top-5 Classes:", top5.indices.squeeze().tolist())
print("Top-5 Probabilities:", top5.values.squeeze().tolist())

📁 Repository Structure | 仓库结构说明

|-- adapter/              # LoRA adapter weights
|-- vit_classifier.pt     # Fine-tuned classifier head
|-- README.md             # This model card

⚠ Notes | 说明注意

This model is trained only on ImageNet-1K (no external data). 该模型 仅使用 ImageNet-1K 数据训练，未使用额外数据。
Suitable for downstream transfer learning. 可作为迁移学习基础模型。
For commercial use, check Imagenet licensing terms. 商业使用需遵守 ImageNet 数据许可协议。

📜 License | 许可证

Follows original ViT model license: 遵循原始 ViT 模型的开源许可协议。

🙌 Acknowledgements | 致谢

Google ViT
PEFT
Hugging Face Transformers
LoRA Paper

⭐ Support | 支持与交流

If this model is helpful, please star ⭐ the repo or open an issue! 如果这个模型对你有帮助，欢迎 点个 Star ⭐ 或提 Issue！