This is a fine-tuned version of Microsoft's Phi-3-Mini-4k-Instruct model, adapted for generating natural language descriptions of payment transactions using LoRA (Low-Rank Adaptation).

Model Description

This model converts structured payment transaction data into clear, customer-friendly language. It was fine-tuned using LoRA on a synthetic payments dataset covering various transaction types.

Training Data

The model was trained on a dataset of 500+ synthetic payment transactions including:

Standard payments (ACH, wire transfer, credit/debit card)
Refunds (full and partial)
Chargebacks
Failed/declined transactions
International transfers with currency conversion
Transaction fees
Recurring payments/subscriptions

Example Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = "microsoft/Phi-3-mini-4k-instruct"
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype="auto",
    device_map="auto"
)

# Load LoRA adapters
model = PeftModel.from_pretrained(model, "aamanlamba/phi3-payments-finetune")
tokenizer = AutoTokenizer.from_pretrained(base_model)

# Generate description
prompt = """<|system|>
You are a financial services assistant that explains payment transactions in clear, customer-friendly language.<|end|>
<|user|>
Convert the following structured payment information into a natural explanation:

inform(transaction_type[payment], amount[1500.00], currency[USD], sender[Acme Corp], receiver[Global Supplies Inc], status[completed], method[ACH], date[2024-10-27])<|end|>
<|assistant|>
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Expected output:

Your ACH payment of $1,500.00 to Global Supplies Inc was successfully completed on October 27, 2024.

Training Details

Training Configuration

Base Model: microsoft/Phi-3-mini-4k-instruct
Fine-tuning Method: LoRA (Low-Rank Adaptation)
LoRA Rank: 16
LoRA Alpha: 32
Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Quantization: 8-bit (training), float16 (inference)
Training Epochs: 3
Learning Rate: 2e-4
Batch Size: 1 (with 8 gradient accumulation steps)
Hardware: NVIDIA RTX 3060 (12GB VRAM)
Training Time: ~35-45 minutes

Training Loss

Initial Loss: ~3.5-4.0
Final Loss: ~0.9-1.2
Validation Loss: ~1.0-1.3

Model Size

LoRA Adapter Size: ~15MB (only the adapter weights, not the full model)
Full Model Size: ~7GB (when combined with base model)

Supported Transaction Types

Payments: Standard payment transactions
Refunds: Full and partial refunds
Chargebacks: Dispute and chargeback processing
Failed Payments: Declined or failed transactions with reasons
International Transfers: Cross-border payments with currency conversion
Fees: Transaction and processing fees
Recurring Payments: Subscriptions and scheduled payments
Reversals: Payment reversals and adjustments

Limitations

Trained on synthetic data - may require additional fine-tuning for production use
Optimized for English language only
Best performance on transaction patterns similar to training data
Not suitable for handling real financial transactions without human oversight
Should not be used as the sole system for financial communication

Ethical Considerations

This model was trained on synthetic, anonymized data only
Does not contain any real customer PII or transaction data
Should be validated for accuracy before production deployment
Implement human review for customer-facing financial communications
Consider regulatory compliance (PCI-DSS, GDPR, etc.) in your jurisdiction

Intended Use

Primary Use Cases:

Generating transaction descriptions for internal systems
Creating customer-friendly payment notifications
Automating payment communication drafts (with human review)
Training and demonstration purposes
Research in financial NLP

Out of Scope:

Direct customer communication without review
Real-time transaction processing without validation
Compliance-critical communications
Medical or legal payment descriptions

How to Cite

If you use this model in your research or application, please cite:

bibtex

@misc{phi3-payments-finetuned,
  author = {aamanlamba},
  title = {Phi-3 Mini Fine-tuned for Payments Domain},
  year = {2024},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/aamanlamba/phi3-payments-finetune}}
}

Training Code

The complete training code and dataset generation scripts are available on GitHub:

Repository: github.com/aamanlamba/phi3-tune-payments
Includes: Dataset generator, training scripts, testing utilities, and deployment guides

Acknowledgements

Base model: Microsoft Phi-3-Mini-4k-Instruct
Fine-tuning method: LoRA: Low-Rank Adaptation of Large Language Models
Training framework: HuggingFace Transformers + PEFT
Inspired by: NVIDIA AI Workbench Phi-3 Fine-tuning Example

License

This model is released under the MIT license, compatible with the base Phi-3 model license.

Contact

For questions or issues, please open an issue on the model repository or contact the author.

Note: This is a demonstration model. Always validate outputs for accuracy before use in production financial systems.