Covenant Extractor Model

Model Description

• Base Model: meta-llama/Llama-3.2-3B-Instruct
• Task: Financial Covenant Extraction
• Training Method: LoRA Fine-tuning
• Language: English
• License: Same as base model

Intended Use

• Extract covenant details from credit agreement sections
• Structure the information into standardized JSON format
• Handle various types of financial covenants (leverage ratios, coverage ratios, etc.)

Input Format

### Instruction: Extract covenant details from the following credit agreement section and structure it into JSON format only.

### Input: Section 4.2:
The Borrower shall maintain a Fixed Charge Coverage Ratio of not less than 1.25:1.00 for any fiscal quarter ending after June 30, 2024.

### Response:

Output Format

Training Details

• Training Method: LoRA (Low-Rank Adaptation)
• LoRA Config:
  • Rank: 16
  • Alpha: 32
  • Target Modules: q_proj, k_proj, v_proj, o_proj
  • Dropout: 0.1
• Training Parameters:
  • Batch Size: 4
  • Gradient Accumulation Steps: 16
  • Learning Rate: 1e-4
  • Number of Epochs: 3
  • Weight Decay: 0.01
  • Max Gradient Norm: 1.0

Limitations

• Only processes English language credit agreements
• Best suited for standard financial covenants
• May require adjustment for complex or non-standard covenant structures

Citation

@misc{covenant-extractor,
  author = {[Bikram Adhikari]},
  title = {Covenant Extractor: Fine-tuned LLM for Financial Covenant Analysis},
  year = {2024}
}