Model Name: bart-indian-law Author: Mithravardhan P N License: MIT Library: Transformers Base Model: facebook/bart-large-cnn Tags:
generated_from_trainer
legal
summarization
indian-law
bart
fine-tuned
Model Description
bart-indian-law is a fine-tuned version of the facebook/bart-large-cnn model, specifically tailored for summarizing legal texts related to Indian law. It was fine-tuned on a custom dataset comprising summaries of sections from the Indian Penal Code (IPC) and articles from the Constitution of India. The model is designed to generate concise and accurate summaries of legal provisions, making it a valuable tool for legal professionals, researchers, and students.
The model leverages the BART architecture, known for its strong performance in sequence-to-sequence tasks like text summarization. It was trained to capture the nuances of Indian legal language and produce summaries that retain critical details while being clear and succinct.
Intended Uses & Limitations
Intended Uses
Legal Text Summarization: Generate summaries of Indian Penal Code sections and Constitution articles.
Legal Research: Assist researchers in quickly understanding key points of legal provisions.
Educational Purposes: Help students and educators summarize complex legal texts for study or teaching.
Document Analysis: Support legal professionals in summarizing lengthy legal documents related to Indian law.
Limitations
Domain-Specific: The model is fine-tuned on Indian legal texts (IPC and Constitution) and may not generalize well to other domains or legal systems.
Dataset Scope: The training data is limited to summaries of IPC sections and Constitution articles, which may not cover all aspects of Indian law.
Language: The model is trained on English texts and may not handle multilingual legal documents effectively.
Potential Bias: The model may reflect biases present in the training data or the base model (facebook/bart-large-cnn).
Factual Accuracy: While designed for summarization, the model may occasionally produce incomplete or overly simplified summaries, requiring human verification for critical applications.
Training and Evaluation Data
The model was trained on a custom dataset consisting of:
Indian Penal Code (IPC) Sections: Text descriptions of IPC sections paired with their summaries, sourced from ipc_sections_summary.json.
Constitution of India Articles: Text descriptions of constitutional articles paired with their summaries, sourced from constitution_articles_summary.json.
The dataset was split into training (90%) and validation (10%) sets using a random seed of 42 for reproducibility. The training data included structured entries with a "text" field (e.g., "IPC Section X: Title" or "Constitution Article Y: Title") and a corresponding "summary" field.
Dataset Statistics
Total Entries: Not explicitly specified, but the training process completed 3475 steps with a batch size of 4 over 5 epochs, suggesting approximately 2776 training examples (3475 steps * 4 batch size / 5 epochs).
Validation Set: Approximately 10% of the total dataset, used for evaluation during training.
Training Procedure
The model was fine-tuned using the Hugging Face transformers library with the following setup:
Mixed Precision Training: Enabled (Native AMP with FP16 on CUDA)
Output Directory: bart-indian-law
Evaluation Strategy: Per epoch
Save Strategy: Per epoch, with a limit of 2 checkpoints
Predict with Generate: Enabled
Push to Hub: Enabled (model uploaded to mithra99/bart-indian-law)
Training Results
The model was trained for 3475 steps (5 epochs), with the following loss metrics:
Epoch
Training Loss
Validation Loss
1
0.8791
0.5521
2
0.4522
0.5147
3
0.2968
0.5202
4
0.2462
0.5484
5
0.2063
0.5788
Final Validation Loss: 0.5788
Observation: The training loss consistently decreased, indicating effective learning. The validation loss reached its lowest point at epoch 2 (0.5147) but slightly increased in later epochs, suggesting potential overfitting or the need for early stopping in future iterations.
Framework Versions
Transformers: 4.50.3
PyTorch: 2.6.0+cu124
Datasets: 3.5.0
Tokenizers: 0.21.1
Training Code
The training pipeline included the following key steps:
Data Preparation: Loaded and normalized JSON data from IPC and Constitution datasets, creating a unified dataset with "text" and "summary" fields.
Model Setup: Initialized the BART model and tokenizer from facebook/bart-large-cnn, along with a data collator for sequence-to-sequence tasks.
Tokenization: Preprocessed the dataset by tokenizing inputs (max length 1024) and targets (max length 128), with truncation and padding.
Training Configuration: Defined training arguments for the Seq2SeqTrainer, including mixed precision training and hub integration.
Training Execution: Ran the training loop with periodic evaluation and checkpoint saving.
Model Export: Saved and uploaded the model, tokenizer, and configuration files to the Hugging Face Hub (mithra99/bart-indian-law).
The training code included error handling, safe file uploads, and support for multiple model formats (PyTorch, SafeTensors, and optionally TensorFlow).
Evaluation Metrics
Loss: The primary metric reported is the validation loss, with a final value of 0.5788.
Additional Metrics: No BLEU, ROUGE, or other summarization-specific metrics were reported in the provided results. Future evaluations could include these to better assess summary quality.
How to Use the Model
Prerequisites
Install the required libraries:
pip install transformers torch datasets
Example Code
python
1from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
23# Load the model and tokenizer4model_name ="mithra99/bart-indian-law"5tokenizer = AutoTokenizer.from_pretrained(model_name)6model = AutoModelForSeq2SeqLM.from_pretrained(model_name)78# Input text (example: IPC section description)9input_text ="IPC Section 302: Punishment for murder. Whoever commits murder shall be punished with death, or imprisonment for life, and shall also be liable to fine."1011# Tokenize input12inputs = tokenizer(input_text, max_length=1024, truncation=True, padding="max_length", return_tensors="pt")1314# Generate summary15summary_ids = model.generate(16 inputs["input_ids"],17 max_length=128,18 min_length=30,19 num_beams=4,20 length_penalty=2.0,21 early_stopping=True,22 no_repeat_ngram_size=323)2425# Decode summary26summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)27print("Summary:", summary)
Expected Output
For the above input, the model might produce a summary like:
Summary: IPC Section 302 prescribes death or life imprisonment and a fine for committing murder.
Notes
Adjust max_length, min_length, and other generation parameters based on your needs.
Ensure input texts are structured similarly to the training data (e.g., "IPC Section X: Title" or "Constitution Article Y: Title") for best results.
The model is hosted on the Hugging Face Hub and can be accessed directly: mithra99/bart-indian-law.
Model Artifacts
The following files are available on the model’s Hugging Face Hub repository:
config.json: Model configuration.
model.safetensors: Model weights in SafeTensors format.
pytorch_model.bin: Model weights in PyTorch format.
