You can deploy this model and start earning money today!

RichardErkhov/
nimamegh_-_roberta_cnn_legal-gguf

Name: nimamegh_-_roberta_cnn_legal-gguf
Rating: 2.5 (147 reviews)

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

roberta_cnn_legal - GGUF

Model creator: https://huggingface.co/nimamegh/
Original model: https://huggingface.co/nimamegh/roberta_cnn_legal/

Name	Quant method	Size
roberta_cnn_legal.Q2_K.gguf	Q2_K	0.15GB
roberta_cnn_legal.IQ3_XS.gguf	IQ3_XS	0.16GB
roberta_cnn_legal.IQ3_S.gguf	IQ3_S	0.16GB
roberta_cnn_legal.Q3_K_S.gguf	Q3_K_S	0.16GB
roberta_cnn_legal.IQ3_M.gguf	IQ3_M	0.17GB
roberta_cnn_legal.Q3_K.gguf	Q3_K	0.18GB
roberta_cnn_legal.Q3_K_M.gguf	Q3_K_M	0.18GB
roberta_cnn_legal.Q3_K_L.gguf	Q3_K_L	0.2GB
roberta_cnn_legal.IQ4_XS.gguf	IQ4_XS	0.2GB
roberta_cnn_legal.Q4_0.gguf	Q4_0	0.2GB
roberta_cnn_legal.IQ4_NL.gguf	IQ4_NL	0.2GB
roberta_cnn_legal.Q4_K_S.gguf	Q4_K_S	0.2GB
roberta_cnn_legal.Q4_K.gguf	Q4_K	0.22GB
roberta_cnn_legal.Q4_K_M.gguf	Q4_K_M	0.22GB
roberta_cnn_legal.Q4_1.gguf	Q4_1	0.22GB
roberta_cnn_legal.Q5_0.gguf	Q5_0	0.24GB
roberta_cnn_legal.Q5_K_S.gguf	Q5_K_S	0.24GB
roberta_cnn_legal.Q5_K.gguf	Q5_K	0.25GB
roberta_cnn_legal.Q5_K_M.gguf	Q5_K_M	0.25GB
roberta_cnn_legal.Q5_1.gguf	Q5_1	0.26GB
roberta_cnn_legal.Q6_K.gguf	Q6_K	0.27GB
roberta_cnn_legal.Q8_0.gguf	Q8_0	0.35GB

Original model description:

license: mit language:

en pipeline_tag: sentence-similarity datasets:
darrow-ai/LegalLensNLI metrics:
f1 base_model:
ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli library_name: transformers

roberta_cnn_legal

Overview

This repository hosts the uOttawa model developed for Subtask B (Legal Natural Language Inference) in the LegalLens-2024 shared task. The task focuses on classifying relationships between legal texts, such as determining if a premise (e.g., a summary of a legal complaint) entails, contradicts, or is neutral with respect to a hypothesis (e.g., an online review).

Model Details

Model Type: Transformer-based model combined with a Convolutional Neural Network (CNN)
Framework: PyTorch, Transformers library
Training Data: LegalLensNLI dataset provided by the LegalLens-2024 organizers
Architecture: Integration of RoBERTa (ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli) with a custom CNN for keyword pattern detection
Use Case: Classifying relationships between legal documents for applications like legal case matching and automated reasoning

Model Architecture

The model architecture consists of:

RoBERTa model: Responsible for capturing contextual information from the input text.
CNN model: Used for keyword detection, including an embedding layer and three convolutional layers with filter sizes (2, 3, 4).
Fully connected layer: Combines the outputs from RoBERTa and CNN for the final classification.

Installation

To use this model, clone this repository and make sure to have the following installed:

bash

pip install torch
pip install transformers

Quick Start

Load the model and run inference using the Hugging Face Transformers library:

code

from transformers import AutoTokenizer, AutoModelForSequenceClassification

# Load the model and tokenizer
model = AutoModelForSequenceClassification.from_pretrained("nimamegh/roberta_cnn_legal")
tokenizer = AutoTokenizer.from_pretrained("nimamegh/roberta_cnn_legal")

# Example inputs
premise = "The cat is on the mat."
hypothesis = "The animal is on the mat."
inputs = tokenizer(premise, hypothesis, return_tensors='pt')

# Get predictions
outputs = model(**inputs)
predictions = outputs.logits.argmax(dim=-1)

# Print the prediction result
print("Predicted class:", predictions.item())

# Interpretation (optional)
label_map = {0: "Entailment", 1: "Neutral", 2: "Contradiction"}
print("Result:", label_map[predictions.item()])

Training Configuration

Learning Rate: 2e-5
Batch Size: 4 (train and evaluation)
Number of Epochs: 20
Weight Decay: 0.01
Optimizer: AdamW
Trainer Class: Used for fine-tuning with early stopping and warmup steps

Evaluation Metrics

The model was evaluated using an F1-score across multiple domains in the validation set:

Average F1-score: 88.6%

Result

Performance on Hidden Test Set: F1-score of 0.724, achieving 5th place in the LegalLens-2024 competition.
Comparison:
- Falcon 7B: 81.02% (average across domains)
- RoBERTa base: 71.02% (average)
- uOttawa Model: 88.6% (average on validation)

Citation

bibtex

@misc{meghdadi2024uottawalegallens2024transformerbasedclassification,
      title={uOttawa at LegalLens-2024: Transformer-based Classification Experiments}, 
      author={Nima Meghdadi and Diana Inkpen},
      year={2024},
      eprint={2410.21139},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2410.21139}, 
}

RichardErkhov/nimamegh_-_roberta_cnn_legal-gguf