import grug
from grug.backends.modern import ModernBackend

comp = grug.Compressor(ModernBackend(model_name='akshayballal/grug-mbert-control-meetingbank-v2'))
result = comp.compress(text, rate=0.4)

Training

json

{
  "model_name": "bert-base-multilingual-cased",
  "epochs": 6,
  "learning_rate": 1e-05,
  "batch_size": 32,
  "max_length": 512,
  "weight_decay": 0.01,
  "warmup_ratio": 0.06,
  "device": "cuda",
  "seed": 0,
  "push_to": "akshayballal/grug-mbert-control-meetingbank-v2",
  "select_on": "val_f1",
  "patience": 2,
  "cs_weight": 0.0
}

Final epoch

json

{
  "epoch": 5,
  "train_loss": 0.32868274898842126,
  "val_loss": 0.3353261986993394,
  "val_accuracy": 0.8496049377751864,
  "val_precision": 0.7996827124859441,
  "val_recall": 0.777914413912268,
  "val_f1": 0.7886483794751173,
  "val_perplexity": 1.398396465940305
}

The training corpus is CC-BY-NC-SA-4.0, so this model inherits a non-commercial constraint.