Views
No views yet
deberta-v3-large model on a dataset of ITIL and DOLCE-related texts, with a total of 5500 chunks (256 tokens each).1augmetations = {'class': 'ITIL or Ontology, one word {ITIL} or {Ontology}',
2 'topic': 'Topic in 5 words',
3 'nearest ontology': 'find nearest analogy, process, class or entity from DOLCE ontology domain. 1 words, short sentence to explain, max 20 words',
4 'nearest ITIL': 'find nearest analogy, process, class or entity from enterprise IT service domain. 1 words, short sentence to explain, max 20 words',
5 'summary': 'text esscence in 20 words, no intro, just meaning'}1training_args = TrainingArguments(
2 output_dir="./deberta-finetuned-ITIL-DOLCE",
3 overwrite_output_dir=True,
4 eval_strategy="steps",
5 per_device_train_batch_size=16,
6 per_device_eval_batch_size=2,
7 num_train_epochs=100,
8 save_steps=5_000,
9 eval_steps=500,
10 logging_steps=500,
11 fp16=True
12)Trained on raw data
Evaluation results: {'eval_loss': 3.6031947135925293, 'eval_runtime': 26.6242, 'eval_samples_per_second': 21.71, 'eval_steps_per_second': 10.855, 'epoch': 100.0}
Perplexity: 36.7153422841205
Trained on augmented data
Evaluation results: {'eval_loss': 1.4712743759155273, 'eval_model_preparation_time': 0.0065, 'eval_runtime': 76.2792, 'eval_samples_per_second': 22.732, 'eval_steps_per_second': 2.845}
Perplexity: 4.3547812347309875