hashtag – AI Model by jjae | AlphaNeural AI

You can deploy this model and start earning money today!

jjae/
hashtag

like

0

text2text-generation

generated_from_trainer

gogamza/kobart-summarization

endpoints_compatible

Views

No views yet

modelling

This model is a fine-tuned version of gogamza/kobart-summarization on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.5862

Model description

This model generates hash tag from input text.

Training and evaluation data

This model was trained by the self-instruction process. All data used for fine-tuning this model were generated by chatGPT 3.5.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5.6e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 300
num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss
1.8136	1.42	500	0.6526
0.4651	2.85	1000	0.5862
0.2643	4.27	1500	0.6752
0.1642	5.7	2000	0.6840
0.1078	7.12	2500	0.7554

Framework versions

Transformers 4.37.1
Pytorch 2.1.2+cu118
Datasets 2.16.1
Tokenizers 0.15.0

How to Get Started with the Model

Use the code below to get started with the model. You can adjust hyperparameters to fit on your data.

python

from transformers import PreTrainedTokenizerFast, BartForConditionalGeneration
tokenizer = PreTrainedTokenizerFast.from_pretrained("jjae/kobart-hashtag")
model = BartForConditionalGeneration.from_pretrained("jjae/kobart-hashtag")

def make_tag(text):
  input_ids = tokenizer.encode(text, return_tensors="pt").to(device)
  output = model.generate(input_ids = input_ids, bos_token_id = model.config.bos_token_id,
                          eos_token_id = model.config.eos_token_id, length_penalty = 3.0, max_length = 50, num_beams = 4)
  decoded_output = tokenizer.decode(output[0], skip_special_tokens=True)
  return decoded_output