This model is a fine-tuned version of
gpt2 on the
baneks dataset for 1 epoch. It achieved
1.9752 loss during training.
Model evaluation has not been performed.
The model is a fine-tuned variant of the base
gpt2 architecture with causal language modeling head.
The model should be used for studying abilities of natural language models to generate jokes.
The model is trained on a list of anecdotes pulled from a few vk communities (see
baneks dataset for more details).
1{
2 'name': 'AdamWeightDecay',
3 'learning_rate': {
4 'module': 'transformers.optimization_tf',
5 'class_name': 'WarmUp',
6 'config': {
7 'initial_learning_rate': 5e-05,
8 'decay_schedule_fn': {
9 'module': 'keras.optimizers.schedules',
10 'class_name': 'PolynomialDecay',
11 'config': {
12 'initial_learning_rate': 5e-05,
13 'decay_steps': 28462,
14 'end_learning_rate': 0.0,
15 'power': 1.0,
16 'cycle': False,
17 'name': None
18 },
19 'registered_name': None
20 },
21 'warmup_steps': 1000,
22 'power': 1.0,
23 'name': None
24 },
25 'registered_name': 'WarmUp'
26 },
27 'decay': 0.0,
28 'beta_1': 0.9,
29 'beta_2': 0.999,
30 'epsilon': 1e-08,
31 'amsgrad': False,
32 'weight_decay_rate': 0.01
33}
