Currently it is just a proof of concept and does a great job identifiying Olivine It currently is not ready for a production enviroment but the results are promising, with an improved dataset I'm confident better results could be acheived.

Training hyperparameters

The following hyperparameters were used during training:

optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 3e-05, 'decay_steps': 300, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}
training_precision: float32

Training results

Train Loss	Validation Loss	Train Accuracy	Epoch
1.6519	1.7095	0.2	0
1.5905	1.6747	0.2	1
1.5690	1.6342	0.2	2
1.5170	1.5931	0.2	3
1.4764	1.5528	0.6	4
1.3835	1.5079	0.6	5
1.3420	1.4717	0.6	6
1.3171	1.4232	0.6	7
1.2897	1.3905	0.6	8
1.2702	1.3794	0.6	9
1.2023	1.3351	0.6	10
1.1480	1.3384	0.6	11
1.1434	1.3419	0.6	12
1.0499	1.3226	0.6	13
1.0672	1.2647	0.6	14
1.0526	1.1533	0.6	15
1.0184	1.1546	0.6	16
0.9505	1.2491	0.6	17
0.9578	1.2809	0.4	18
0.8890	1.1803	0.6	19

Framework versions

Transformers 4.32.1
TensorFlow 2.12.0
Datasets 2.14.4
Tokenizers 0.13.3