Training Loss	Epoch	Step	Validation Loss
6.6606	0.3019	2000	6.4737
4.7655	0.6039	4000	4.5854
2.7083	0.9058	6000	2.6127
1.6726	1.2077	8000	1.6041
1.561	1.5097	10000	1.5083
1.4795	1.8116	12000	1.4200
1.3174	2.1135	14000	1.2703
1.2471	2.4155	16000	1.1988
1.1931	2.7174	18000	1.1477
1.1343	3.0193	20000	1.1033
1.1029	3.3213	22000	1.0772
1.082	3.6232	24000	1.0502
1.0626	3.9251	26000	1.0297
1.0371	4.2271	28000	1.0193
1.0141	4.5290	30000	0.9944
1.0069	4.8309	32000	0.9864
0.9783	5.1329	34000	0.9786
0.9733	5.4348	36000	0.9587
0.9709	5.7367	38000	0.9556
0.9394	6.0386	40000	0.9453
0.9392	6.3406	42000	0.9364
0.9315	6.6425	44000	0.9239
0.9241	6.9444	46000	0.9219
0.9158	7.2464	48000	0.9187
0.904	7.5483	50000	0.9148
0.9072	7.8502	52000	0.9043
0.8835	8.1522	54000	0.9047
0.8844	8.4541	56000	0.8969
0.8854	8.7560	58000	0.8900
0.8542	9.0580	60000	0.8951
0.8632	9.3599	62000	0.8867
0.8626	9.6618	64000	0.8816
0.8656	9.9638	66000	0.8756
0.842	10.2657	68000	0.8889
0.8506	10.5676	70000	0.8771
0.851	10.8696	72000	0.8776
0.8236	11.1715	74000	0.8739
0.8359	11.4734	76000	0.8720
0.8323	11.7754	78000	0.8678
0.8126	12.0773	80000	0.8653
0.8162	12.3792	82000	0.8659
0.8168	12.6812	84000	0.8716
0.8221	12.9831	86000	0.8677
0.8009	13.2850	88000	0.8617
0.8065	13.5870	90000	0.8627
0.8049	13.8889	92000	0.8593
0.7882	14.1908	94000	0.8600
0.7941	14.4928	96000	0.8625
0.7977	14.7947	98000	0.8542
0.775	15.0966	100000	0.8567
0.7833	15.3986	102000	0.8597
0.79	15.7005	104000	0.8524
0.7862	16.0024	106000	0.8511
0.7665	16.3043	108000	0.8512
0.7719	16.6063	110000	0.8484
0.7787	16.9082	112000	0.8472
0.7559	17.2101	114000	0.8513
0.7634	17.5121	116000	0.8432
0.7665	17.8140	118000	0.8403
0.747	18.1159	120000	0.8543
0.7499	18.4179	122000	0.8471
0.7545	18.7198	124000	0.8431
0.7343	19.0217	126000	0.8444
0.74	19.3237	128000	0.8422
0.7438	19.6256	130000	0.8384
0.7474	19.9275	132000	0.8489
0.7302	20.2295	134000	0.8396
0.7366	20.5314	136000	0.8368
0.7346	20.8333	138000	0.8363
0.7197	21.1353	140000	0.8304
0.7268	21.4372	142000	0.8416
0.7291	21.7391	144000	0.8403
0.7116	22.0411	146000	0.8345
0.7131	22.3430	148000	0.8335
0.7209	22.6449	150000	0.8347
0.7193	22.9469	152000	0.8358
0.7087	23.2488	154000	0.8327
0.7117	23.5507	156000	0.8349
0.7166	23.8527	158000	0.8297
0.6993	24.1546	160000	0.8307
0.7031	24.4565	162000	0.8263
0.7082	24.7585	164000	0.8357
0.6888	25.0604	166000	0.8289
0.6959	25.3623	168000	0.8326
0.6978	25.6643	170000	0.8288
0.6998	25.9662	172000	0.8230
0.6913	26.2681	174000	0.8240
0.6924	26.5700	176000	0.8264
0.6938	26.8720	178000	0.8208
0.683	27.1739	180000	0.8237
0.686	27.4758	182000	0.8271
0.686	27.7778	184000	0.8166
0.6722	28.0797	186000	0.8290
0.6796	28.3816	188000	0.8204
0.6791	28.6836	190000	0.8228
0.6844	28.9855	192000	0.8238
0.6714	29.2874	194000	0.8255
0.6745	29.5894	196000	0.8213
0.6756	29.8913	198000	0.8262
0.6661	30.1932	200000	0.8208
0.669	30.4952	202000	0.8198
0.6685	30.7971	204000	0.8239
0.6577	31.0990	206000	0.8202
0.662	31.4010	208000	0.8229
0.667	31.7029	210000	0.8179
0.664	32.0048	212000	0.8150
0.66	32.3068	214000	0.8236
0.6591	32.6087	216000	0.8133
0.6643	32.9106	218000	0.8137
0.6501	33.2126	220000	0.8171
0.6518	33.5145	222000	0.8149
0.6586	33.8164	224000	0.8161
0.6409	34.1184	226000	0.8184
0.6485	34.4203	228000	0.8199
0.6545	34.7222	230000	0.8163
0.6408	35.0242	232000	0.8196
0.6425	35.3261	234000	0.8138
0.6444	35.6280	236000	0.8112
0.6478	35.9300	238000	0.8105
0.6369	36.2319	240000	0.8154
0.6392	36.5338	242000	0.8212
0.6441	36.8357	244000	0.8114
0.6325	37.1377	246000	0.8061
0.6381	37.4396	248000	0.8125
0.6369	37.7415	250000	0.8110
0.6276	38.0435	252000	0.8114
0.6322	38.3454	254000	0.8066
0.6283	38.6473	256000	0.8109
0.6367	38.9493	258000	0.8119
0.6233	39.2512	260000	0.8133
0.6306	39.5531	262000	0.8120
0.6287	39.8551	264000	0.8104
0.6201	40.1570	266000	0.8069
0.6208	40.4589	268000	0.8066
0.6282	40.7609	270000	0.8104
0.6187	41.0628	272000	0.8112
0.6194	41.3647	274000	0.8012
0.621	41.6667	276000	0.8060
0.6226	41.9686	278000	0.8076
0.6151	42.2705	280000	0.8041
0.6163	42.5725	282000	0.8092
0.6177	42.8744	284000	0.8043
0.6094	43.1763	286000	0.8101
0.6131	43.4783	288000	0.8027
0.6114	43.7802	290000	0.8043
0.6065	44.0821	292000	0.8052
0.6102	44.3841	294000	0.8093
0.6105	44.6860	296000	0.8010
0.6132	44.9879	298000	0.8047
0.6038	45.2899	300000	0.8053
0.6086	45.5918	302000	0.8073
0.6078	45.8937	304000	0.7995
0.5997	46.1957	306000	0.8066
0.6024	46.4976	308000	0.8055
0.6042	46.7995	310000	0.8122
0.5983	47.1014	312000	0.8014
0.6017	47.4034	314000	0.8045
0.6026	47.7053	316000	0.8068
0.596	48.0072	318000	0.7992
0.5987	48.3092	320000	0.8055
0.5965	48.6111	322000	0.8014
0.6023	48.9130	324000	0.8030
0.5931	49.2150	326000	0.8020
0.5943	49.5169	328000	0.8031
0.5959	49.8188	330000	0.7998
0.587	50.1208	332000	0.7965
0.5924	50.4227	334000	0.8010
0.5927	50.7246	336000	0.7990
0.5882	51.0266	338000	0.7995
0.5881	51.3285	340000	0.7974
0.592	51.6304	342000	0.8014
0.5895	51.9324	344000	0.8009
0.5849	52.2343	346000	0.8021
0.5846	52.5362	348000	0.7985
0.585	52.8382	350000	0.8040
0.5857	53.1401	352000	0.7980
0.5828	53.4420	354000	0.7974
0.585	53.7440	356000	0.7998
0.579	54.0459	358000	0.7956
0.5815	54.3478	360000	0.7979
0.5803	54.6498	362000	0.8016
0.5845	54.9517	364000	0.7974
0.5778	55.2536	366000	0.7995
0.5803	55.5556	368000	0.7974
0.5794	55.8575	370000	0.7955
0.5776	56.1594	372000	0.7964
0.5758	56.4614	374000	0.7955
0.5782	56.7633	376000	0.7911
0.5752	57.0652	378000	0.7956
0.5765	57.3671	380000	0.7940
0.573	57.6691	382000	0.7960
0.5767	57.9710	384000	0.7955
0.5739	58.2729	386000	0.7963
0.5733	58.5749	388000	0.7985
0.5702	58.8768	390000	0.7953
0.5719	59.1787	392000	0.8007
0.5723	59.4807	394000	0.7966
0.5711	59.7826	396000	0.7979
0.5692	60.0845	398000	0.7934
0.5707	60.3865	400000	0.7954

Framework versions

Transformers 4.45.1
Pytorch 2.4.1+cu121
Datasets 3.0.1
Tokenizers 0.20.0

psktoure/BERT_TS_WordPiece_phonetic_wikitext_0.9

BERT_TS_WordPiece_phonetic_wikitext_0.9

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

psktoure/
BERT_TS_WordPiece_phonetic_wikitext_0.9