KGSAGAR/

Sarvam-1-text-normalization-3r

This model is a starting point of experiment on training SLM on desired task of text normalization

Use the code below to get started with the model.

-- bf16 mixed precision -- 4 bit quantized

Step Training Loss Validation Loss 1 3.166600 2.925514 2 2.839400 2.907682 3 2.987000 2.872351 4 2.739300 2.820903 5 2.800800 2.770396 6 2.827800 2.720905 7 2.775500 2.672389 8 2.718000 2.624753 9 2.759500 2.577636 10 2.685700 2.531116 11 2.714900 2.485125 12 2.661100 2.439770 13 2.486600 2.395308 14 2.264400 2.351529 15 2.411000 2.309095 16 2.406700 2.267282 17 2.379900 2.226102 18 2.278600 2.185202 19 2.084900 2.144496 20 2.167400 2.103827 21 2.150100 2.062842 22 2.153600 2.021548 23 1.904500 1.979849 24 2.036300 1.938160 25 2.003500 1.897460 26 1.925700 1.858673 27 1.917400 1.823796 28 1.801900 1.787577 29 1.780000 1.745841 30 1.989900 1.702347 31 1.812200 1.661288 32 1.838700 1.622226 33 1.639400 1.582157 34 1.638400 1.540874 35 1.626800 1.500206 36 1.608600 1.462778 37 1.433900 1.426152 38 1.583300 1.388150 39 1.369400 1.349104 40 1.631500 1.311011 41 1.306200 1.274240 42 1.365200 1.238086 43 1.296300 1.203870 44 1.511700 1.172958 45 1.705600 1.144534 46 1.033700 1.116540 47 1.038900 1.089593 48 1.218700 1.064211 49 1.152100 1.040486 50 0.978100 1.020181 51 0.971000 1.003481 52 0.986600 0.988935 53 1.003700 0.975595 54 0.889700 0.963460 55 0.948800 0.952305 56 1.120200 0.942338 57 0.775800 0.933127 58 0.804700 0.924533 59 0.894900 0.916417 60 0.912300 0.909007 61 0.863000 0.902386 62 0.929200 0.896640 63 0.743500 0.891563 64 0.941600 0.887086 65 0.970500 0.883202 66 1.073300 0.879857 67 0.968600 0.876835 68 0.865200 0.874028 69 0.864400 0.871377 70 1.174100 0.868969 71 0.793300 0.866754 72 0.722700 0.864863 73 0.724500 0.863133 74 0.569600 0.861578 75 0.665500 0.860159 76 1.101700 0.858839 77 0.864500 0.857639 78 0.772900 0.856516 79 0.716800 0.855460 80 1.084500 0.854492 81 0.910600 0.853567 82 0.949000 0.852729 83 1.120600 0.851993 84 0.909700 0.851361 85 1.083200 0.850814 86 0.906900 0.850328 87 0.705300 0.849894 88 0.911400 0.849513 89 0.817300 0.849191 90 0.852000 0.848923 91 0.828100 0.848704 92 0.838500 0.848527 93 0.936700 0.848389 94 1.065900 0.848281 95 0.800000 0.848203 96 0.654700 0.848148 97 1.098800 0.848113 98 0.746300 0.848092 99 0.729500 0.848084 100 0.677400 0.848081

[More Information Needed]

Input Sentence काम की कलाएँ 24 हैं जिनका सम्बन्ध सम्भोग के आसनों से है, 20 द्यूत सम्बन्धी, 16 कामसुख सम्बन्धी और 4 उच्चतर कलाएँ Normalized Output Output: काम की कलाएँ चौबीस हैं जिनका सम्बन्ध सम्भोग के आसनों से है, बीस द्यूत सम्बन्धी, सोलह कामसुख सम्बन्धी और चार उच्चतर कलाएँ हैं

Model can take a sentence in Hindi language and normalize specific entities, including: Dates (any format) Currencies Scientific units to Hindi