asr_inverse_text_normalization
Finetuned a facebook/bart-base Pretrained model on the ASR inverse text normalization dataset by treating it as a seq2seq task. Other approaches which may be considered is by considering it as a TokenClassification task and the one mentioned here
https://machinelearning.apple.com/research/inverse-text-normal.
Model description
BART (Bidirectional and Auto-Regressive Transformers) is a pre-trained transformer-based neural network model developed by Facebook AI Research (FAIR) for various natural language processing (NLP) tasks
The BART architecture is based on the Transformer model, which is a type of neural network architecture that processes sequential input data, such as text, by applying self-attention mechanisms to capture the relationships between different words in the input sequence.
BART includes both auto-regressive and bidirectional encoder-decoder transformer architectures, which enable it to perform both generation and prediction tasks
BART was trained on a diverse range of NLP tasks, including machine translation, summarization, and question answering, and has shown strong performance across multiple benchmarks.
Its training process involves corrupting text with different types of noise and training the model to reconstruct the original text, which has been shown to improve the model's ability to generalize to new tasks and outperform other pre-trained language models like GPT and BERT
The model flavour which was chosen is that of "facebook/bart-base" and columns "after" is used as the source while "before" column is used as the targets.
Intended uses & limitations
This model can be used as an out-of-the-box solution to the invesrse text normalization which can convert ASR generated un-normalized text such as
"my c v v for my card is five six seven and it expires on november twenty three" -> "my CVV for my card is 567 and it expires on November 23"
The model needs to be explored for various min and max length setting at the time of generation for your specific usecase
How to use
1
2>>> from transformers import pipeline
3>>> generator = pipeline(model="pavanBuduguppa/asr_inverse_text_normalization")
4
5>>> generator("my c v v for my card is five six seven and it expires on november twenty three")
6
Training data
All credits and rights for the training data belongs to Google. The data was merely obtained and processed for this model and the original data can be found here
https://www.kaggle.com/competitions/text-normalization-challenge-english-language/data
