Model Card for GEMMA-2 Post-Processing Model
This model is based on the GEMMA-2 architecture and is fine-tuned specifically to improve YouTube auto-generated captions.
It post-processes the captions by correcting misaligned words that occur due to phonetic similarities but do not fit the context.
The model is designed to enhance the quality of speech-to-text output,
making the captions more accurate and readable by identifying and fixing contextually incorrect words with similar pronunciations.
Model Details
Model Description
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
- Developed by: SangWoo Han
- Model type: Causal Language Model (AutoModelForCausalLM)
- Language(s) (NLP): English
- Finetuned from model [optional]: google/gemma-2-9b-it
Uses
Direct Use
This model can be directly applied to improve the quality of YouTube auto-generated captions by post-processing them.
Without any additional fine-tuning, the model can correct phonetically similar words that are incorrectly transcribed but out of context,
resulting in more accurate and contextually appropriate captions.
Users can input raw captions generated from YouTube videos, and the model will output refined, corrected text.
Out-of-Scope Use
The model is not designed for general-purpose language correction,
grammar checking, or tasks unrelated to phonetic error correction in captions.
It is not suitable for handling highly technical or specialized language that it has not been trained on,
nor for correcting non-phonetically related errors in transcriptions.
Misuse could include attempts to employ the model for purposes such as translation,
sentiment analysis, or plagiarism detection, which are beyond its intended scope.
Bias, Risks, and Limitations
The model may exhibit biases inherent in the dataset used during training,
including but not limited to regional accents, linguistic variations, and dialects.
The correction capability may also perform unevenly across different languages or types of content.
Additionally, the model may struggle with extremely noisy audio or highly specialized terminology not present in the training data.
Recommendations
It is recommended that users be aware of potential biases toward certain accents or dialects.
Testing the model on a diverse range of speech inputs can help evaluate its effectiveness across various demographics.
In cases where specific speech patterns or accents are frequent, further fine-tuning may be required to mitigate these biases.
Users should also remain mindful of the limitations in correcting technical jargon or highly specialized content that the model has not been trained on.
How to Get Started with the Model
1from transformers import (
2 AutoModelForCausalLM,
3 AutoTokenizer,
4 AutoModelForSeq2SeqLM,
5 BitsAndBytesConfig
6)
7import torch
8from peft import PeftModel, PeftConfig
9
10peft_model_id = "navgod/Gemma-2-9b-translator"
11config = PeftConfig.from_pretrained(peft_model_id)
12model = AutoModelForCausalLM.from_pretrained(peft_model_id)
13model = PeftModel.from_pretrained(model, peft_model_id)
14tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
15
16model = model.to("cuda")
Training Details
Training Data
Training Procedure
1from peft import LoraConfig
2
3peft_config = LoraConfig(
4 r=16,
5 lora_alpha=32,
6 target_modules=["gate_proj", "up_proj", "down_proj"],
7 lora_dropout=0.2,
8 bias="none",
9 task_type="CAUSAL_LM",
10)
11
12training_arguments = TrainingArguments(
13 output_dir=new_model,
14 per_device_train_batch_size=4,
15 per_device_eval_batch_size=1,
16 gradient_accumulation_steps=2,
17 optim="adamw_torch",
18 num_train_epochs=1,
19 eval_strategy="steps",
20 eval_steps=0.2,
21 logging_steps=1,
22 warmup_steps=10,
23 logging_strategy="steps",
24 learning_rate=2e-4,
25 fp16=False,
26 bf16=False,
27 group_by_length=True,
28 report_to="wandb"
29)
Training Hyperparameters
- Training regime: [More Information Needed]
Evaluation
Testing Data, Factors & Metrics
Testing Data
[More Information Needed]
Factors
[More Information Needed]
Metrics
[More Information Needed]
Results
[More Information Needed]
Summary
Environmental Impact
Carbon emissions can be estimated using the
Machine Learning Impact calculator presented in
Lacoste et al. (2019).
- Hardware Type: [More Information Needed]
- Hours used: [More Information Needed]
- Cloud Provider: [More Information Needed]
- Compute Region: [More Information Needed]
- Carbon Emitted: [More Information Needed]
Technical Specifications [optional]
Model Architecture and Objective
[More Information Needed]
Compute Infrastructure
[More Information Needed]
Hardware
[More Information Needed]
Software
[More Information Needed]
Citation [optional]
BibTeX:
[More Information Needed]
APA:
[More Information Needed]
Glossary [optional]
[More Information Needed]
More Information [optional]
[More Information Needed]
Model Card Authors [optional]
[More Information Needed]
Model Card Contact
[More Information Needed]
