Views
No views yet
| Attribute | Value |
|---|---|
| Architecture | Encoder-Decoder Transformer |
| Parameters | ~30,325,164 |
| d_model | 256 |
| Encoder layers | 14 (RoPE + Flash Attention) |
| Decoder layers | 6 (causal, cross-attention) |
| Vocabulary size | 3,500 (SentencePiece BPE) |
| Language embedding | Yes (Vietnamese=0, English=1) |
| Normalization | RMSNorm |
| Activation | SiLU (Swish) |
| Positional encoding | Rotary (RoPE) |
Cong123779/AI2Text-Bilingual-ASR-Dataset:| Hyperparameter | Value |
|---|---|
| Batch size | 32 (effective 128 w/ grad-accum × 4) |
| Learning rate | 3e-4 |
| Epochs | 50 |
| Warmup | 3% of training steps |
| Mixed precision | bfloat16 (AMP) |
| Gradient clipping | 0.5 |
| CTC weight | 0.2 |
| Scheduled sampling | 1.0 → 0.5 (linear) |
1import torch
2from pathlib import Path
3import sys
4
5# Clone the repo and add to path
6sys.path.insert(0, "AI2Text")
7
8from models.asr_base import ASRModel
9from preprocessing.sentencepiece_tokenizer import SentencePieceTokenizer
10from preprocessing.audio_processing import AudioProcessor
11
12# Load tokenizer
13tokenizer = SentencePieceTokenizer("models/tokenizer_vi_en_3500.model")
14
15# Load model
16checkpoint = torch.load("best_model.pt", map_location="cpu")
17config = checkpoint.get("config", {})
18
19model = ASRModel(
20 input_dim=80,
21 vocab_size=3500,
22 d_model=256,
23 num_encoder_layers=14,
24 num_decoder_layers=6,
25 num_heads=8,
26 d_ff=2048,
27 num_languages=2,
28)
29model.load_state_dict(checkpoint["model_state_dict"])
30model.eval()
31
32# Transcribe
33audio_processor = AudioProcessor(sample_rate=16000, n_mels=80)
34features = audio_processor.process("audio.wav") # (time, 80)
35features = features.unsqueeze(0) # (1, time, 80)
36lengths = torch.tensor([features.size(1)])
37
38with torch.no_grad():
39 tokens = model.generate(
40 features, lengths=lengths,
41 language_ids=torch.tensor([0]), # 0=vi, 1=en
42 max_len=128,
43 sos_token_id=tokenizer.sos_token_id,
44 eos_token_id=tokenizer.eos_token_id,
45 pad_token_id=tokenizer.pad_token_id,
46 )
47 text = tokenizer.decode(tokens[0].tolist())
48 print(text)