Beta

Explore
Marketplace
Neural Labs
Chat
Wallet
Docs

Beta

Explore Marketplace Neural Labs Wallet Docs

Build

Manage

My assets Manage wallet Organization

Community

Discord X (Twitter)Telegram

Docs Privacy Policy Terms and Conditions

Proud member of

Proud member of

Docs Cookie Policy

Privacy Policy Terms and Conditions

© 2026 AlphaNeural AI. All rights reserved.

bert-base-cantonese – AI Model by indiejoseph | AlphaNeural AI

You can deploy this model and start earning money today!

indiejoseph/
bert-base-cantonese

like

0

generated_from_trainer

google-bert/bert-base-chinese

autotrain_compatible

endpoints_compatible

Views

No views yet

bert-base-cantonese

This model is a continue pre-train version of bert-base-chinese on Cantonese Common Crawl dataset with 198m tokens.

Model description

This model has extended 500 more Chinese characters which very common in Cantonese, such as 冧, 噉, 麪, 笪, 冚, 乸 etc.

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 24
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 8
total_train_batch_size: 192
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 1.0

Training results

Framework versions

Transformers 4.35.0.dev0
Pytorch 2.1.1+cu121
Datasets 2.14.6
Tokenizers 0.14.1