This is an ONNX version of yacht/latte-mc-bert-base-chinese-ws. It was automatically converted and uploaded using this Hugging Face Space.

See the pipeline documentation for token-classification: https://huggingface.co/docs/transformers.js/api/pipelines#module_pipelines.TokenClassificationPipeline

This is a variant of the pre-trained model BERT model. The model was pre-trained on texts in the Chinese language and fine-tuned for word segmentation based on bert-base-chinese. This version of the model processes input texts with character-level with word-level incorporated with a lattice structure.

The scripts for the pre-training are available at tchayintr/latte-ptm-ws.

The LATTE scripts are available at tchayintr/latte-ws.

The model architecture is described in this paper.

The model is trained on multiple Chinese word segmented datasets, including ctb6, sighan2005 (as, cityu, msra, pku), sighan2008 (sxu), and cnc. The datasets can be accessed from here.

The pre-trained model is distributed under the terms of the Creative Commons Attribution-ShareAlike 4.0.

This model was trained with GPU servers provided by Okumura-Funakoshi NLP Group.