Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
hubertusz-tiny-wiki-seq128-ONNX – AI Model by onnx-community | AlphaNeural AI
You can deploy this model and start earning money today!
onnx-community
/
hubertusz-tiny-wiki-seq128-ONNX
like
0
transformers.js
onnx
bert
pretraining
generated_from_keras_callback
hubert
hu
wikipedia
SzegedAI/hubertusz-tiny-wiki-seq128
quantized
apache-2.0
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
hubertusz-tiny-wiki-seq128 (ONNX)
This is an ONNX version of
SzegedAI/hubertusz-tiny-wiki-seq128
. It was automatically converted and uploaded using
this Hugging Face Space
.
Usage with Transformers.js
See the pipelines documentation:
https://huggingface.co/docs/transformers.js/api/pipelines
hubert-tiny-wiki-seq128
Fully trained model with the second phase of training is available here:
SzegedAI/hubert-tiny-wiki
This model was trained from scratch on the Wikipedia subset of Hungarian Webcorpus 2.0 with MLM and SOP tasks.
Pre-Training Parameters:
Training steps: 500.000
Sequence length: 128 (the model is capable for 512)
Batch size: 1024
Framework versions
Transformers 4.21.3
TensorFlow 2.10.0
Datasets 2.4.0
Tokenizers 0.12.1
Acknowledgement
Artificial Intelligence - National Laboratory - Hungary
