This model was created through vocabulary pruning of the original
microsoft/mdeberta-v3-base model while maintaining full quality for Latin and Cyrillic-based languages.
1from transformers import AutoTokenizer, AutoModel
2
3tokenizer = AutoTokenizer.from_pretrained("rustemgareev/mdeberta-v3-base-lite")
4model = AutoModel.from_pretrained("rustemgareev/mdeberta-v3-base-lite")
5
6# Example usage
7text = "This is an example text in English."
8inputs = tokenizer(text, return_tensors="pt")
9outputs = model(**inputs)
10
11# Verify that the model outputs hidden states (embeddings)
12print("Success! Output shape (batch_size, sequence_length, hidden_size):")
13print(outputs.last_hidden_state.shape)
This model is distributed under the
MIT License.
