Views
No views yet
Gemma2Model with bidirectional attention (is_decoder=False).Gemma is provided under and subject to the Gemma Terms of Use found at https://ai.google.dev/gemma/terms
1from transformers import AutoModel, AutoTokenizer
2
3model = AutoModel.from_pretrained("knowledgator/t5gemma-2-text-encoder-270m")
4tokenizer = AutoTokenizer.from_pretrained("knowledgator/t5gemma-2-text-encoder-270m")
5
6inputs = tokenizer("Your text here", return_tensors="pt", padding=True, truncation=True)
7outputs = model(**inputs)
8
9token_embeddings = outputs.last_hidden_state # (batch, seq_len, 640)
10pooled = outputs.last_hidden_state.mean(1) # mean pooling -> (batch, 640)