Views
No views yet
c-index (rnk) = 0.8881c-index (rnk) observed during training (not the final epoch).| Repo | Role |
|---|---|
sfidan42/turkish_embedding_fine_tuned_q_enc_best | Query encoder (best epoch) |
sfidan42/turkish_embedding_fine_tuned_a_enc_best | Answer / passage encoder (best epoch) |
sfidan42/Quora_QA_Turkish_First_Answers, sfidan42/Quora_QA_Turkish_Second_Answerssfidan42/Turkish-QA-Ranked1from transformers import AutoTokenizer, AutoModel
2import torch
3
4tokenizer = AutoTokenizer.from_pretrained("sfidan42/turkish_embedding_fine_tuned_a_enc_best")
5model = AutoModel.from_pretrained("sfidan42/turkish_embedding_fine_tuned_a_enc_best")
6
7def embed(texts):
8 enc = tokenizer(texts, padding=True, truncation=True, max_length=128, return_tensors="pt")
9 with torch.no_grad():
10 o = model(**enc).last_hidden_state
11 m = enc["attention_mask"].unsqueeze(-1)
12 return (o * m).sum(1) / m.sum(1) # mean pooling
13
14query_vec = embed(["Türkiye'nin başkenti neresidir?"])
15passage_vec = embed(["Türkiye'nin başkenti Ankara'dır."])
16score = torch.nn.functional.cosine_similarity(query_vec, passage_vec)