You can deploy this model and start earning money today!

cs4248-nlp/
ft-student-all-minilm-l6-v2-taco-20260326-110507

Name: ft-student-all-minilm-l6-v2-taco-20260326-110507
Availability: InStock

sentence-transformers

safetensors

code-search

embeddings

knowledge-distillation

mit

Views

No views yet

cs4248-nlp/ft-student-all-minilm-l6-v2-taco-20260326-110507

Code-search embedding model trained with the CS4248 two-phase KD pipeline.

Model details

Field	Value
Role	`ft-student`
Phase	Phase 1
Method	`ft-student`
Dataset	`BAAI/TACO`
Teacher	`sentence-transformers/all-mpnet-base-v2`
Student base	`sentence-transformers/all-MiniLM-L6-v2`
Phase 1 epochs	`20`
Phase 1 patience	`3`
Phase 2 epochs	`10`
Phase 2 patience	`3`
Batch size	`64`
Eval batch size	`64`
Learning rate	`2e-05`
Seed	`42`
Run timestamp	`20260326-110507`

Usage

python

from transformers import AutoModel, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("cs4248-nlp/ft-student-all-minilm-l6-v2-taco-20260326-110507")
model = AutoModel.from_pretrained("cs4248-nlp/ft-student-all-minilm-l6-v2-taco-20260326-110507")

Mean-pool the last hidden state to get a fixed-size embedding:

python

import torch

def mean_pool(model_output, attention_mask):
    token_embeddings = model_output.last_hidden_state
    mask = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return (token_embeddings * mask).sum(1) / mask.sum(1).clamp(min=1e-9)

inputs = tokenizer("your query here", return_tensors="pt", truncation=True, max_length=160)
with torch.no_grad():
    outputs = model(**inputs)
embedding = mean_pool(outputs, inputs['attention_mask'])