Views
No views yet
BAAI/bge-small-en-v1.5.
Encode with the BGE query prefix, exactly as the app does:1from sentence_transformers import SentenceTransformer
2import joblib, numpy as np
3
4encoder = SentenceTransformer("BAAI/bge-small-en-v1.5")
5prefix = "Represent this sentence for searching relevant passages: "
6vec = encoder.encode([prefix + message], normalize_embeddings=True)[0]
7
8clf = joblib.load("clf_category.joblib")
9print(clf.predict(vec.reshape(1, -1))[0])random_state=42.| target | accuracy | macro-F1 | majority baseline | beats baseline |
|---|---|---|---|---|
category | 0.9882 | 0.9882 | 0.1290 | yes |
urgency | 0.4471 | 0.3061 | 0.4584 | no |
clf_category.joblib is reliable and is what the app leads with. Its accuracy is
high partly because the training data is synthetic and spec-conditioned — the
generator was told which category to write about — so expect materially lower
numbers on real support tickets.clf_urgency.joblib does not work and is published for completeness. It scores
below the majority-class baseline, because urgency was sampled independently of
the text the generator wrote, so the message carries almost no urgency signal. Do
not use it to make decisions. See
notebook 04.positive labels
contradict their own text, so the app reads sentiment with an LLM instead.