This model repository contains artifacts from an AMIS commodity relevance classifier training run. It includes the Transformer model, any configured TF-IDF or sentence-embedding baselines, prediction files, and the training report.

Dataset: faodl/amis-agri-fertilizers
Dataset subset: ``
Text column: chunk_text
Label column: label
Transformer: FacebookAI/xlm-roberta-base
Generated at: 2026-05-19T13:37:49.429687+00:00

Dataset Summary

Split	Rows	Label 0	Label 1	Unique groups	Mean text length
train	4785	3580	1205	2305	700.6
validation	972	754	218	494	691.1
test	1065	807	258	495	710.2

Threshold Comparison on Test Split

Model	Threshold	Accuracy	Precision	Recall	F1	ROC AUC	Average precision
logistic_tfidf	0.500	0.893	0.790	0.760	0.775	0.932	0.863
logistic_tfidf	0.570	0.895	0.858	0.678	0.758	0.932	0.863
xgboost_tfidf	0.500	0.944	0.950	0.810	0.874	0.955	0.930
xgboost_tfidf	0.224	0.948	0.918	0.864	0.890	0.955	0.930
embedding-logistic_sentence_embeddings	0.500	0.911	0.786	0.868	0.825	0.965	0.916
embedding-logistic_sentence_embeddings	0.703	0.927	0.885	0.802	0.841	0.965	0.916
embedding-svm_sentence_embeddings	0.500	0.931	0.900	0.802	0.848	0.965	0.919
embedding-svm_sentence_embeddings	0.568	0.924	0.912	0.760	0.829	0.965	0.919
embedding-lightgbm_sentence_embeddings	0.500	0.925	0.887	0.791	0.836	0.965	0.906
embedding-lightgbm_sentence_embeddings	0.579	0.924	0.890	0.783	0.833	0.965	0.906
transformer	0.500	0.970	0.938	0.938	0.938	0.990	0.974
transformer	0.869	0.973	0.949	0.938	0.943	0.990	0.974

Confusion Matrices on Test Split

Rows are true labels and columns are predicted labels.

logistic_tfidf at threshold 0.500

True / Predicted	NOT_RELEVANT	RELEVANT
NOT_RELEVANT	755	52
RELEVANT	62	196

logistic_tfidf at threshold 0.570

True / Predicted	NOT_RELEVANT	RELEVANT
NOT_RELEVANT	778	29
RELEVANT	83	175

xgboost_tfidf at threshold 0.500

True / Predicted	NOT_RELEVANT	RELEVANT
NOT_RELEVANT	796	11
RELEVANT	49	209

xgboost_tfidf at threshold 0.224

True / Predicted	NOT_RELEVANT	RELEVANT
NOT_RELEVANT	787	20
RELEVANT	35	223

embedding-logistic_sentence_embeddings at threshold 0.500

True / Predicted	NOT_RELEVANT	RELEVANT
NOT_RELEVANT	746	61
RELEVANT	34	224

embedding-logistic_sentence_embeddings at threshold 0.703

True / Predicted	NOT_RELEVANT	RELEVANT
NOT_RELEVANT	780	27
RELEVANT	51	207

embedding-svm_sentence_embeddings at threshold 0.500

True / Predicted	NOT_RELEVANT	RELEVANT
NOT_RELEVANT	784	23
RELEVANT	51	207

embedding-svm_sentence_embeddings at threshold 0.568

True / Predicted	NOT_RELEVANT	RELEVANT
NOT_RELEVANT	788	19
RELEVANT	62	196

embedding-lightgbm_sentence_embeddings at threshold 0.500

True / Predicted	NOT_RELEVANT	RELEVANT
NOT_RELEVANT	781	26
RELEVANT	54	204

embedding-lightgbm_sentence_embeddings at threshold 0.579

True / Predicted	NOT_RELEVANT	RELEVANT
NOT_RELEVANT	782	25
RELEVANT	56	202

transformer at threshold 0.500

True / Predicted	NOT_RELEVANT	RELEVANT
NOT_RELEVANT	791	16
RELEVANT	16	242

transformer at threshold 0.869

True / Predicted	NOT_RELEVANT	RELEVANT
NOT_RELEVANT	794	13
RELEVANT	16	242

Validation-Tuned Thresholds

logistic_tfidf: threshold 0.570 (validation F1 0.874); test F1 change vs 0.5: -0.017.
xgboost_tfidf: threshold 0.224 (validation F1 0.898); test F1 change vs 0.5: +0.016.
embedding-logistic_sentence_embeddings: threshold 0.703 (validation F1 0.841); test F1 change vs 0.5: +0.016.
embedding-svm_sentence_embeddings: threshold 0.568 (validation F1 0.850); test F1 change vs 0.5: -0.020.
embedding-lightgbm_sentence_embeddings: threshold 0.579 (validation F1 0.847); test F1 change vs 0.5: -0.003.
transformer: threshold 0.869 (validation F1 0.911); test F1 change vs 0.5: +0.005.

Artifacts

logistic_tfidf: /content/agri-fertilizers-classifier/baselines/logistic
xgboost_tfidf: /content/agri-fertilizers-classifier/baselines/xgboost
embedding-logistic_sentence_embeddings: /content/agri-fertilizers-classifier/baselines/embedding-logistic
embedding-svm_sentence_embeddings: /content/agri-fertilizers-classifier/baselines/embedding-svm
embedding-lightgbm_sentence_embeddings: /content/agri-fertilizers-classifier/baselines/embedding-lightgbm
transformer: /content/agri-fertilizers-classifier/transformer

Inference

Install the runtime dependencies:

pip install transformers torch huggingface_hub pandas joblib scikit-learn xgboost sentence-transformers lightgbm

Transformer

python

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

MODEL_ID = "faodl/agri-fertilizers-classifier"

texts = [
    "Rice export prices increased after new procurement rules were announced.",
    "The finance ministry released its monthly fuel tax bulletin.",
]

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, subfolder="transformer")
model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID, subfolder="transformer")
threshold = float(getattr(model.config, "threshold", 0.5))

encoded = tokenizer(
    texts,
    truncation=True,
    padding=True,
    max_length=256,
    return_tensors="pt",
)

with torch.no_grad():
    logits = model(**encoded).logits
    probabilities = torch.softmax(logits, dim=-1)[:, 1].tolist()

for text, probability in zip(texts, probabilities):
    label = model.config.id2label[int(probability >= threshold)]
    print({"text": text, "probability_positive": probability, "label": label})

TF-IDF Baselines

Available baseline names in this run: "logistic", "xgboost".

python

import json
import joblib
from huggingface_hub import hf_hub_download

MODEL_ID = "faodl/agri-fertilizers-classifier"
BASELINE = "logistic"

texts = [
    "Maize production forecasts were revised after delayed rains.",
    "The central bank published new exchange rate statistics.",
]

model_path = hf_hub_download(
    repo_id=MODEL_ID,
    repo_type="model",
    filename=f"baselines/{BASELINE}/{BASELINE}_tfidf.joblib",
)
report_path = hf_hub_download(
    repo_id=MODEL_ID,
    repo_type="model",
    filename="report.json",
)

pipeline = joblib.load(model_path)
with open(report_path, encoding="utf-8") as handle:
    report = json.load(handle)

threshold = next(
    result["validation_best_threshold"]["threshold"]
    for result in report["results"]
    if result["model_type"] == f"{BASELINE}_tfidf"
)

probabilities = pipeline.predict_proba(texts)[:, 1]
for text, probability in zip(texts, probabilities):
    label = "RELEVANT" if probability >= threshold else "NOT_RELEVANT"
    print({"text": text, "probability_positive": float(probability), "label": label})

Sentence-Embedding Baselines

Available embedding baseline names in this run: "embedding-logistic", "embedding-svm", "embedding-lightgbm".

python

import joblib
from huggingface_hub import hf_hub_download
from sentence_transformers import SentenceTransformer

MODEL_ID = "faodl/agri-fertilizers-classifier"
BASELINE = "embedding-logistic"

texts = [
    "Wheat export inspections rose as demand from importers increased.",
    "The sports ministry announced a new stadium renovation plan.",
]

model_path = hf_hub_download(
    repo_id=MODEL_ID,
    repo_type="model",
    filename=f"baselines/{BASELINE}/{BASELINE}.joblib",
)
artifact = joblib.load(model_path)
embedding_model = SentenceTransformer(artifact["embedding_model_name"])
embeddings = embedding_model.encode(
    texts,
    batch_size=artifact.get("embedding_batch_size", 64),
    convert_to_numpy=True,
    normalize_embeddings=artifact.get("normalize_embeddings", True),
)
probabilities = artifact["classifier"].predict_proba(embeddings)[:, 1]
threshold = artifact["validation_best_threshold"]["threshold"]

for text, probability in zip(texts, probabilities):
    label = "RELEVANT" if probability >= threshold else "NOT_RELEVANT"
    print({"text": text, "probability_positive": float(probability), "label": label})

Files

REPORT.md: Markdown report for this training run.
report.json: Machine-readable report containing metrics and thresholds.
transformer/: Fine-tuned Transformer artifacts, when Transformer training is enabled.
baselines/: TF-IDF and sentence-embedding baseline artifacts, when baseline training is enabled.
*/validation_predictions.csv and */test_predictions.csv: Split-level predictions.