Part of leap-kt-toolkit, a systematic re-implementation of published Knowledge Tracing models under one protocol. This repository holds every fold of every dataset this model has been run on, with the per-epoch training logs and the exact user split alongside the checkpoints.
Protocol
User-level 80/20 train/test split · 5-fold cross-validation over the training portion · held-out fold as validation · early stopping patience 10 on validation AUC · max 200 epochs.
Every cell in the project runs under identical settings; a cell that cannot is recorded as a documented failure rather than re-run under bespoke settings.
Results
dataset
AUC
ACC
F1
published reference
delta
algebra2005
0.7650 ± 0.0002
0.7937
0.8752
—
—
assist2009
0.7171 ± 0.0004
0.7144
0.8052
—
—
assist2012
0.6841 ± 0.0001
0.7152
0.8214
—
—
assist2015
0.7019 ± 0.0002
0.7471
0.8472
—
—
assist2017
0.6213 ± 0.0002
0.6535
0.3053
—
—
bridge2algebra2006
0.7461 ± 0.0002
0.8426
0.9119
—
—
dbe_kt22
0.7026 ± 0.0004
0.7766
0.8689
—
—
ednet500
0.6204 ± 0.0013
0.6513
0.7770
—
—
junyi
0.7366 ± 0.0001
0.7381
0.8313
—
—
Per-fold values are in each dataset's summary.json. The mean is never reported without the spread — 0.75 ± 0.001 and 0.75 ± 0.09 are different claims.
Why these numbers may differ from other reproductions
Multi-concept questions are not expanded into multiple rows. Toolkits that do expand them place consecutive test positions carrying the same question and the same response, so a model is shown the answer one step before predicting it; on ASSIST2009 that is around 37% of positions and lifts DKT from a published ~0.75 to ~0.89 AUC. Here concepts are an extra axis on the interaction rather than extra rows, so the leak is not expressible and every interaction is scored exactly once.
Cells carrying a published reference value are additionally leak-audited before release: train/test user disjointness, no window crossing the split boundary, exactly-once scoring, and a label-shuffle control that must collapse AUC to chance. Cells with no comparable published number rely on the structural guarantee above rather than on that audit.
Files
<dataset>/summary.json mean ± std and per-fold AUC
<dataset>/split.json the exact user partition, with a checksum
<dataset>/fold<k>/checkpoint/ config.json + weights
<dataset>/fold<k>/epochs.jsonl every epoch's train loss and validation metrics;
each row carries its own model/dataset/fold
<dataset>/fold<k>/run.json protocol and package version for that run
Provenance
Produced by leap-kt at commit(s) 1ad8f67, 6c6da46, 7b83c7d, 7e2fc9f, e3a8dc3.
