IPIP-BFFM Sparse Quantile Models
XGBoost quantile regression models for the 50-item
IPIP Big-Five Factor Markers (BFFM) personality assessment, exported as ONNX for cross-platform inference.
What These Models Do
Each model takes up to 50 item responses (Likert 1--5) and predicts Big Five domain scores (Extraversion, Agreeableness, Conscientiousness, Emotional Stability, Intellect). The exported calibration regimes are fit for full 50-item completion and the primary domain-balanced 20-item sparse regime.
Key capability: sparse input. The models produce accurate predictions even when most items are unanswered (NaN). This allows fixed short-form assessments (as few as 20 items) without retraining or switching models.
How It Works
- 15 models in one graph -- 5 domains x 3 quantiles (q05, q50, q95), merged into a single ONNX file
- Structured sparsity augmentation -- training responses are masked into structured partial-response patterns (not uniform random dropout): focused buckets spanning 10-50 retained items (a 10-20-item target-assessment range, a 21-35-item transition range, and a 36-50-item near-complete range, each keeping a minimum number of items per domain), explicit injection of the Mini-IPIP 4-per-domain 20-item pattern, and roughly 15% imbalanced patterns that allow 0-item domains so the model also sees skewed coverage. Within each bucket the retained items are filled by information-rank-weighted sampling (items with higher cross-domain information are more likely to be kept). The deployed operating point is the domain-balanced 4-per-domain 20-item form
- Quantile regression -- pinball loss at tau = 0.05, 0.50, 0.95 provides median predictions with empirical 90% prediction intervals whose coverage is validated for the full_50 and sparse_20_balanced runtime regimes (raw quantile spreads; no post-hoc width adjustment is applied). Empirical 90% prediction-interval coverage is approximately 89.5% for the deployed domain-balanced 20-item form (held-out baseline evaluation), approximately 89.8% for the sparse_20_balanced runtime regime (validation under random balanced 20-item masking), and approximately 92.6% for the full_50 regime (validation)
- Norms-based percentiles -- raw predictions are converted to population percentiles using z-score norms fit on the training split only (n = 422,326); validation and test rows are held out so the norms do not leak into the percentile targets
Variants
| Variant | Description |
|---|
ablation_focused | Research ablation variant |
ablation_none | Research ablation variant |
reference | Primary published model |
The primary model is reference. Other variants are research ablations that isolate the contribution of each sparsity augmentation strategy.
Each variant directory contains:
model.onnx -- merged ONNX model (5 domains x 3 quantiles)config.json -- runtime configuration, feature names, and normsREADME.md -- variant-specific model card with performance tablesprovenance.json -- full audit trail (git hash, data snapshot, training config)
Intended Use
Research and educational score-recovery of the 50-item IPIP-BFFM from partial responses (as few as 20 items), and reproducible study of sparse short-form scoring. Outputs are population percentiles with empirical 90% prediction intervals.
Out of Scope
Not validated for clinical diagnosis, employment or other high-stakes selection, or any individual consequential decision. The models recover a self-report questionnaire score, not an external personality trait.
Limitations
- Norms come from self-selected, anonymous online respondents (OSPP) and may not represent any general population; English-language IPIP items only.
- No demographic-subgroup or measurement-invariance analysis has been performed; accuracy may vary by gender, age, or region.
- No external / out-of-distribution validation: every reported number is on a held-out split of the same OSPP dataset, so these are score-recovery (recovering the full-scale score from a subset of its own items), not external-trait, metrics. The full-50 r ~ 1 is a score-recovery ceiling, not trait validity.
- The deployed 20-item Emotional Stability subscale is composed entirely of reverse-keyed items, so it is more sensitive to acquiescence and careless responding; the full 50-item assessment is unaffected.
- Accuracy degrades with fewer items; 20 items is the recommended minimum for reliable scoring.
- See each variant's
README.md for per-model performance and the repository README.md / docs/research.md for the full limitations discussion.
Source Code
Training pipeline, evaluation scripts, and inference packages (Python + TypeScript):
github.com/sprice/bffm-xgbLicense
CC0 1.0 Universal -- Public Domain Dedication
