This is a StrataBERT diagnostic checkpoint from run_001. Claim status: diagnostic_only. It is not a release-quality checkpoint and must not be used for public quality or efficiency claims.
Architecture
tokens -> embeddings -> [global attention / bidirectional SSM / local attention]* -> mask-aware pooling -> task head
Raw text is not embedded in this card or the frozen eval indices.
Objective Mix
task: 1.0
Teacher Models
No teacher model is used for this checkpoint.
Licenses
Project code license: MIT. Dataset audit summary:
ag_news_v001: restricted_noncommercial_unclear; No standard permissive license is declared.
arxiv_classification_v001: needs_review_full_text_rights; Selected HF repo does not declare a data license.
bc5cdr_v001: needs_review_bc5cdr_tner_mirror; No source-license research entry is present; manifest note: Canonical bigbio/bc5cdr script is disabled by current datasets versions; executable manifest uses TNER BC5CDR converted parquet.
conll2003_v001: restricted_avoid_publication_claims; Highest-risk MVP dataset because the source text is Reuters copyrighted newswire.
eurlex57k_v001: needs_review_lexglue_eurlex; No source-license research entry is present; manifest note: HF datasets metadata inspected with datasets.load_dataset_builder('coastalcph/lex_glue', 'eurlex') on 2026-06-10.
hyperpartisan_news_v001: needs_review_hyperpartisan_mirror; No source-license research entry is present; manifest note: HF parquet metadata inspected on 2026-06-10 via jonathanli/hyperpartisan-longformer-split.
imdb_v001: restricted_noncommercial_unclear; HF license tag is other rather than a permissive license.
openpii_1m_v001: approved_cc_by_4_0_attribution_required; No source-license research entry is present; manifest note: HF datasets metadata inspected with datasets.load_dataset_builder('ai4privacy/pii-masking-openpii-1m', 'default') on 2026-06-10.
patent_classification_v001: needs_review_mirror_license; The selected ccdv sample repo does not declare its own license.
pubmed_200k_rct_v001: needs_review_pubmed_rct_mirror; No source-license research entry is present; manifest note: HF parquet metadata inspected on 2026-06-10.
scicite_v001: needs_review_allenai_scicite; No source-license research entry is present; manifest note: Legacy dataset script is disabled by current datasets versions; executable manifest uses HF converted parquet files.
twenty_newsgroups_v001: needs_review_dataset_card_blank; No source-license research entry is present; manifest note: HF parquet metadata inspected on 2026-06-10 via refs/convert/parquet.
Intended Uses
Local smoke testing of StrataBERT checkpoint loading, evaluation scripts, and metadata plumbing.
Reproducibility checks for run_001 diagnostic artifacts.
Out-of-Scope Uses
Public benchmark claims.
Production classification or token-classification deployment.
Commercial reuse of dataset-derived behavior without legal review of the relevant datasets.
Random or tiny diagnostic training only; no release-quality pretraining.
Mandatory ModernBERT, Ettin, DeBERTa-v3, Longformer, BigBird, and embedding baselines are still pending.
Long-context 2k/4k/8k claims are unsupported by this card.
Dataset license caveats remain unresolved for public claims.
Ethical and Privacy Considerations
This checkpoint is diagnostic and should not be deployed. Dataset provenance and privacy review are incomplete for release use, and token-classification public claims require a publication-safe dataset replacement or legal approval.
