This repository provides the stratified augmented BERT classifier used in experiments evaluating whether grammatical perspective influences moral acceptability predictions.
The model is fine-tuned on the Justice dataset and predicts binary moral acceptability judgments for short scenario descriptions.
This model corresponds to the model trained on a stratified subset of perspective-augmented training data, balancing exposure across perspectives.
Related models
This model is part of a set of three models used in the experiments:
trained on a stratified subset of perspective-augmented training data
All three models are available on Hugging Face.
Model description
just-bert-aug-strat is a BERT-base-uncased model fine-tuned for binary classification.
The model predicts whether a scenario is morally acceptable or unacceptable based on natural language descriptions.
It serves as the stratified augmented model, trained on a controlled subset of perspective-rewritten data.
Experimental context
The model is part of a study evaluating whether grammatical perspective influences moral acceptability predictions in language models.
The experiment compares:
Baseline model trained on original sentences
Augmented models trained on perspective-rewritten sentences
The analysis measures:
prediction flip rate
probability bias across perspectives
confidence instability
statistical significance of differences
Training data
The model was trained using the Justice dataset, with perspective-rewritten variants.
Instead of using all generated variants, the training data is stratified across perspectives, which tries to balance representation while limiting dataset expansion.
Example input:
You used to walk around the park with your coworker on lunch but you don't anymore because she reads on her lunch break.
Example prediction:
acceptable
Training uses a subset of perspective variants.
Model details
Property
Value
Architecture
BERT
Base model
bert-base-uncased
Task
Binary classification
Labels
moral acceptability
Framework
Hugging Face Transformers
Evaluation
Models were evaluated on four dataset splits.
Model
test
test_hard
test_pers
test_hard_pers
just-bert-base
Acc 0.7456 / F1 0.7714
Acc 0.5838 / F1 0.6433
Acc 0.7457 / F1 0.7731
Acc 0.5832 / F1 0.6426
just-bert-aug-strat
Acc 0.7496 / 0.7707
Acc 0.5848 / F1 0.6381
Acc 0.7515 / F1 0.7759
Acc 0.5877 / F1 0.6458
just-bert-aug-full
Acc 0.7726 / F1 0.7899
Acc 0.6145 / F1 0.6641
Acc 0.7711 / F1 0.7907
Acc 0.6152 / F1 0.6674
Evaluation splits:
test - original scenarios
test_hard - challenging scenarios
test_pers - perspective-rewritten scenarios
test_hard_pers - challenging scenarios with perspective rewrites
Full experimental results and statistical analyses are available in the accompanying repository.
Intended use
This model is intended for research purposes, including:
studying moral judgment prediction in language models
evaluating robustness to linguistic framing
analyzing perspective sensitivity in NLP systems
The model should not be used in real-world moral decision-making systems.
Limitations
The model inherits limitations from both:
the Justice dataset
the BERT architecture
Known limitations include:
potential sensitivity to linguistic framing
potential biases present in the training data
limited reasoning about complex moral scenarios
The purpose of the accompanying research is precisely to investigate these sensitivities.
Ethical considerations
Predictions produced by the model represent statistical patterns learned from the dataset, not objective moral judgments.
Outputs should therefore be interpreted cautiously and used only for research purposes.
Repository
Full experimental code and additional models are available at:
