RoBERTa Fine-Tuned on Adult Dataset
This repository contains a RoBERTa-based model fine-tuned for tabular classification on the UCI Adult dataset (also known as the "Census Income" dataset). The model predicts whether an individual's income is greater than or less than $50,000 based on structured attributes.
Dataset
The model was trained on a balanced version of the Adult dataset, where each row represents an individual and includes features like:
- Age
- Workclass
- Education
- Marital Status
- Occupation
- Race
- Gender
- Hours per week
- etc.
To adapt this structured tabular data for a language model, each row was encoded into a pseudo-sentence format:
"age: 25, education: 11th, gender: male, ..., income: than 50,000"
The model learns to predict whether the masked token is "greater" or "less".
Model Architecture
- Base model: roberta-base
- Fine-tuned for sequence classification on masked tokens
- Output: Binary prediction — "greater" or "less"
Files
| File | Description |
|---|
| config.json | RoBERTa model configuration |
| model.safetensors | Fine-tuned model weights |
| tokenizer_config.json | Tokenizer configuration |
| special_tokens_map.json | Mapping for special tokens (e.g., ) |
| vocab.json | Vocabulary file |
| merges.txt | BPE merge rules for tokenizer |
| training_args.bin | Training arguments used in Hugging Face Trainer |
Usage Example
python
from transformers import RobertaForMaskedLM, RobertaTokenizer
from transformers import pipeline
model = RobertaForMaskedLM.from_pretrained("ETdanR/RoBERTa_FT_adult")
tokenizer = RobertaTokenizer.from_pretrained("ETdanR/RoBERTa_FT_adult")
fill_mask = pipeline("fill-mask", model=model, tokenizer=tokenizer)
prompt = "age: 35, education: Bachelors, gender: female, occupation: Prof-specialty, income: than 50,000"
result = fill_mask(prompt)
print(result)
Citation
If you use this model, please cite this repository or mention:
Fine-tuning of RoBERTa on a balanced version of the UCI Adult Census dataset for tabular classification.
Authors
