You can deploy this model and start earning money today!

csarron/
mobilebert-uncased-squad-v1

Name: mobilebert-uncased-squad-v1
Availability: InStock

Views

No views yet

MobileBERT fine-tuned on SQuAD v1

MobileBERT is a thin version of BERT_LARGE, while equipped with bottleneck structures and a carefully designed balance between self-attentions and feed-forward networks.

This model was fine-tuned from the HuggingFace checkpoint google/mobilebert-uncased on SQuAD1.1.

Details

Dataset	Split	# samples
SQuAD1.1	train	90.6K
SQuAD1.1	eval	11.1k

Fine-tuning

Python: 3.7.5
Machine specs:

CPU: Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz

Memory: 32 GiB

GPUs: 2 GeForce GTX 1070, each with 8GiB memory

GPU driver: 418.87.01, CUDA: 10.1
script:

shell
1# after install https://github.com/huggingface/transformers 2 3cd examples/question-answering 4mkdir -p data 5 6wget -O data/train-v1.1.json https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v1.1.json 7 8wget -O data/dev-v1.1.json https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v1.1.json 9 10export SQUAD_DIR=`pwd`/data 11 12python run_squad.py \ 13 --model_type mobilebert \ 14 --model_name_or_path google/mobilebert-uncased \ 15 --do_train \ 16 --do_eval \ 17 --do_lower_case \ 18 --train_file $SQUAD_DIR/train-v1.1.json \ 19 --predict_file $SQUAD_DIR/dev-v1.1.json \ 20 --per_gpu_train_batch_size 16 \ 21 --per_gpu_eval_batch_size 16 \ 22 --learning_rate 4e-5 \ 23 --num_train_epochs 5.0 \ 24 --max_seq_length 320 \ 25 --doc_stride 128 \ 26 --warmup_steps 1400 \ 27 --output_dir $SQUAD_DIR/mobilebert-uncased-warmup-squad_v1 2>&1 | tee train-mobilebert-warmup-squad_v1.log

It took about 3 hours to finish.

Results

Model size: 95M

Metric	# Value	# Original (Table 5)
EM	82.6	82.9
F1	90.0	90.0

Note that the above results didn't involve any hyperparameter search.

Example Usage

python

from transformers import pipeline

qa_pipeline = pipeline(
    "question-answering",
    model="csarron/mobilebert-uncased-squad-v1",
    tokenizer="csarron/mobilebert-uncased-squad-v1"
)

predictions = qa_pipeline({
    'context': "The game was played on February 7, 2016 at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California.",
    'question': "What day was the game played on?"
})

print(predictions)
# output:
# {'score': 0.7754058241844177, 'start': 23, 'end': 39, 'answer': 'February 7, 2016'}

Created by Qingqing Cao | GitHub | Twitter

Made with ❤️ in New York.