Views
No views yet
1BASE_MODEL=monologg/biobert_v1.1_pubmed
2python run_squad.py \
3 --version_2_with_negative \
4 --model_type albert \
5 --model_name_or_path $BASE_MODEL \
6 --output_dir $OUTPUT_MODEL \
7 --do_eval \
8 --do_lower_case \
9 --train_file $SQUAD_DIR/train-v2.0.json \
10 --predict_file $SQUAD_DIR/dev-v2.0.json \
11 --per_gpu_train_batch_size 18 \
12 --per_gpu_eval_batch_size 64 \
13 --learning_rate 3e-5 \
14 --num_train_epochs 3.0 \
15 --max_seq_length 384 \
16 --doc_stride 128 \
17 --save_steps 2000 \
18 --threads 24 \
19 --warmup_steps 550 \
20 --gradient_accumulation_steps 1 \
21 --fp16 \
22 --logging_steps 50 \
23 --do_train| val | |
|---|---|
| exact | 75.97068980038743 |
| f1 | 79.37043950121722 |
| total | 11873.0 |
| HasAns_exact | 74.13967611336032 |
| HasAns_f1 | 80.94892513460755 |
| HasAns_total | 5928.0 |
| NoAns_exact | 77.79646761984861 |
| NoAns_f1 | 77.79646761984861 |
| NoAns_total | 5945.0 |
| best_exact | 75.97068980038743 |
| best_exact_thresh | 0.0 |
| best_f1 | 79.37043950121729 |
| best_f1_thresh | 0.0 |
SQuAD V2 allows the model to score if a paragraph contains an answer:1start_scores, end_scores = model(input_ids)
2span_scores = start_scores.softmax(dim=1).log()[:,:,None] + end_scores.softmax(dim=1).log()[:,None,:]
3ignore_score = span_scores[:,0,0] #no answer scores
4