Research Article
Pharmacovigilance with Transformers: A Framework to Detect Adverse Drug Reactions Using BERT Fine-Tuned with FARM
Table 2
Comparison of results yielded by FARM-BERT with the results yielded by baseline models applied on Twitter and PubMed datasets.
| Models | Features | Twitter | PubMed | | | | | | |
| SVM | Word -grams | 0.701 | 0.650 | 0.675 | 0.711 | 0.682 | 0.695 | ADR terms | 0.503 | 0.514 | 0.508 | 0.539 | 0.558 | 0.548 | Sentence embeddings | 0.604 | 0.644 | 0.624 | 0.671 | 0.611 | 0.641 | Word -grams+ADR terms+sentence embeddings | 0.729 | 0.688 | 0.708 | 0.717 | 0.706 | 0.711 |
| MLP | Word -grams | 0.711 | 0.661 | 0.686 | 0.719 | 0.684 | 0.701 | ADR terms | 0.512 | 0.524 | 0.518 | 0.521 | 0.544 | 0.532 | Sentence embeddings | 0.615 | 0.645 | 0.630 | 0.685 | 0.666 | 0.675 | Word -grams+ADR terms+sentence embeddings | 0.727 | 0.738 | 0.732 | 0.733 | 0.756 | 0.744 |
| LSTM | Word2vec word embeddings | 0.779 | 0.798 | 0.788 | 0.801 | 0.792 | 0.796 | Fasttext word embeddings | 0.786 | 0.812 | 0.799 | 0.825 | 0.798 | 0.811 | Glove word embeddings | 0.771 | 0.782 | 0.776 | 0.810 | 0.786 | 0.798 |
| CNN | Word2vec word embeddings | 0.854 | 0.799 | 0.826 | 0.861 | 0.806 | 0.833 | Fasttext word embeddings | 0.863 | 0.801 | 0.832 | 0.877 | 0.819 | 0.848 | Glove word embeddings | 0..843 | 0.803 | 0.823 | 0.872 | 0.798 | 0.835 |
| BERT | BERT embeddings | 0.831 | 0.850 | 0.870 | 0.920 | 0.930 | 0.910 |
| FARM-BERT | BERT embeddings | 0.840 | 0.861 | 0.896 | 0.982 | 0.964 | 0.976 |
|
|