Pharmacovigilance with Transformers: A Framework to Detect Adverse Drug Reactions Using BERT Fine-Tuned with FARM

Table 2

Comparison of results yielded by FARM-BERT with the results yielded by baseline models applied on Twitter and PubMed datasets.

Models	Features	Twitter			PubMed
		SVM	Word -grams	0.701	0.650	0.675	0.711	0.682	0.695
ADR terms	0.503		0.514	0.508	0.539	0.558	0.548
Sentence embeddings	0.604		0.644	0.624	0.671	0.611	0.641
Word -grams+ADR terms+sentence embeddings	0.729		0.688	0.708	0.717	0.706	0.711
MLP	Word -grams	0.711	0.661	0.686	0.719	0.684	0.701
	ADR terms	0.512	0.524	0.518	0.521	0.544	0.532
	Sentence embeddings	0.615	0.645	0.630	0.685	0.666	0.675
	Word -grams+ADR terms+sentence embeddings	0.727	0.738	0.732	0.733	0.756	0.744
LSTM	Word2vec word embeddings	0.779	0.798	0.788	0.801	0.792	0.796
	Fasttext word embeddings	0.786	0.812	0.799	0.825	0.798	0.811
	Glove word embeddings	0.771	0.782	0.776	0.810	0.786	0.798
CNN	Word2vec word embeddings	0.854	0.799	0.826	0.861	0.806	0.833
	Fasttext word embeddings	0.863	0.801	0.832	0.877	0.819	0.848
	Glove word embeddings	0..843	0.803	0.823	0.872	0.798	0.835
BERT	BERT embeddings	0.831	0.850	0.870	0.920	0.930	0.910
FARM-BERT	BERT embeddings	0.840	0.861	0.896	0.982	0.964	0.976