A deep relation extraction from biomedical texts with attention mechanisms and domain-specific contextual representations

Abstract

The biomedical relation extraction (RE) tasks aim to study the interaction between pre-defined entities from biomedical literature: Bacteria Biotope (BB) and Drug-Drug interactions (DDI) tasks. Some previous investigations have used feature-based models; others have presented deep-learning-based models such as convolutional and recurrent neural networks used with the shortest dependency paths (SDPs). Although SDPs contain valuable and concise information, sections of significant information necessary to define bacterial location relationships are often neglected. In addition, the traditional word embedding used in previous studies may suffer from word ambiguation across linguistic contexts. Here, we present a deep learning model for biomedical RE. The model incorporates feature combinations of SDPs and full sentences with various attention mechanisms. We also used pre-trained contextual representations based on domain-specific vocabularies. In order to assess the model’s robustness, we introduced a mean F score on many models using different random seeds. The experiments were conducted on the BB corpus in BioNLP-ST’16 and the DDI corpus in BioNLP-ST’13. For the BB task, our experimental results revealed that our proposed model performed better (in terms of both maximum and average F scores; 60.77% and 57.63%, respectively) compared with other existing models. For the DDI task, our proposed model also gets state-of-the-art performance with a maximum F score of 80.3% and a mean F score of 77.7%. In conclusion, we demonstrated that our proposed contributions to this task can be used to extract rich lexical, syntactic, and semantic features that effectively boost the model’s performance. Moreover, we analyzed the correct and incorrect predictions of our model to determine the related factors that affected the model’s performance.