Table of Contents Author Guidelines Submit a Manuscript
BioMed Research International
Volume 2016, Article ID 4248026, 9 pages
http://dx.doi.org/10.1155/2016/4248026
Research Article

ChemTok: A New Rule Based Tokenizer for Chemical Named Entity Recognition

1Computer Engineering Department, Eastern Mediterranean University, Famagusta, Northern Cyprus, Mersin 10, Turkey
2Information Technology Department, Eastern Mediterranean University, Famagusta, Northern Cyprus, Mersin 10, Turkey

Received 21 November 2015; Revised 10 December 2015; Accepted 10 December 2015

Academic Editor: Yudong Cai

Copyright © 2016 Abbas Akkasi et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Named Entity Recognition (NER) from text constitutes the first step in many text mining applications. The most important preliminary step for NER systems using machine learning approaches is tokenization where raw text is segmented into tokens. This study proposes an enhanced rule based tokenizer, ChemTok, which utilizes rules extracted mainly from the train data set. The main novelty of ChemTok is the use of the extracted rules in order to merge the tokens split in the previous steps, thus producing longer and more discriminative tokens. ChemTok is compared to the tokenization methods utilized by ChemSpot and tmChem. Support Vector Machines and Conditional Random Fields are employed as the learning algorithms. The experimental results show that the classifiers trained on the output of ChemTok outperforms all classifiers trained on the output of the other two tokenizers in terms of classification performance, and the number of incorrectly segmented entities.