Abstract

Patient record keeping plays a vital role in diagnoses and cures. Due to a shortage of time, most doctors write prescriptions manually in Pakistan. At times, it becomes difficult for pharmacists to read prescriptions properly. As a result, they may dispense the wrong medicine. This might cause risky and deadly effects on the patient’s health. This paper proposes an online handwritten medical prescription recognition system that lets doctors write prescriptions on a tablet using a stylus and automatically recognizes the medicine. We use signature verification techniques to recognize the doctor’s handwriting to overcome the problem of misinterpretation of the medicine name by the pharmacist. The proposed system stores different features like the pen coordinates, time, and several pen-ups and pen-downs. Besides using features already proposed in the literature for signature verification, we propose some new features that greatly enhance recognition accuracy. We built a dataset of 24 medicine names from two users and compared results using newly proposed features. We have obtained 84%, 78%, 77.47% 77.31%, 74.17%, 60%, 38.5%, 68%, and 61.64% accuracies for 9 users using SVM classifier.

1. Introduction

Computers are used in almost every domain of daily life, like businesses, industries, entertainment, education, personal management, and research activities. Data can be processed and reproduced in a speedy way using computers. Patient record management helps practitioners to diagnose and continue the care timely. Computerized patient record management systems are used to maintain the record of patients and employees working in the hospital [1, 2]. Health care is a broad area that deals with health care information, medical device information, pharmaceutical information, hospital management, and biological system. In the health care system, patient’s precaution and patient care are the major goals [3, 4].

In developing countries like Pakistan, most hospitals, especially in the public sector, are not computerized. Due to a high patient-to-doctor ratio, doctors have a hectic schedule where they have to prescribe or take notes while standing or in walking conditions in emergency cases. Handwritten prescriptions are widely used in the tropical areas of mid-Asia. Especially in Pakistan, doctors mostly prefer to write handwritten prescriptions because they feel comfortable writing the prescription manually, even if they have enough time to access and use a computer.

Handwritten prescriptions have several potential threats associated with them. Unreadable handwriting prescription and the incapability of pharmacists to understand medicine names in medical prescriptions are causing a notable number of patients to expire [5]. Patients may get delayed or wrong medical dosage due to wrong interpretation of handwritten prescription, which may result in further severity of disease and even patient’s death.

Biometric verification is employed in several real-life business applications. It provides numerous benefits like it is difficult to be stolen, hacked, and forged. Biometrics deals with bioscience, which means the automatic identification of a human’s physiological or behavioral characteristics. The biometric method is preferred over passwords and PINs for easiness of use, accuracy, and case sensitivity. There are different biometric types, such as fingerprint, iris, and face recognition [6].

Biometrics can be found in an extensive range of applications which include physical access control systems, logical access control services, consumer identification, prescription identification, and authentication. Handwritten signature verification is still widely used techniques. The biometric system performs two tasks: verification and identification [7].

The signature verification system is intended to verify the individuality of a person by recognizing their handwritten signature. Signature verification contains two types online and offline signature verification [8]. Offline techniques capture image of the signature after the person has completed writing. Online signature verification techniques capture feature which person is signing. It is done by using a tablet and stylus and recording features pen coordinates, time, pressure, etc. Signature verification includes three basic steps [912]. The first step is preprocessing and contains the data in raw form. After preprocessing, the features from obtained data are extracted in numeric form. In the third step, the classifier obtains results and checks whether the signatures are genuine or forged. There are many classifiers, such as neural network (NN), support vector machine (SVM), nearest neighbor, hidden Markov model (HMM), time delay neural networks, and Naive Bayes, which have been employed for signature verification [9]. Several studies in the related work are found on the optimization algorithm that can be used to solve machine learning healthcare-related optimization problems, such as the Bat algorithm [13] and particle swarm optimization [14].

For effective results, features should be dense enough and provide a better understanding of signatures because it is considered personal identification [15, 16]. In [8] offline recognition system, they used a static representation of documents to take a signature on paper which is later scanned which includes cheque, form, and documentation authentication. Offline signature verification techniques are famous for limited information [15]. Offline processing is also a difficult task due to the deficiency of dynamic characteristics [17].

On the other hand, an online recognition system uses dynamic representation, in which information can be stored at runtime. Electronic tablets [18, 19] and smartphones have on-line recognition writing interface [8, 2022]. Most applications are based online, where a person acts and the system automatically derives data for authentication. Online verification is stronger than the other approach because it provides a higher level of security and stores dynamic features such as pressure, coordinates, pens up, and pens down. Online signature verification can be divided into two types: (i)The parametric approach is used to extract the features from signals such as speed, pressure, coordinates, and the number of the pen up and pen down(ii)The functional approach is used for analyzing the online signatures

The representation of offline verification is shown in Figure 1.

The contribution of this work is as follows: (i)We propose an online medical prescription recognition system to overcome problems of unreadable handwriting prescription(ii)We use signature verification techniques to recognize the medicine name prescribed by a doctor(iii)We use tablet and stylus, like used in [23, 24], to enable doctors to prescribe in the same way they normally do

2. Literature Review

2.1. Signature Verification

Signature verification is a broad area in biometric. It has been considered an essential component of the study conducted by the researchers due to the extensively utilized signature verification techniques for experimentation.

In [25], the author proposed a secure retrieval of the classified information system using the neuro-fuzzy technique. The neuro-fuzzy technique was based on a fuzzy neural network. Dynamic signatures were tested by using Svc 2004 database. They extracted different features, including total time duration, average pen pressure, and dynamic pen pressure. The verification system was suitable; the obtained EER of the overall system was 3.952%. The proposed scheme was beneficial for the practical application of classified information.

Online signature verification uses effective and simple techniques. In their research [26], the author employed two types of features, the first was based on the histogram, and the second was based on quantized features using a model-free Manhattan distance classifier. Several tests were conducted on MCYT and SUSIG datasets. The obtained results of the proposed technique were similar to state-of-art algorithms despite their simplicity and efficiency.

The author reviewed the signature verification system, which included feature extraction algorithms [27], tablet PC, digitized pen, HMM’s, modified dynamic time warping technique (DTW), and NN techniques and methodologies.

They discussed the main challenges of signature verification: signature inconsistency and intraperson variability. Different steps performed for signature verification included signature acquisition, preprocessing, feature extraction, threshold selection enrolment, and matching. The steps performed in preprocessing were smoothing, normalization, and segmentation.

In the feature extraction step, performance evaluation parameters were involved, which derived false acceptance rate and false reject rate. The obtained results of the false acceptance rate (FAR) were 0.25%, and the false reject rate (FRR) was 0.5%.

In [28], the study is carried out to propose both online and offline verification approaches. The webcam was used for data collection, preprocessing, and feature extraction, including pen up and pen down, and then, classification results were obtained based on online verification. However, for offline verification, they collected data through the image, performed preprocessing and feature extraction, and in the end, obtained results from different classifiers. When those steps were completed on both online and offline approaches, then these were combined and used SVM for final verification.

Discrete cosine transformed (DCT) and sparse representation techniques [29] are used for signature verification and discussed new properties of DCT, including time-series and extracting different energy features ( coordinate, pressure, azimuth, and altitude). For experimental evaluation, they used SUSIG-visual and SVC2004 databases. In the end, the obtained error rate was 0.33%.

In 2016, the author [15] proposed a two-stage classification approach combining generative and discriminative modeling principles for online handwritten character recognition. The first stage was based on HMM and presented a few unknown patterns in candidate characters. HMM returned the top-ranking character out of the total number of classes. In the second stage, they used SVM frequency count analysis and chose one character from the candidate character class.

The frequency count analysis was used for pairwise classifiers and same-shape characters. The two-stage approach was better than the single-stage.

In [30], they used handwritten signature verification techniques to propose hand-worn devices for genuine and forged signatures. Sixty-six applicants were included in this experiment. Data were collected in two stages; the major one was participants providing genuine signatures, and the second was forged signatures. They collected three types of datasets from accelerometer and gyroscope, which included the acceleration of accelerometer, angle of acceleration, and angle of velocity. This method provided 0.98 AUC and 0.05 EER high degree of accuracy amongst genuine and forged signatures.

In [31], the author introduced handwriting recognition techniques for Chinese characters conVent (conventional neural network) and direct Map (direction decomposition). Direct Map and conVent provided better efficiency and accuracy than the other techniques. These techniques reduced the mismatch problem between the train and tested the data. The ConVent was interesting and straightforward for the recognition system.

In [32], the authors carried out the NN model for signature verification using autoassociative memory. NN model reads the image of signature in the form of matrices. In [33], an automatic signature recognition approach was introduced. Gabor filter techniques were used for preprocessing signature images and then performed linear discriminant analysis. After that, they applied NNs used for matching the trained data. If the data matched to original data, then making a decision data is authentic or inauthentic. The obtained results from the proposed system were very high, 99.5%, rejection rate of 73%, and computational time of method were 0.87 s.

2.2. Prescription Recognition

The authors [5] used MediPic and Android applications to resolve the recognition problem. They used optical character recognition technology to scan the medicine name and convert it into a digital script. They used Tesseract for character recognition and an inner key algorithm to match the characters. The inner key algorithm matched the character and returned the best results in the database. The MediPic provided more efficient results and helped to decrease the misunderstanding of medicine names.

In [34], the author presented a design of printed traditional Chinese medicine (TCM) prescription and filing system based on the Microsoft office document imaging (MODI) and optical character recognition (OCR) engine.

An optical recognition engine [35] extracted complete information about the disease and stored the data in the database and made a file for further use. TCM provides information for future use. In 2017, [36] implemented a hospital information management system for medical records. Using fingerprints of patients for authentication because health is the main aim or goal of the hospital.

In addition to the health of patients in hospitals, their privacy and security are also considered. They used different technologies for biometric authentication, which included data management, system design, unified modelling language, biometrics, and computer programming. The efficiency of the hospital was increased by using a medical record system with biometric authentication.

In [37], handwritten medical prescription systems were introduced; a prescription is written by doctors based on word spotting and uses an information retrieval approach. They proposed two approaches: the first was Tandem-HMM, used for word spotting, and the second was domain knowledge, used to reduce the text information, which increased the performance.

Different steps were involved: the first one was developing a diagnostic system. The second was information extraction, the third was wrong medication detection, and the fourth was a statistical analysis of medical prescriptions prescribed by doctors. The obtained accuracy was increased by 15.42%, which was a good achievement.

Different techniques are used for medical prescription recognition problems, but our focus is to use signature verification techniques for medical prescriptions. A summary table concerning different coordinates is presented in Table 1.

3. Methodology

Patients’ death due to the wrong drug intake caused by misinterpreted prescriptions by pharmacists happens frequently. Most doctors write prescriptions manually because prescribing a stylus on a tablet is not very common or practiced regularly in Pakistan for prescription writing. Therefore, sometimes, it causes harmful effects on patients. Our work is based on handwritten medical prescriptions, but we employed signature verification techniques.

3.1. Signature Verification vs. Handwritten Medical Prescription

(i)Through the signature verification technique, we can verify a person’s identity by recognizing their handwritten signature. Signature verification uses a pattern matching technique to verify the signatures, but most of the time, it is difficult to read the signature because there is no pen up and pen down movement Signatures can be forged easily. Offline signature verification is only based on shapes(ii)Through the handwritten medical prescription, the doctor’s handwriting is recognizable. The forgeries or forged signatures are automatically involved in the signature verification. Forged signatures are referenced or duplicate signatures of a person. Online handwritten medical prescription recognition is based on character recognition. Pen up and pen down are associated with handwritten medical prescriptions. There are two types of handwritten verification techniques called online and offline verification. Both techniques are linked with signature verification and handwritten medical prescription recognition

3.2. Offline and Online Handwritten Verification

(i)The offline technique deals with the only shape of the writing and is used for limited information due to its static representation of documents to take a signature or prescription on paper. This prescription is later scanned. This approach is difficult because the segmentation is performed on scanned data. Through the offline approach, we cannot calculate the speed and pressure of a pen(ii)Compared to the offline technique, the online approach is more secure and accurate due to the variation of feature value each time and provides relevant results. The online approach deals with dynamic features such as speed, pen up, pen down, and pressure. A tablet or smartphone is used for an online approach that stores the pen tip movements as well as pen up and pens down, switching

In this article, we have proposed signature verification techniques for medical prescription recognition systems to overcome the misinterpretation of medicine names. There are five steps involved in medical prescription recognition explained below: (i)In the phase of data acquisition, we collected data from two users and 9 users, and we used the stylus for writing a medicine’s name on the tablet, which stores the movements of the stylus(ii)Feature extraction is the second step. We extracted different features from the original data obtained from the medical prescription system. Several extracted features involved: pen up with respect to - coordinates, pen down with respect to x-y coordinates, the total time between pen down to pen up, midpoints of total time with respect to - coordinates, quarter midpoints of total time with respect to - coordinates, and three-quarter of total time with respect to coordinates(iii)Preprocessing is the third step in which the unwanted data have been removed, i.e., useless columns or extra columns and adding missing values(iv)We have carried out different classifiers for the experimental evaluation, which involves Naive Bayes, SVM, gradient boosted, and decision tree. Figure 2 presents a flow of the complete research activity

3.3. Data Acquisition

Data acquisition is a process that measures the physical condition and converts it into digital numeric values. The doctor prescribes the tablet using a stylus. Our proposed system calculates the movement of a stylus according to and coordinates.

Medicine name has already been stored in the database of the proposed system. The user has to put the medicine name by using the styles into the GUI interface of the proposed system. After that, system generates values according to pen movement in the form of and coordinates. The system generates an event that includes the number of -coordinates, -coordinates, total time, and pens up, and pens down. The real-time data experiment of the proposed system is presented in Figures 3 and 4.

Original data obtained from the proposed medical prescription application is given in Table 2.

3.4. Feature Extraction

We extracted the features from the acquired data. When the medicine name is written on the GUI of the proposed system, then the original number of strokes or movements of the stylus are recorded and accumulated in the database. We extracted two types of features called spatial and local spatial features.

3.4.1. Spatial Features

Spatial features are also called static features which are extracted from the shape of the signature or original. The extracted spatial feature is shown in Table 3.

3.4.2. Local Spatial Features

Local spatial features are extracted for signature verification which includes: coordinates, total time, pen up, pen down, and angle.

Apart from the spatial features, we have extracted different features, including the following: (i)Pen down with respect to the coordinate (dnx): the system calculates values according to the coordinate as soon as the stylus touches the screen of the tablet(ii)Pen down with respect to the coordinate (dny): the system calculates values according to the coordinate as soon as the stylus touches the screen of the tablet(iii)Pen up with respect to coordinate (upx): the system calculates values according to the coordinate when the user takes a pause or lifts the pen for the first time while writing(iv)Pen up with respect to the coordinate (upy): the system calculates values according to the coordinate when the user takes a pause or lifts the pen for the first time while writing(v)Total time of pen up (time): the total time of pen up is the total time of lifting the pen from the tablet by the user according to coordinates(i)Midpoints of pen up with respect to the coordinate(midpoints ): pen up midpoints are middle points of the total time of pen up from the tablet with respect to the coordinate or the total time between pen up and pen down(ii)Midpoints of pen up with respect to the coordinate (midpoints ): midpoints are middle points of the total time of pen lifting from the tablet with respect to the coordinate. Or the total time between pen up and pen down(iii)Quartiles of pen up with respect to coordinate (quarter ): quartiles of pen up are the first total time divided into four equal parts with respect to the coordinate(iv)Quartiles of pen up with respect to coordinate (quarter ): quartiles of pen up are the total time divided into four equal parts with respect to the coordinate(v)Third quartile midpoints of pen up with respect to coordinate (3quarter ): third quartile pen up midpoints are the three equal parts out of one-quarter of total time with respect to the coordinate(vi)Third quartile midpoints of pen up with respect to coordinate (3quarter ): third quartile pen up midpoints are the three equal parts out of one-quarter of total time with respect to the coordinate

The accuracy obtained by using the feature includes upx1, upy1, dnx1, dny1, time1, midpoints1x, midpoints1y, quarter1x, quarter1y, 3quarter1x, and 3quarter1y with SVM was not high enough. To boost the accuracy, we extracted some additional features, including total time, midpoints of total time, quarter time of midpoints, and 3quarter time of midpoints. We used a six-time pen up and pen down, as shown in Table 4.

3.5. Preprocessing

There exist some irrelevant data obtained from feature extraction which cannot be used in its original form. It contained useless content such as the first pen down column, the first pen down column, and attributes containing missing values and extra columns. This type of data has not been used in our experiments. We have cleaned up (error-free) the irrelevant data to get the best results.

3.5.1. Remove Columns

Two columns are considered useless because the first pen down concerning the -coordinate and the first pen down concerning the -coordinate contain “0” values. These columns have been removed. Furthermore, we took only six times to pen up and pen down, so extra columns are removed as unnecessary. This data does not affect the analysis. The description of data that is removed is given in Table 5.

3.6. Missing Values

Several attributes of the columns contain null values during writing medicine names. The null values affected the result analysis because the classifier does not accept these types of data and shows an error to add values containing null values. Therefore, to improve the results, we replaced “0” with the null attribute in Table 6.

3.7. Dataset

We collected data for 100 medicine from 2 users while data for 24 medicine from nine users. Each medicine contains at least 10 samples for the training dataset. The total number of training data is 3000 medicine samples. Each sample is labeled with its respective medicine name. The user name is also included to train the classifier for each user separately. The sample of training data is shown in Table 7.

The classifier employed in this work is given the same features as input, the same classes of medicine, and the same number of medicines. We used the KNIME analytics platform for analysis. It is an open-source platform to verify the online signature (medicine name) of samples.

3.7.1. -Measures

We have measured the performance of eleven features, including pen up , pen up , pen down , pen down , total time, midpoint , midpoint , quarter , quarter , 3quarter , and 3quarter , with different classifiers. Features are ranked using all the eleven feature ranking metrics. The performance of feature ranking metrics is compared with different classifiers. Macroaverage and microaverage are used to calculate classification performance. Precision and recall are the harmonic means of -measure. Description of macro--measure is given below. where is the precision and is the recall values of class . Furthermore, it sums up the global precision and recalls for all classes of the dataset in the micro- measure as given below. where is the precision and is the recall which is dependent on the overall performance of the classification decisions within the entire dataset. where tp is the true positive value, fp is the false positive, and fn is the false negative value, respectively.

4. Results and Discussion

The accuracy obtained using this chunk of the dataset is 90% using SVM, as compared to Naive Bayes 70%, gradient boosted 50%, and 40% from the decision tree. The experimental results reveal that SVM outperformed other states of the art algorithms for this experimentation. Empirical results of the pen up and pen down features are obtained from user 1 and user 2; data are shown in Figure 5.

The comparison evaluation on the features: upx1, upy1, dnx1, dny1, time1, midpoints1x, midpoints1y, quarter1x, quarter1y, 3quarter1x, and 3quarter1y have been carried out. In this experimental setup, we used 24 medicine data and achieved 90% with SVM, 75% Naive Bayes, 70% gradient boosted, and 85% decision tree. The accuracy comparison of 20 medicine with 100 medicines for two users is shown in Figure 6. The accuracy can be enhanced if we consider more pen up and pen down features.

We observed that the sample size of the medicine is directly proportional to the accuracy. The more the sample size of medicines increases, the better the accuracy was obtained. The experimental results of classifiers obtained from user 1 and user 9 are shown in Table 8.

We can see from Table 8 that SVM performed better for every user from user 1 to user 9 with higher accuracy of 84% and 78%, 77.47%, 77.31%, 74.1%, and so on. Some users have shown low accuracy because of the slow writing of the medicine name because the time difference of the pen up and pen down was high, which is the reason for less accuracy.

We have computed the score, which is the measurement of the test’s accuracy defining the weighted harmonic mean of the precision and recall of the test data of all the 9 users. Table 9 presents the performance of each classifier for -measure obtained data from user 1 to user 9.

The first column of the table shows all the users, and the rest columns confers the -measure of all 9 users’ data obtained according to from SVM, Naive Bayes, decision tree, and gradient boosted. The graphical representation of empirical results is shown in Figure 7.

4.1. Comparison with Other Features

Through our medical prescription features, we obtained better accuracy; however, additional features can be extracted from original data as described in [41], certain features have been extracted from the handwritten medical prescription application, and the accuracy has been calculated. The proposed features are discussed: (i)Pen length (PL): the path length is the total length covered by the user’s pen tip throughout the signature creation(ii)Pen diagonal length (DL): diagonal length is the maximum (, ) and minimum (, ) points in the - coordinate(iii)Time length (TL): the total time of writing the complete signature (the time period between the first pen down and last pen up)(iv)Mean speed (MS): mean speed is the average speed and velocity of the user writing the signature(v)Covariance - (CXY): covariance means to measure the scattered points on the signature path(vi)Vector length ratio (VLR): calculate all vector points of the signatures from the beginning to each - coordinate

We employed the same classifiers with additional features described above obtained from user 1 to user 9 for signature verification. The results of individual classifiers are shown in Table 10.

The comparison of both types of features obtained from user 1 and user 9 is shown in Figures 8 and 9, which infers that the SVM is not proven as a better choice for signature verification features [41].

Furthermore, the overall performance of each classifier is appeared as lower than the proposed prescription recognition system. Further, the observation is recorded that the proposed prescription recognition system was employing signature verification techniques performed better than the others. This confirmed that the local spatial features (extracted features) depend on spatial features (original features).

We obtained significant results from both signature verification features and handwritten prescription recognition systems. The line curves from Figures 8 and 9 show that the overall performance of each classifier remains lower as compared to the proposed handwritten medical prescription recognition system.

5. Conclusion

Patients face difficulties when reading doctor-prescribed medication names. This research focused on the performance and analysis of various classifiers for the newly established handwritten recognition system of medical prescription. We used signature verification techniques to recognize the misinterpreted medical prescription issue better. For the performance evaluation of the proposed system, we introduced new features to increase the performance of the prescription system. We achieved 84%, 59%, 57%, and 56% with SVM, Naive Bayes, decision tree, and gradient boosted. Furthermore, the experiment is extended to determine the -measure with 84%, 62%, 59%, and 51% from SVM, Naive Bayes, decision tree, and gradient boosted, respectively. The experimental results revealed that the proposed system based on the handwritten medical prescription data outperformed in terms of better recognition accuracy. This research would help to recognize the prescription in a better way in the area of health care. In the future, we are intended to extract additional features based on statistics. The proposed handwritten medical prescription recognition system opens a new direction for medical prescription recognition.

Data Availability

No data were used to support this study.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

This work is funded by the King Saud University from Riyadh, Saudi Arabia, Project Number (RSP-2021/164), King Saud University, Riyadh, Saudi Arabia.