Abstract
This study developed an assistive system for the severe physical disabilities, named “code-maker translator assistive input device” which utilizes a contest fuzzy recognition algorithm and Morse codes encoding to provide the keyboard and mouse functions for users to access a standard personal computer, smartphone, and tablet PC. This assistive input device has seven features that are small size, easy installing, modular design, simple maintenance, functionality, very flexible input interface selection, and scalability of system functions, when this device combined with the computer applications software or APP programs. The users with severe physical disabilities can use this device to operate the various functions of computer, smartphone, and tablet PCs, such as sending e-mail, Internet browsing, playing games, and controlling home appliances. A patient with a brain artery malformation participated in this study. The analysis result showed that the subject could make himself familiar with operating of the long/short tone of Morse code in one month. In the future, we hope this system can help more people in need.
1. Introduction
The majority of users employ either the keyboard or the mouse to access a computer and its functions. Nowadays there exists a myriad of assistive mechanisms which mimic the functions of these input devices for individuals with severe disabilities. These include inductive tongue computer interface [1], eye-controlled device [2], head-controlled device [3, 4], infrared-controlled devices [5, 6], voice-controlled device [7], and physiological signals assistive device (such as Electrooculogram (EOG) switch [8–10], Electromyography (EMG) switch [11–13], and Electroencephalogram (EEG) [14–20]). However, many assistive input devices provide a mode of switch scanning in which one master unit follows the sequence check status of the slave unit for the input of text characters, and this tends to be rather slow.
Morse code is a powerful alternative for access to augmentative and alternative communication (AAC) devices and computer; this is useful for people who can spell and who can only use one switch, where they activate it for a short or a long time or use two switches and where one represents dots and the other dashes. The dots and dashes are used to represent letters of the alphabet. The difficulty most commonly encountered by users is the requirement to conform to the standard long to short tone ratio of 3 : 1 prescribed in standard Morse code. In 2002, the author Dr. Wu presented the fuzzy Morse code recognition algorithm [21, 22] in a single-chip microprocessor that can trace the user’s operating time and automatically adjust the threshold value between long and short elements of Morse code in a real time. At the same time, he made a Morse code text input system (McTin) [23, 24] for the people with severe disabilities (such as quadriplegic) to operate the computer. McTin still has some disadvantages which need to be improved; these disadvantages are the following: first, the input interface only fits mechanical switches, second, when the user is operating the Morse code, McTin cannot provide the sound feedback, third, the fuzzy recognition algorithm can update to improve the recognition rate, and, forth, McTin needs multiline to connect with computer.
In order to improve the above McTin disadvantages, this study presents a code-maker translator assistive input device (CMTAID), which utilizes a contest fuzzy recognition algorithm and Morse codes encoding providing the keyboard and mouse functions for users with severe disabilities to access a standard personal computer, smartphone, and tablet PC.
2. Methods and Implementation
This study presents code-maker translator assistive input device (CMTAID) which includes three parts that are assistive input switches, a contest fuzzy recognition algorithm in microprocessor, and product design.
2.1. Assistive Input Switch
Morse code assistive input interface can use various input modes but just needs to provide the binary output; due to a variety of different diseases, patients should design a suitable assistive input interface; this study is divided into two kinds of input interfaces that are passive and active input interfaces, respectively. The passive switch which is based on mechanical switches that can provide simple binary output (on/off); McTin applies this kind of input interface for spinal cord injury subjects through mouth to use, that is, mouth controlled nipple switch [23]. The active switch is based on sensors, such as utilizing light, sound, and the other analog signal as the signal source. In this paper, we will present an active infrared sensor switch for the patients whose hands functions have not been fully degraded and they can choose it for use. By using an LED which produces light at the same wavelength as what the sensor is looking for, you can look at the intensity of the received light. When an object is close to the sensor, the light from the LED bounces off the object and into the light sensor (Figure 1). This results in a large jump in the intensity, which we already know can be detected using a threshold. This study combines the light sensor with a comparator of the operational amplifier (OPA) to provide a binary output; the OPA reference voltage value can set up the reflected distance of IR light detected by sensor. In order to avoid the multitrigger from the unstable IR reflected signal, we designed a circuit of inverting Schmitt trigger with reference voltage to increase the stability of the special IR switch. At the same time we consider a problem of the user’s operating habit; we designed two modes to operate this switch: one is object present to change the trigger state and the other is no object present to change the trigger state. Additionally, this study is using the 555-oscillator IC to design the feedback sound of Morse code (dot and dash) for the severe disabilities to learn Morse code more easily and an external speaker output block. The appearance of special IR switch is shown in Figure 2. Although this study presents the use of a special IR switch for the input of Morse codes, the form of the input device can be modified to suit the particular requirements of users with different degrees of physical disability [13, 25].
2.2. Contest Fuzzy Recognition Algorithm
Morse code consists of a series of long (dash) and short (dot) tones separated by periods of silence (space). Standard Morse code specifies a ratio of 3 : 1 for duration of the long to short elements (tone or silence). If there is a significant variation in this ratio over the input of a Morse code transmission, most recognition systems struggle to identify the input elements correctly. In practice, many users with disabilities experience great difficulty in maintaining a stable input rate and pattern. CMTAID employs the contest fuzzy recognition algorithm (CFRA, Figure 3) to trace the tendencies of a particular user when the long or short elements are generated and then modifies the threshold value () which distinguishes the two elements accordingly. In other words, is an adjusted recognition level that follows the user’s typing speed by CFRA. Figure 3(a) is the structure of the CFRA; this structure fits to trace the variation of the duration time of assistive switch pressed/released. is the time of long element, which is the duration time of the assistive switch pressed or released. is the fuzzy time value of long element. Similarly, is the time of short element and is the fuzzy time value of short element. The symbol LSR refers to the ratio between long and short elements of Morse code. In Figure 3(b), the is the input signal, the , , , and are the duration times of the assistive switch pressed, and the , , and are the duration times of the assistive switch released. When the released times are all smaller than , the Morse code sequences will combine and convert it to a significant character “” by CMTAID and translator to the PC, smartphone, and tablet PC through the USB HID interface that is generated by arduino.
(a)
(b)
The recognition procedure is described as below.
Step 1 (contest region recognition). Consider the following. If then belongs in long element region (). Else if then belongs in short element region (). Else belongs in unclear region. Here, is the standard deviation of .
Step 2 (fuzzy adjustment). Consider the following. If belongs in long or short element region then one has the following. Use one-node fuzzy algorithm [21, 23] to calculate the and value and then to modify . For example, the prediction error , an input to the fuzzy algorithm, is created by the difference between and , such that In the fuzzy algorithm, a linguistic fuzzy rule is utilized to calculate modified error . Five linguistic parameters are as follows: LN, negative large; SN, negative small; ZE, zero; SP, positive small; LP, positive large. Fuzzy rule 1: if is LN then is LN (typing highest speed). Fuzzy rule 2: if is SN then is SN (typing high speed). Fuzzy rule 3: if is ZE then is ZE (typing normal speed). Fuzzy rule 4: if is SP then is SP (typing slow speed). Fuzzy rule 5: if is LP then is LP (typing lowest speed). The output variable () of fuzzy recognition is by the center of gravity method in (2), where the variable is the membership grade of the th premise in the inference rule and is the center value of the th conclusion in inference rule: Based on the values of and , the predictive output is updated by The same goes for the short element in the same derivation. Else, calculate the middle point (mp) between and and compare it with the to classify the belonging in long/short element region: If > mp then belongs in long element region; Else, belongs in short element region.
Step 3 ( modification). Consider
2.3. Product Design
The product design of CMTAID as shown in Figure 4(a) shows the front panel of the CMTAID, which has two kinds of input socket and a speed adjustment switch. The phone jack input socket provides a connection point for the mechanical type switch that is a passive input switch. The USB micro-B input socket provides a connection point for the sensor type switch that is an active input switch. The speed adjustment switch allows modifying the operating speed of the device (i.e., initial threshold value of fuzzy algorithm) for users with different degrees of disability. A maximum of ten discrete speed levels (i.e., 0~9) is available, where a higher speed level corresponds to a slower typing speed. The user is free to change the speed by pushing the “+” or “−” button. Figure 4(b): the CMTAID incorporates eight indicator lights within its top panel, that is, Keyboard (“KB”), Mouse (“MS”), Training (“TR”), Shift, Ctrl, Alt, Caps Lock (“Caps”), and one-key/two-key input mode (“2”). This arrangement provides the user with a simple means of ascertaining the current state of the device. Figure 4(c) is a back cover. Figure 4(d) clearly illustrates the only one USB micro-B output socket required for connection to a PC in the keyboard-mode, mouse-mode, and practice-mode.
(a)
(b)
(c)
(d)
3. Results and Discussion
3.1. Code-Maker Translator Assistive Input Device
Figure 5 is photographs of CMTAID finished product, the volume size of CMTAID is cm3, Figure 5(a) is CMTAIDs on the view, Figure 5(b) is rear view, and Figure 5(c) is front view; details can refer to Figure 4. Photographs of the CMTAID with special IR switch of left hand are presented in Figure 5(d). The extra assistive switch provides another input selection for trainer or subject such as general mechanical switch and special nipple switch [23, 24].
(a)
(b)
(c)
(d)
3.2. Fuzzy Performance Testing
This section presents the recognition analysis for two algorithms: sliding fuzzy [24] and contest fuzzy by using human-typed data. There are twenty-five human-typed data sets: fifteen data sets typed by wireless experts and the other ten data sets typed by a teenager with cerebral palsy. Tables 1 and 2 show the mean value (mean) and standard deviation (std) of long and short elements of Morse code. Tone refers to the press time of switch and silence refers to the release time of switch. As shown in these tables, the mean value and the standard deviation for the disabled person are much larger than those for the expert. Figures 6 and 7 are the recognition results for the expert and the disabled person by two algorithms. In Figure 6, the average recognition rates of the expert data are very high for two algorithms (99.6% of sliding fuzzy, 99.9% of contest fuzzy). In Figure 7, the average recognition rates of the disabled person are 98.6% for sliding fuzzy and 99.27% for contest fuzzy. In order to observe the improving recognition rate of Morse code, we took the data set 3 of expert to describe the result of recognition that is shown in Figure 8. Figures 8(a) and 8(b) are the same silent sequences of Morse code by expert; the symbol definitions are as follows: the ordinate refers to the Morse code time length in millisecond (ms); the abscissa refers to the number of Morse code sequence; “+” refers to the long element (dash or long-silence); “” refers to the short element (dot or short-silence); “−·” refer to the predictive threshold; the green circle refers to the error recognition points; blue circle refers to the result of recognition by contest fuzzy algorithm. Except for improving the recognition rate of assistive input device, the operation time of algorithm is needed to be considered, so we compare the operation time of two fuzzy algorithms with MATLAB simulation in the same computer. Figure 9 shows that the whole recognition operation time of ten data sets by sliding fuzzy is 2.376 seconds and contest fuzzy is 1.837 seconds. This result has proved that contest fuzzy algorithm is more suitable to recognize the unstable patterns typed by the disabled person.
(a) Sliding fuzzy recognition
(b) Contest fuzzy recognition
3.3. Applications
CMTAID is a bridge device between the user and application devices (computer, smartphone, and tablet PC) that can support the Operating System (OS) of Microsoft (Win8, Win7, WinXP,…, etc.) and Google Android 2.0~4.2.2. In computer, we connect with cable USB-A male to USB micro-B male; in smartphone or tablet PC we need select the fit OTG (On to Go) line to connect with them. Then, we can use any apps on PC, smartphone, and tablet PC through operating the CMTAID. Here, app is an abbreviation for application. An app is a piece of software. It can run on the Internet, on your computer, or on your phone, or other electronic devices.
3.4. Case Study
The proposed device has been tested successfully by three individuals with severe spinal cord injuries (C1–C3), which have resulted in their inability to make any functional movements other than those involving their heads. Figure 10 is the one of them; she can operate the computer, tablet PC, and communication with others through the CMTAID, after two months of training. She types the English letters from “a” to “z” and averages 58.3 seconds. Now, this study is still training the other kinds of severe disabilities; there are two spinal cord injuries (C5–C7), two cerebral palsy cases, and one cerebral artery malformation. They never learned Morse code, but all are interested to learn how to operate CMTAID and they gradually progress. In this paper, we presented the learning condition of the subject with a brain artery malformation. Due to the autonomy movement capability of the cerebral artery malformation patient is very weak, we designed the operation training course once a week for subject and collected the operation data, which were the operation time of the user using the IR assistive switch to input Morse code. Figure 11 is the statistics results of the recognition rate of four training actions, which are practicing of the signal short/long tone for ten times, respectively, and two consecutive long/short tones for five times, respectively. There are eight data sets collected, the data analysis showed that the mean value and standard deviation are seconds for single short tone, seconds for two consecutive short tones, seconds for single long tone, and seconds for two consecutive long tones, and they have the best accuracy rate in the sixth week. The analysis results showed that the subject who never learned Morse code can make himself familiar with operating of the long/short tone of Morse code in two months, and he/she can control the basic movements of the mouse; then as long as the subject continued to practice, he/she will be able to successfully use CMTAID to control computers, smartphones, and tablet PCs.
4. Conclusions
In this study, we design a sound feedback IR input interface, Morse code assistive input device with contest fuzzy recognition algorithm and finished product, named “code-maker translator assistive input device (CMTAID).” In summary, this assistive input device has seven features that are small size, easy installing, modular design, simple maintenance, functionality, very flexible input interface selection, and scalability of system functions, when this device combined with the computer applications software or APP programs. The contest fuzzy recognition algorithm can really improve the accuracy of recognized the unstable Morse code and increase signal recognition speed to achieve the purpose of real-time identification. The case study presented within this study has indicated that a two-month training period with one supervised session per week is sufficient to enable the user to operate the CMTAID independently. Based upon the experiences of the present training team during the training period, this study has modified some of Morse codes in order to simplify the operating procedure. The modified Morse look-up tables are provided in the Appendix of this study such that others may share the current authors’ experiences. In the future, we hope this device can help more people in need.
Appendix
Refer to CMTAID Morse Code Table, available at http://biolab.ele.ksu.edu.tw/biolab/images/CMTAID%20Morse%20code%20Table.pdf.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
The current authors gratefully acknowledge the support provided to this study by the Ministry of Science and Technology, Taiwan, under Contracts MOST 103-2627-E-006-001 and MOST 103-2218-E-218-002.