Electronic Guidance Cane for Users Having Partial Vision Loss Disability

Khan, Asad; Ashraf, Muhammad Awais; Javeed, Muhammad Awais; Sarfraz, Muhammad Shahzad; Ullah, Asad; Khan, Muhammad Mehran Arshad

doi:https://doi.org/10.1155/2021/1628996

Wireless Communications and Mobile Computing

On this page

Abstract Introduction Materials and Methods Conclusions Data Availability Conflicts of Interest Authors’ Contributions Acknowledgments References Copyright Related Articles

Special Issue

Metaheuristic Algorithms for Big Data Analytics within the Internet of Things

View this Special Issue

Research Article | Open Access

Volume 2021 | Article ID 1628996 | https://doi.org/10.1155/2021/1628996

Electronic Guidance Cane for Users Having Partial Vision Loss Disability

Asad Khan,¹Muhammad Awais Ashraf,²Muhammad Awais Javeed,²Muhammad Shahzad Sarfraz,³Asad Ullah,⁴and Muhammad Mehran Arshad Khan^5,6

Academic Editor: Mamoun Alazab

Received05 Jul 2021

Revised03 Aug 2021

Accepted27 Aug 2021

Published05 Oct 2021

Abstract

Vision is, no doubt, one of the most important and precious gifts to humans; however, there exists a fraction of visually impaired ones who cannot see properly. These visually impaired disabled people face many challenges in their lives—like performing routine activities, e.g., shopping and walking. Additionally, they also need to travel to known and unknown places for different necessities, and hence, they require an attendant. Most of the time, affording an attendant is not easier and inexpensive, especially when almost 2.5% of the population of Pakistan is visually impaired. There exist some ways of helping these physically impaired people, for example, devices with a navigation system with speech output; however, these are either less accurate, costly, or heavier. Additionally, none of them have shown perfect results in both indoor and outdoor activities. Additionally, the problems become even more severe when the subject/the people are partially deaf as well. In this paper, we present a proof of concept of an embedded prototype which not only navigates but also detects the hurdles and gives alerts—using speech alarm output and/or vibration for the partially deaf—along the way. The designed embedded system includes a cane, a microcontroller, Global System for Mobile Communication (GSM), Global Positioning System (GPS) module, Arduino, a speech output module speaker, Light-Dependent Resistor (LDR), and ultrasonic sensors for hurdle detection with voice and vibrational feedback. Using our developed system, physically impaired people can reach their destination safely and independently.

1. Introduction

Vision is, undoubtedly, one of the most important senses for humans as humans get 83% of information from the environment via this sense. Unfortunately, as per this study [1], carried out by the World Health Organization (WHO) in 2011, 285 million people are visually impaired, 39 million are completely blind, and 246 million had weak vision/sight. In Pakistan alone, ~2 million people are suffering from blindness/sightlessness and visual impairment [2]. Vision loss disability or sightlessness is a phenomenon of lacking the vision perceptions, due to physiological or neurological mental factors, having a visual acuity of just 1-2/10 with both eyes open (less than or equal to 30 degrees [3]). This fraction of people faces many challenges, like performing simple daily routine activities, and hence always remains dependent on others [4–6]. This is not just the loss of the individual but also for the country, as they are unable to play their role in the growth of their country’s economy [7]. Using a “white cane” simple stick (introduced by James Biggas of Bristol in 1921 [8]), as an indication, was the only way out for a visually impaired person to move around. The US congress proclaimed October 15 the “White Cane Safety Day,” to show love, respect, and sympathy to the visually impaired people.

In the 1960s, the research on assistive technologies—associated with data transmission [9, 10], navigation, and orientation aids, to confer accurate help to the visually handicapped person—was initiated [11–14]. For example, the first approach to observe the hurdle (coming in the way of an impaired person) was the thought of a vector field bar graph. The technique, somehow, solved the indoor navigational activities; however, it was less helpful for the outdoor activities [14]. Hence, the developed system was considered overcomplicated and less accurate and needed a mobile terminal to send and receive the information [15]. Later, the use of a microcontroller in a renowned system, namely, “NAVEBELT and Guidance,” consisted of a series of devices (placed at the belt), to observe the hurdles [16]. Afterward, Yuan provided a device for varying sensing and surrounding discovery, intending to assist a visually impaired person. The device measures active triangulation and observes the environmental feature exploitation using the Kalman filter tracking [17, 18]. Another proposed solution takes the help of a camera and a transmitter to transmit the real-time video to the server—to be monitored and assisted by another person. This system also proved not accurate especially in outdoor applications and could not help the person much [19]. Bolgiano and Meeks [20] proposed a visual aid system based on the GPS, GSM, and ultrasonic sensors to detect the hurdles and to obtain the information related to user location [21].

Nowadays, different kinds of those canes are introduced, e.g., smart cane [22] and optical maser cane [23]; however, these tools have many constraints: unnecessary longer length of the cane and limitations in recognizing obstacles, making it difficult for the person to access public places. Recently, some other solutions are also proposed like GPS devices for landmark identification (near-infrared (IR) lightweight or radio frequencies), supersonic obstacle detectors (sonar, UltraCane, Miniguide, Palm-sonar, Ultra-Body-Guard, and iSONIC cane), and optical devices (the laser long cane) [24–29]. However, these devices have shown to be less useful in crowded environments especially in outdoor environments, mainly because of the multiple reflections. Alternatively, several techniques have been developed to revisit the quality of life of visually impaired people by introducing smart devices with built-in signal processing and sensing technology. These referred to electronic travel aid (ETA) devices which facilitate the blind to maneuver freely in an atmosphere dynamically changing in real time. As per the literature, ETAs are broadly classified into 2 major types: sonar input systems (laser signal, infrared signals, or supersonic signals) [30, 31] and camera input systems (consisting principally of a mini-CCD camera and fuzzy system) [32–35]. Bat K sonar, sensible cane, smart vision, and guide cane to observe obstacles in front of the person by transmitting and receiving the mirrored wave normally used supersonic sensors or optical maser sensors [36–39]. In response to detected obstacles to warn the person, it produces either associate audio or vibration. Systems like voice [40], Sound View [32], SVETA [41], and CASBLIP [42] use a single camera or stereo video cameras mounted on a wearable device to capture pictures. These captured pictures are resized and processed to regenerate the corresponding speech, audio, musical sounds, or vibrations. In such systems, the frequency of warning sound signals is related to the orientation of pixels. Some advanced systems use Global Positioning System (GPS) integration with the main system. It is also noteworthy that GPS receivers are beneficial for understanding the present location of the subject and close landmarks. Some solutions are already accessible within the market such as UltraCane [43], iSONIC [44], and Teletact [45, 46]. These products facilitate the visually impaired ones by grouping the data collected through sensors and then transmitting the recommendations through vibration or sound messages to the user. ETAs provide a warning by using sensing modality or/and tactile signals once an obstacle is estimated within the range and recommend the user to avoid it.

Blind aid and security systems have already been prepared with many other solutions. However, none of them is capable of completely deterring the needs of the visually impaired person. In summary, the main contributions of the proposed work are as follows: (i)We provide an accurate and usable proof-of-concept electronic stick to make the user’s life easier(ii)Our approach used supersonic sensing technology to detect the hurdle and generates an audio alarm/vibration to get the user’s attention(iii)Our prototype uses the LDR (Light-Dependent Resistor or photoresistor) to detect the darkness around the user and generates a darkness alert and lights the stick in parallel(iv)Additionally, our system also detects water and smoke as well. Our solution uses GSM and GPS for the detection of the location of a user, and in case of an emergency, a user can press an emergency button to automatically generate and distribute a SMS to the desired person

The rest of this paper is structured as follows: next, we will discuss in Section 2 a concept of the indoor guide system based on partial vision loss which is presented with a detailed description of the design and further development of electronic apparatus used in the proposed electronic guidance cane, based on outdoor detection of objects, and direction indication-simulation system based only on the detection of objects. In addition, our experimental results for a lab-scale prototype are provided, and at last Section 3 concludes the paper.

2. Materials and Methods

Presently, visually impaired people normally use a white cane as a tool for directing them when they move or walk for their daily routine work. Here, in Figure 1, a smart electric cane prototype design was developed as a tool that can serve as an electric guidance cane for visually impaired persons being a more efficient and helpful technique than the conventional one. The prototype explains different parts of the proposed system, and an ultrasonic sensor is used on the top side and bottom right and left for obstacle avoidance. The vibrator is fixed on the top of a stick which vibrates when an obstacle is encountered which helps in alarming the visually impaired person and allows that person to change their path. On the bottom of the cane, an IR sensor is used for pit and staircase detection to guide someone in that direction where there are fewer obstacles. To avoid sleeping in a water area, a water sensor is attached to the initial side of the guided system, and for fire protection, a fire sensor is placed on the initial bottom; a light sensor is useful at night which alerts the people in the surrounding area that a visually impaired person is walking and allows space so that the person can walk easily.

For further discussion and simplicity, Figure 2 illustrates the overall working mechanism and the features included in the proposed designed electric guidance cane concept. As shown within the figure, the Arduino Uno microcontroller is the heart of the proposed system as all the individual units are controlled and interfaced with the heart of the machine. The supersonic detector and the GPS unit act because the input to the microcontroller provides the obstacle data and user data severally when walking and using this embedded system. The LCD, GSM unit, and buzzer unit essentially are the outputs results from the microcontroller. The buzzer unit gives alerts via voice when the user encounters an obstacle in front of a specified distance. In case of emergency and any failure, the GSM unit provides message service to the expensive ones of the user. IR and LDR represent the obstacle avoidance sensor as a heat-sensitive sensor and light-dependent resistance, respectively. LDR and IR first sense the intensity value of an object in front of the user and send it to a microcontroller. The rest of the sensors including the water sensor work in the same procedure. After receiving the data, the microcontroller converts it into different discrete values from initial step 0–1023 and checks whether the received value is above the threshold level (a limit value that is set independently by the visually impaired person from the range of discrete values: 0–1023) or not; it will then be considered as there is no object in front of an electric guidance cane, and the buzzer alarming will remain off; if the received value is below the threshold level, the microcontroller will consider it as an object or hurdle in front of the guidance cane. During outdoor activities, if the value of IR and LDR is high and detects an object in front of a visually impaired person, then the vibrator will be on, or if the ultrasonic sensor value is high and identifies an object, then beep will be generated, voice coming from the speaker will tell the user to move right or left, and the light on the stick will also shine. For the practical decomposition of the merchandise, the structure of the proposed architecture is convenient and efficient enough to cover all aspects of the observant because it demonstrates the presence of every module ranging from its arrival until the service is served to the desired user and it describes that the modules are interlinked and intercepted with one another and work along to attain the required goal.

2.1. Electronic Components

Multiple electronic materials and apparatus are used for building electronic guidance cane circuits. Our proposed circuit designs contain these materials and apparatus that are described in Table 1.

2.1.1. Arduino Mega2560

The Arduino Mega2560 microcontroller board (licensed under a Creative Commons Attribution-Share-Alike 2.5) is based on the ATmega328 series controllers, and the schematic of the Arduino Mega consists of three blocks: the voltage regulator, the ATmega2560 with supporting circuitry, and the header. The default input is Arduino (Atmega), so they do not need to be explicitly declared as inputs with pin Mode () when we are using them as inputs. This means that it takes very little current and energy consumption to move the input pin from one state to another. Arduino Mega based on ATmega328 15, the main features of Arduino Mega (ATmega2560 running at 16 MHz with an external resonator (0.5% tolerance), also USB connection off the board it which supports auto-reset,5 V regulator which weighs covers <=2 grams the current performance DC input 5 V up to 12 V, Onboard power and status LEDs.

2.1.2. Ultrasonic Sensors

The sensor consists of three parts: transmitter, receiver, and control circuit. For robotics and hardware embedded systems, ultrasonic sensors are relatively simple and easy to interface. We have used the sonar sensor HC-SR04 in our project to detect hurdles on the left and right and in front of the stick and on the footstep. Sensors calculate the distance between the hurdle and receiver of the sensor. To get a desired obstacle localization effect, it is not enough to just improve the performance of the sonar sensor; filtering the measured data is additionally necessary. In this paper, the sequential sonar returned data is processed through the dynamic filtering method using the orientation and therefore the trajectory information of the cane control box. It can sense hurdle in a range of 2-80 cm. We can set the range of sensor according to our desire. The working principle of sonar needs a high-level signal for at least of 10 μs using an IO trigger. It sends pulses of 40 kHz, and the pulse back signal is scanned. Sonar has main parts called transducers who send pulses of eight 40 kHz and sense returned pulses or echo: an ultrasonic sensor emits ultrasonic wave through a transmitter which spread in the air when it hit any hurdle and then returns to the ultrasonic sensor receiver and a counter stops counting time when it receives reflected wave. For the basic purpose of choosing, we have used three ultrasonic sensors on the left and right and in front of the stick to detect any hurdle accurately. Ultrasonic sensors are very useful and cost-effective. There is no effect of sunlight on the performance of these sensors. We can use it for both indoor and outdoor systems. It has a range of 2-400 cm. We have a fixed range at 45 cm which means if there is a hurdle in 45 cm, then there would be an alert for a blind person in a form of audio voice. In Table 2, we discuss the specifications of ultrasonic distance sensor-HC-SR04 embedded with an electric cane control box using an Arduino microcontroller.

In the control box, we use four HC-SR04 sensors, which have the main functionality to detect obstacles in front of the cane within the mentioned distance in the table and guide a disabled person about obstacles and hurdles indoor and outdoor and further demonstrate a person to move left or right via signals sent to a microcontroller and attached speaker pronounce voice command, as shown in Figure 3.

Table 2 demonstrates the values of specific working voltage, measurement range, I/O pins needed, operating current, and dimensions.

2.1.3. GPS Module

In 1960, U.S. Air Force invented a satellite-based navigation system. The tracking sensitivity of GPS is -165 dBm. The working principle of GPS is that it can connect with 24 satellites which continuously encircle the earth once in 12-hour duration and provide very useful information regarding time, velocity, and position on earth. We can identify the position of any object by measuring the distance of that object from the satellite. The distance from satellite is where is the speed of light () and is the time period of satellite.

GPS needs three satellites for a 2D position, but for accuracy it needs four satellites for providing the location of any object on earth. When someone turned on the GPS module, it firstly downloads orbital information of all satellites and saves information in its memory. Then, the receiver calculates the distance between a satellite and itself by multiplying the velocity of the transmitted signal which is the speed of light by the time they receive the transmitted signal from a satellite. Along with the measure of travel time by the receiver, it should also know the exact position of the satellite for accuracy. GPS transmits a signal on two different carrier frequencies: one is L1 which is 1575.42 MHz and the other is L2 that is 1227.60 MHz which is more precise and used for military purposes. In our paper on the purpose of GPS, we are using the (GY-NEO6MV2) GPS module. These apparatus provide the ability to find the path and exact location of a visually impaired person. It may also use GPS for path planning. This GPS receiver with better performance having 20 million correlations helps in finding satellite location as soon as possible when it is turned on. It has six engines and a member of the NEO-6 series of GPS receivers. The power supply range is from a 3 V to 5 V LED signal indicator and 3 configuration pins with a 1-time pulse which supports DGFS (SBAS, WAAS, EGNOS, and MSAS) at a maximum altitude of 50,000 m with 500 m/s velocities. The performance of GPS in both cold and warm climate starts to be delivered after 27 sec.

2.1.4. GSM Module

GSM was developed in 1970 by Bell Laboratories Inc. This device has great importance in communication to transmit and receive data by using mobiles across the world. It had made communication easy since its development. It works at different frequency bands in which the band is used depending on the application. GSM handles multiple access at a time that is why today’s communication system has become better than past years’ systems. Its works between frequencies 850 MHz-00 MHz and 1800 MHz-1900 MHz, and data rates from 64 kbps to 120 Mbps can be conceded by this system. Time division multiple access overlapping of one signal to another signal was common in an older communication system. To avoid this major error in a communication system, engineers introduced a new method that is time division multiple access. In this method, some specific time slots were assigned to every user that helped a lot to solve many problems regarding overlapping, but one thing is important that frequency remained the same for all users. The use of the GSM module is that because of its better spectrum competence, international roaming, support for new facilities, and real-time clock with alarm management, phone calls are secure by using encryption and short message service (SMS). Most secure telecommunications currently accessible as security strategies are standardized for it. This module can be used in several applications in GPRS mode remote data logging, transaction terminals, weather stations, security applications, and supply chain management.

2.1.5. Voltage Regulator

In this prototype, a voltage regulator, LM 7805, is used. Voltage regulators are used for regulating voltages to our desired circuit voltage requirement, as shown in Figure 4. It will protect our circuit whenever there is excess voltage or current level beyond our circuit limit. It provides a constant voltage at its output terminal. It is a member of 78xx linear voltage regulator ICs, and xx indicates that it will provide a constant voltage level. The voltage regulator LM 7805 ratings described are its input voltage range from 7 V to 35 V, , and output voltage and .

2.1.6. RF Module

The RF module wireless system design has two overriding constraints: which operates over a certain limit and specific distance and transfers a specific quantity of convergence results within a data rate. The RF modules are very small in dimension and have a wide operating voltage range; it operationally works between 3 V and 12 V. The working techniques of RF modules are RF transmitter and receiver modules. When transmitting logic 0, the transmitter draws no power while the carrier frequency thus consumes significantly less power in battery operation. From the transmitter, the data is usually sent serially which is received by the tuned receiver. Duly interfaced to two microcontrollers for data transfer, the RF transmitter and receiver are working at frequency 433 MHz, receiver current supply 3.5 mA, sensitivity 105Dbm, operating voltage 5 V. low power consumption with a transmitter frequency range of 433.92 MHz, and transmitter supply voltage of 3 V~6 V with output power of 4~12 Dbm.

2.1.7. Vibratory Motor

The vibratory motor is using a vibrator in an electric stick which will activate when there is water in the path of a blind person. It is connected to the bottom of the stick through two terminals when there is water current which will flow through terminals, and the vibrator is turned on and alerts a blind person.

2.1.8. Switches

An electric stick had one switch to control power consumption of the battery to get maximum time for the functioning of a blind person stick. The switch is used for the main power to turn on or off. The working usage is that it installed this switch just to save battery consumption.

2.1.9. Capacitor

Capacitors are used to filter out frequencies. It acts as a high-pass filter allowing high frequencies to pass and blocking the DC signal as well as a low-pass filter allowing low frequencies to pass and blocking the AC signal. We connect the capacitor in parallel to our circuit instead of series to block the AC signal and get only the DC signal.

2.1.10. Diode

A semiconductor diode is a device used for the unidirectional flow of current. It blocks current in the reverse direction. It has two terminals: anode (positive) and cathode (negative). A diode is a PN junction. It works when it is forward biased and behaves like a short circuit. When it is reverse biased, it behaves like an ideal open circuit.

2.1.11. SD Card Module Shield

In a smart electric stick to store the voice for four ultrasonic sensors, an SD card shield is installed with Arduino Mega. It is very useful to store different voice files for right, left, and front hurdle at the footstep, fire, water, etc. The SD module can be used by likely different microcontrollers for saving data rates like Arduino, AVR, PIC, and ARM. The module shield follows technical parameters which are operating voltage of 5 V, SPI interface, and dimensions of .

2.2. Indoor and Outdoor Prototype of the Electric Guidance Cane

As we see in Figure 5, it shows the embedded hardware circuit design of an electric guidance cane based on hurdle detection with multiple hardware items using Arduino Uno, with the sonar sensor HC-SR04. In this methodology, the beep alarm and audio signal will turn to a high state only with the detection of an object in front of the user; otherwise, there is no action and the audio will remain off at that time.

In this scenario, a photocell LDR sensor, LEDs, LCD screen, resistors (220 ohms), micro-SD socket (SD card), ultrasonic sensor (HC SR04), speaker (SPKR 1), rotatory potentiometer (large) (R3), round pushbutton (S1), vibration motor (ROB 08-449), Adafruit Ultimate GPs, GSM (sim 800 L), water level sensor, flame sensor, and single Arduino Uno Mega (2560-ReV3) were used. One leg of the LDR sensor is attached to the microcontroller analog PIN A3, and another leg to a 5 V pin, and the same is done with a resistor which is attached to the GND port of the microcontroller. Besides, the threshold value for the sonar sensor (HC SR04) was adjusted to 5 from the discrete values (0–1023) for understanding whether it is a hurdle or not. On the other side, right and left pins of the sonar sensor pin VCC are connected to a speaker and potentiometer, and trig and echo pin is related to pin D34 and D36 Arduino Uno as well as GNDs. The front sensor (HC SR040) VCC was connected the same as left and right pins, and trig and echo were connected to pins D13 and D14 Arduino Uno and GNDs. Here, Arduino Mega ADK analog pins A1 and A0 are connected to the LDR R1 side and water sensor sport, and other to 3v3 and GND. The other ground of battery’s negative also put against to the battery’s positive. Furthermore, the D52 (SCK) and D50 (MISO) of Arduino were connected to the GSM port sim_rxd, sim, and txd, as we can see in Figure 5. The Arduino pins D28, D30, and D32 are related to the LDR positive side and D26 with the flame sensor port Ky-026. The other considerable connections are Txl D18 to GPS pin rx, D16 to SD card pin (U RSV), D2-D5 with LCD PIN DB7, and DB6-DB5 and DB4. The rest of the Arduino D8 and D12 with push-button leg 1 is connected to the LCD cathode side.

At the last part of the circuit design, GND negative part of all the obstacle avoidance sensors was connected to the GND port and all VCC (input voltage) to the Arduino 5 V pin. Initially, the sonar HC-Sr04 has a distance set from 2 cm to 1 m (by default) at the start; if there was a hurdle detected, the vibrator vibrates and will generate a beep through the speaker and the voice (turn left and right and forward) will be heard by the blind person.

2.2.1. Indoor Prototype of the Electric Guidance Cane

Concerning indoor cane usage patterns, a user follows it to fight the steer in its center where they are walking in order that any open proportional font window, door, or split unit mechanical device and house objects might not hurt them. They will swipe the electric guidance cane on the ground rather than the left-right direction technique if the floor surface is okay enough. It is quite difficult to measure the huge area square for the navigation, which does not need personal or infrastructure help. On stairs, they use the pencil vogue technique. They follow the railing of the steps or wall with manus. When returning down, they use a guidance cane to feel and observe the depth and breadth of stairs by a sound alarm and swiping cane. For escalator use in a mall in the door, the bumped start line allows them to comprehend the initial line. To get out of it on the finish, they either get information by sloping down reeling at the top or place a guidance cane on a further next step in order that they shall recognize once it touches the ground level area. One subject aforesaid that he uses to face on heel once approaching towards finish; therefore, he could safely find himself on escalators. For carry usage and other altitude purposes, they use a guidance cane to grasp once the carry arrived if it does not ring. Then, they use a cane to gauge the height of a lift’s floor from the building floor, step in, and face within the center of the carry.

(1) Results and Discussion. In the beginning, Figure 6 shows the flow of the electric guidance cane for every step of the blind walking. It conjointly shows the sensors and actuator’s work and also the control method done by Arduino Uno.

The flowchart of the obstacle detector using the supersonic sensing element is shown in Figure 6, which has 2 components: the initial half part deals with the obstacle detection and the other second half deals with distance measuring and alerting the users counting on the distance of the near obstacle to avoid collision. Counting on the gap of the obstacle from the person, four zones of the area unit formed: far zone, near zone, shut zone, and zone. If the next detected object is at ≥2 meters or more, then it comes underneath the way (safe) zone. If the next item object is found at 1 meter or more, then it comes underneath some close to the zone; if the item object is found at 100 cm or more, then it comes underneath the shut zone; and if the item is detected at 100 cm, then it comes underneath the zone. A voice instruction in conjunction with vibration and a buzzer alert voice is processed to a user at each zone to alarm him/her and let individuals around that visually handicapped person to assist. For usage, a disabled person can control it very easily which is our main point to give more priority for partially disabled person and old age humans. When a disabled person wants to use it, he/she just needs to press “Button,” then all features and functions will be activated like a microcontroller and sensors with GPS location. He uses his cane to move on road—indoor or outdoor, the cane will guide the user about their path and obstacles via voice (turn right, left, etc.) and send the location to person guardian; also, it can be used to track a disabled person in case of emergency.

Figure 7 shows the final demonstration of the proposed smart aid electronic stick that detects the obstacles and object’s movement indoor using Arduino Uno. Figure 7(a) represents a lab-scale prototype of an electronic stick. Figure 7(b) shows the movement of the forward base because the sensed object intensity value of the sonar sensor was less than the threshold value and there was no obstacle detected by any of the obstacle avoidance sensors, so as a result, no beeping alarm turns on. The formal design of the proposed model and methodology can be seen in Figures 7(c) and 7(d), with the motive that only those obstacles will be detected in an indoor area, and the rest of the module including the voice alarm, vibrator, and beep will keep maintaining their state and start working in this condition. As an example, as discussed in Figure 7(c), the sonar sensor detects the initial hurdle and provides a voice assistant to handle the obstacles. Therefore, if the sonar sensor observes the sense intensity value or a value above the threshold, it suggests solutions to handle the obstacles. Moreover, when the object moves to the first obstacle, it indicates to turn right; that condition becomes true. Similarly, when an object moves to the second obstacle, it indicates to turn left (Figure 7(d)). The demonstrated results proposed the efficiency of the idea and give immediate validation for the RF module, vibrator, speaker, sonar sensor (HC SR04), and proposed model. These types of applications can be implemented and installed in the indoor area, parking area, hotels, schools’ shopping malls, college’s boundary, inside sectors, hospitals, and malls and homes to detect objects and give a straight path to the blind person.

(a)

(b)

(c)

(d)

Figure 7

Result diagrams of the indoor area of the smart aid electronic stick to move “forward,” “turn right,” and “turn left” at object detection. (a) In the real-time simulation, the proposed system finds the path by the situation of the surface and the place of the object on it and helps decide to which direction to move. (b) In the representation, they check the obstacles if there is any and then change their path which is dictated by speaker voice. (c) When an object comes in front of an obstacle avoidance sensor, the first set dictates a command to turn left while the other remaining dictations are neglected. (d) When motion is in front of the second obstacle sensor, only the part of the second set of the visually impaired person dictates a command to turn right, and all the other positions are neglected.

2.2.2. Outdoor Prototype of a Smart Stick of Visually Impaired Human

As per our vision, the central idea of this work is to create such an innovation for our current visually impaired persons and an embedded system so that the results of location accuracy and direction can be improved. As presented in Figure 8, in outside use, all of the themes use left and right sound of the cane technique on a flat surface path. The reason of this can be that all of them use a cane with a vacant tip, not with a roller ball that will be swiped on the surface. In an outdoor setting, when a visually impaired person walks with angular deviation, three speech types will be on: turn “right, turn “left,” or “forward,” dependent on sensor detection varying frequency on the pedestrian’s adherence to the way.

(1) Results and Discussion. The practicality of the supersonic sensor device remained the same as shown in Figure 6; this could be an additional display in Figure 8. During this, a combination of GPS and GS-modem technologies would possibly offer additional aid for the visually impaired persons.

Initially, whenever there is an emergency, the blind individuals have to be compelled to press the trigger button that activates the GPS and GSM. GPS identifies the placement of the visually handicapped person in real time which is sent to GSM within the sort of coordinates. An alert message is sent together with the precise location of the visually handicapped person to the receiver. For additional aid, supersonic detectors with voice recognition also are accustomed to finding an obstacle and active torch sensors. Therefore, this stick will not be misused by others except for the approved users. Figure 9 shows the design results of the outdoor prototype of an electric guidance cane for users having a vision loss disability using the Arduino Mega microcontroller. In this way, Figure 9(a) represents a visually impaired person from outside of the city where the location can be traced, and a user can also follow the path which he decided at the initial start level when he starts a journey. On the other hand, in Figures 9(b) and 9(c), the person turns right and left on the surface sonar sensor placed on the electric cane because there was a motion that was detected by the first IR obstacle avoidance sensor. At the same time, the object moved forward from the second avoidance sensor, then the third obstacle avoidance sensor detected its motion, and a voice alarm coming from the speaker helps the person to choose which side is better. Meanwhile, when a fire was detected by the flame sensor placed on the cane, the vibrator automatically operates to alert the person who is visually disabled; also, the water sensor senses the road water or depth of the road if there is any, as seen in Figures 9(b) and 9(c). In Table 3, the functionality of old systems differs from the new purposed embedded system of the electric guidance cane for users having vision loss problem. In the old system, a basically single microcontroller was used with single distance and infrared sensor to guide the user in only prescribed one way direction. On other hand, some used only an obstacle sensor or flame sensor and different engineers used GPS for location. At the end, we study and concluded that the overall solution of this problem is to minimize the cost and increase the usability of the device with lots of other features. In this study, the authors have designed a low-cost system. Therefore, the system has various productive functions. First, it detects obstacles in multiple directions and informs the user through voice and beep alert, detecting also flame and ground depth. Second, the system also detects water existence and movement. Third, the system is also beneficial as it sends and receives the real-time location. At the end, if the visually impaired lost their direction, then he/she alarms other persons nearby by pressing a button on the cane, hinting them that he/she lost his/her path, and a message is also sent to the guardian via longitude and latitude as we discuss in Figures 10(a) and 10(b) with a real-time location of that person.

(a)

(b)

(c)

Figure 9

Result status diagrams of further enhanced work with an outdoor prototype of the electric guidance cane of visually impaired human. (a) During an outdoor real-time simulation, working on location, GPS and GSM modules were employed to check the longitude and latitude. (b) Display hurdle in front of the first sonar obstacle avoidance sonar sensor: turn right. (c) Motion in front of the second sonar obstacle avoidance sonar sensor: turn left. Water, road depth, and flame are detected in front of the stick, so a vibrator vibrates and the speaker generates a relevant direction command.

2.3. User Path Finding

We designed the proposed embedded system and tested its functionality and results with a partially visually impaired person holding it, as featured in Figure 10. The alarm SMSs transmitted from the system to mobile phones of the disabled person’s family and guardian are shown in Figures 10(a) and 10(b) for generating alert and request to need help, respectively. The SMSs with the embedded system device’s current location and tracking path and status requested by the guardians are featured in Figures 10(c) and 10(d), respectively. Finally, a web link is generated, and the current location of the electric guidance cane in Google Maps and the available tracking device are shown in Figures 11(a) and 11(b), respectively.

(a)

(b)

In our country, Pakistani visually impaired persons (aged between 17 and 60 years) who used the prototype, as shown in Figures 8 and 9, respectively, reported clear benefits from the ability to alarm and get connected with their family and guardians. Moreover, the data analyzed to check multiple users how to use this guidance cane and reported that the guidance system improved their life style by giving them additional confidence and life easiness.

We used the microcontroller Arduino-based electric guidance cane with GPS and GSM wireless connectivity, in which the visually impaired person could automatically control the directions based on some sensors which the provide the data in the form of direction, buzzer, and alert via voice, to avoid any hurdles which come in a path or way. In some cases, we required wireless connectivity to make the prototype more scalable to find the current location of a new or existing visually impaired person and online access to the information managed by the microcontroller boards to develop a user-friendly interface, and a message is sent to the required guardian. In this regard, the current location longitude and latitude can be used for access from the Internet, and Google Maps would provide easy-to-use access to the location of visually impaired humans. Meanwhile, to store previous location record history and secure the acquired information of the prototype for further study and analysis, different kinds of searching and dictation algorithms could be used in our proposed designs; in fact, a microcontroller is easy to integrate with several sensors. Moreover, the reported systems are presented as lab-scale prototypes; similarly, the proposed embedded systems can be managed for real-scale facilities by applying other technologies. For the question of wireless connectivity, the GSM and GPS are used as shown in Figure 5. The system can be replaced with different sensor types, or a combination of several types, for better results. For example, one can choose a long-range infrared sensor with a maximum range of 1.5 m or a combination of laser diodes and photoresistors.

The feedback of the test on visually impaired persons about the proposed embedded system after participating in the practical experiment in a university of the system was positive. Based on the results and outputs, this kind of system would aid their navigation. The study also collected usability suggestions for further development of the system and real-life use.

3. Conclusions

In this paper, we conclude, finalized, and developed an electric guidance cane for the safety, protection, and convenience of visually impaired persons. It guides the person using voice and by vibration alert. Therefore, it is useful for people that are visually impaired as well as for those who are hearing impaired. It is used for navigation and hurdle detection with the help of three ultrasonic sensors installed in three different directions, i.e., front, right, and left, and for the footstep of the stick. Using the GSM and GPS, the user can send its location to the concerned person; we have developed a smart algorithm with compact hardware design in a small box which comprises of the GPS module, GSM module, Arduino Mega, and water sensor at the bottom and flame sensor. The battery and other components are inside the stick body. This is an efficient design through which visually impaired people can go anywhere without any problem and can easily avoid hitting any hurdle coming in their path. We have installed a GSM module for locating blind people anywhere on earth. There is a specific mobile number that is saved in coding; whenever a specific message is sent by this number to a blind stick GSM number, the location of the blind person would be sent to that specific number. We have used Arduino to make our hardware compact and to avoid making the stick heavy. It also increases the efficiency of the system. This stick is easy to carry by a blind person without any problem. In the future, we will convert a rechargeable battery to solar and road fraction recharge. In this way, the battery life will increase and interlink with the AI camera which would be able to detect obstacles.

Data Availability

No data were used to support the findings of the study.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this article.

Authors’ Contributions

AK, MAA, and MAJ devised the methodology and acquired funding. MSS, MMAK, and AU carried out the formal analysis and data curation. AK and MSS wrote the original draft, reviewed the writing, and edited the manuscript. AK and AU proofread the manuscript before its final submission. AK, MAA, and MAJ contributed equally to this work.

Acknowledgments

This work was supported by the Guangzhou Government Project under grant no. 62104301.

References

R. P. A. Bourne, S. R. Flaxman, T. Braithwaite, M. V. Cicinelli, A. Das, and Vision Loss Expert Group, “Magnitude, temporal trends, and projections of the global prevalence of blindness and distance and near vision impairment: a systematic review and meta-analysis,” Lancet Global Health, vol. 5, no. 9, pp. e888–e897, 2017.
View at: Publisher Site | Google Scholar
T. R. Fricke, N. Tahhan, S. Resnikoff et al., “Global prevalence of presbyopia and vision impairment from uncorrected presbyopia: systematic review, meta-analysis, and modelling,” Ophthalmology, vol. 125, no. 10, pp. 1492–1499, 2018.
View at: Publisher Site | Google Scholar
M. Varghese, S. S. Manohar, K. Rodrigues, V. Kodkani, and S. Pendse, “The smart guide cane: an enhanced walking cane for assisting the visually challenged,” in 2015 International Conference on Technologies for Sustainable Development (ICTSD), Mumbai, India, 2015.
View at: Publisher Site | Google Scholar
A. Shaha, S. Rewari, and S. Gunasekharan, “SWSVIP-smart walking stick for the visually impaired people using low latency communication,” in 2018 International Conference on Smart City and Emerging Technology (ICSCET), pp. 1–5, Mumbai, 2018.
View at: Publisher Site | Google Scholar
M. P. Agrawal and A. R. Gupta, “Smart stick for the blind and visually impaired people,” in 2018 Second international conference on inventive communication and computational Technologies (ICICCT), pp. 542–545, Coimbatore, 2018.
View at: Publisher Site | Google Scholar
A. S. Al-Fahoum, H. B. Al-Hmoud, and A. A. Al-Fraihat, “A smart infrared microcontroller-based blind guidance system,” Active and Passive Electronic Components, vol. 2013, 7 pages, 2013.
View at: Publisher Site | Google Scholar
T. Miura, Y. Ebihara, M. Sakajiri, and T. Ifukube, “Utilization of auditory perceptions of sounds and silent objects for orientation and mobility by visually-impaired people,” in 2011 IEEE International Conference on Systems, Man, and Cybernetics, pp. 1080–1087, Anchorage, AK, 2011.
View at: Publisher Site | Google Scholar
H. Marion and J. Michael, Assistive Technology for Visually Impaired and Blind People, Springer Science & Business Media, 2010.
View at: Publisher Site
V. Tiponut, D. Ianchis, Z. Haraszy, and I. Bogdanov, “Work directions and new results in electronic travel aids for blind and visually impaired people,” WSEAS Transactions on Systems, vol. 9, pp. 1086–1097, 2010.
View at: Google Scholar
V. Tiponut, S. Popescu, I. Bogdanov, and C. Caleanu, “Obstacles detection system for visually impaired guidance,” in 12th WSEAS International Conference on SYSTEMS, Heraklion, Greece, 2008.
View at: Google Scholar
I. Ulrich and J. Borenstein, “Local obstacle avoidance with look-ahead verification,” in Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065), vol. 3, pp. 2505–2511, San Francisco, CA, USA, 2000.
View at: Publisher Site | Google Scholar
Y. Seki and T. Sato, “A training system of orientation and mobility for blind people using acoustic virtual reality,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 19, no. 1, pp. 95–104, 2011.
View at: Google Scholar
E. S. Narayanan, D. D. Gokul, B. P. Nithin, and P. Vidhyasagar, “IoT based smart walking cane for typhlotic with voice assistance,” in 2016 Online International Conference on Green Engineering and Technologies (IC-GET), pp. 1–6, Coimbatore, 2016.
View at: Publisher Site | Google Scholar
D. Yuan and R. Manduchi, “A tool for range sensing and environment discovery for the blind,” in 2004 Conference on computer vision and pattern recognition workshop, pp. 39–39, Washington, DC, USA, 2004.
View at: Publisher Site | Google Scholar
D. Yuan and R. Manduchi, “Dynamic environment exploration using a virtual white cane,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), vol. 1, pp. 243–249, San Diego, CA, USA, 2005.
View at: Publisher Site | Google Scholar
P. Baranski, M. Polanczyk, and P. Strumillo, “A remote guidance system for the blind,” in The 12th IEEE International Conference on e-Health Networking, Applications and Services, pp. 386–390, Lyon, 2010.
View at: Publisher Site | Google Scholar
T. Miura, K. Ueda, S. Ino, T. Muraoka, and T. Ifukube, “Object's width and distance distinguished by the blind using auditory sense while they are walking,” Journal of the Acoustical Society of America, vol. 123, no. 5, p. 3859, 2008.
View at: Publisher Site | Google Scholar
R. Boldu, D. J. C. Matthies, H. Zhang, and S. Nanayakkara, “AiSee: an assistive wearable device to support visually impaired grocery shoppers,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 4, pp. 1–25, 2020.
View at: Publisher Site | Google Scholar
A. A. Tahat, “A wireless ranging system for the blind long-cane utilizing a smart-phone,” in 2009 10th International Conference on Telecommunications, pp. 111–117, Zagreb, 2009.
View at: Google Scholar
D. Bolgiano and E. Meeks, “A laser cane for the blind,” IEEE Journal of Quantum Electronics, vol. 3, no. 6, pp. 268–268, 1967.
View at: Publisher Site | Google Scholar
S. Shoval, I. Ulrich, and J. Borenstein, “Robotics-based obstacle-avoidance systems for the blind and visually impaired - Navbelt and the guidecane,” IEEE Robotics and Automation Magazine, vol. 10, no. 1, pp. 9–20, 2003.
View at: Publisher Site | Google Scholar
L. Shangguan, Z. Yang, A. X. Liu, Z. Zhou, and Y. Liu, “STPP: spatial-temporal phase profiling-based method for relative RFID tag localization,” IEEE/ACM Transactions on Networking, vol. 25, no. 1, pp. 596–609, 2017.
View at: Publisher Site | Google Scholar
A. Raina and D. Bansal, “Poster: smart-phones as active sensing platform for road safety solutions,” in Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services Companion, pp. 74–74, Singapore, 2016.
View at: Google Scholar
A. Zvironas, M. Gudauskis, and D. Plikynas, “Indoor electronic traveling aids for visually impaired: systemic review,” in 2019 International conference on computational science and computational intelligence (CSCI), pp. 936–942, Las Vegas, NV, USA, 2019.
View at: Publisher Site | Google Scholar
N. A. Giudice and G. E. Legge, “Blind navigation and the role of technology,” in Engineering Handbook of Smart Technology for Aging, Disability, and Independence, A. Helal, M. Mokhtari, and B. Abdulrazak, Eds., pp. 479–500, John Wiley & Sons.
View at: Google Scholar
F. Xiao, Q. Miao, X. Xie, L. Sun, and R. Wang, “Indoor anti-collision alarm system based on wearable Internet of things for smart healthcare,” IEEE Communications Magazine, vol. 56, no. 4, pp. 53–59, 2018.
View at: Publisher Site | Google Scholar
M. F. Saaid, I. Ismail, and M. Z. H. Noor, “Radio frequency identification walking stick (RFIWS): a device for the blind,” in 2009 5th International Colloquium on Signal Processing & Its Applications, pp. 250–253, Kuala Lumpur, 2009.
View at: Publisher Site | Google Scholar
B. Andò, S. Baglio, V. Marletta, and A. Valastro, “A haptic solution to assist visually impaired in mobility tasks,” IEEE Transactions on Human-Machine Systems, vol. 45, no. 5, pp. 641–646, 2015.
View at: Publisher Site | Google Scholar
S. S. Reddy, G. Sathyaprabha, and P. V. V. P. Rao, “Voice based guidance and location indications system for the blind using GSM,” GPS and Optical Device Indicator, vol. 3, pp. 822–832, 2014.
View at: Google Scholar
E. Milios, B. Kapralos, A. Kopinska, and S. Stergiopoulos, “Sonification of range information for 3-D space perception,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 11, no. 4, pp. 416–421, 2003.
View at: Publisher Site | Google Scholar
X. Xing, R. Zhou, and L. Yang, “The current status of development of pedestrian autonomous navigation technology,” in 2019 26th Saint Petersburg International Conference on Integrated Navigation Systems (ICINS), pp. 1–3, Saint Petersburg, Russia, 2019.
View at: Publisher Site | Google Scholar
P. B. L. Meijer, “An experimental system for auditory image representations,” IEEE Transactions on Biomedical Engineering, vol. 39, no. 2, pp. 112–121, 1992.
View at: Publisher Site | Google Scholar
M. Olfati, W. Yuan, A. Khan, and S. H. Nasseri, “A new approach to solve fuzzy data envelopment analysis model based on uncertainty,” IEEE Access, vol. 8, pp. 167300–167307, 2020.
View at: Publisher Site | Google Scholar
M. Ahmad, A. Khan, A. Khan et al., “Spatial prior fuzziness pool-based interactive classification of hyperspectral images,” Remote Sensing, vol. 11, no. 9, article 1136, 2019.
View at: Publisher Site | Google Scholar
R. Chen, Z. Tian, H. Liu, F. Zhao, S. Zhang, and H. Liu, “Construction of a voice driven life assistant system for visually impaired people,” in 2018 International Conference on Artificial Intelligence and Big Data (ICAIBD), pp. 87–92, Chengdu, 2018.
View at: Publisher Site | Google Scholar
Bay Advanced Technologies, “BAT “K” sonar: ultrasonic sensing device for the blind,” 2012, http://www.batforblind.co.nz.
View at: Google Scholar
A. Garg, M. Balakrishnan, K. Paul et al., “Cane mounted knee-above obstacle detection and warning system for the visually impaired,” in Proceedings of the 11th International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, pp. 143–151, New York, NY, 2007.
View at: Google Scholar
D. T. Hartong, F. F. Jorritsma, J. J. Neve, B. J. Melis-Dankers, and A. C. Kooijman, “Improved mobility and independence of night-blind people using night-vision goggles,” Investigative Ophthalmology & Visual Science, vol. 45, no. 6, pp. 1725–1731, 2004.
View at: Google Scholar
A. Ullah, M. Shaheen, A. Khan, M. Khan, and K. Iqbal, “Evaluation of topology-dependent growth rate equations of three-dimensional grains using realistic microstructure simulations,” Materials Research Express, vol. 6, no. 2, article 026523, 2018.
View at: Publisher Site | Google Scholar
I. Ulrich and I. Borenstein, “Correspondence the guide cane — applying mobile robot technologies to assist the visually impaired,” IEEE Transactions on Systems Man and Cybernetics-Part A Systems and Humans, vol. 31, no. 2, pp. 131–136, 2001.
View at: Publisher Site | Google Scholar
M. Nie, J. Ren, Z. Li et al., “SoundView: an auditory guidance system based on environment understanding for the visually impaired people,” in Proceedings of the 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society: Engineering the Future of Biomedicine (EMBC’09), pp. 7240–7243, Minneapolis, MN, USA, September 2009.
View at: Google Scholar
K. W. Lin, T. K. Lau, C. M. Cheuk, and Y. Liu, “A wearable stereo vision system for visually impaired,” in 2012 IEEE International Conference on Mechatronics and Automation, pp. 1423–1428, Chengdu, 2012.
View at: Publisher Site | Google Scholar
L. Dunai, D. Ismael, L. Lengua, I. Tortajada, and S. Brusola, “Obstacle detectors for visually impaired people,” in 2014 International Conference on Optimization of Electrical and Electronic Equipment (OPTIM), Bran, Romania, 2014.
View at: Publisher Site | Google Scholar
M. A. Rahman, M. S. Sadi, M. M. Islam, and P. Saha, “Design and development of navigation guide for visually impaired people,” in 2019 IEEE international conference on biomedical engineering, computer and information Technology for Health (BECITHCON), pp. 89–92, Dhaka, Bangladesh, 2019.
View at: Publisher Site | Google Scholar
A. Riazi, F. Riazi, R. Yoosfi, and F. Bahmeei, “Outdoor difficulties experienced by a group of visually impaired Iranian people,” Journal of current ophthalmology, vol. 28, no. 2, pp. 85–90, 2016.
View at: Google Scholar
A. Kumar, R. Patra, M. Manjunatha, J. Mukhopadhyay, and A. K. Majumdar, “An electronic travel aid for navigation of visually impaired persons,” in 2011 Third International Conference on Communication Systems and Networks (COMSNETS 2011), pp. 1–5, Bangalore, 2011.
View at: Google Scholar

Copyright

Copyright © 2021 Asad Khan et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

1053

Downloads

692

Citations