Abstract
School bullying is a serious problem among teenagers, causing depression, dropping out of school, or even suicide. It is thus important to develop antibullying methods. This paper proposes a physical bullying detection method based on activity recognition. The architecture of the physical violence detection system is described, and a Fuzzy Multithreshold classifier is developed to detect physical bullying behaviour, including pushing, hitting, and shaking. Importantly, the application has the capability of distinguishing these types of behaviour from such everyday activities as running, walking, falling, or doing push-ups. To accomplish this, the method uses acceleration and gyro signals. Experimental data were gathered by role playing school bullying scenarios and by doing daily-life activities. The simulations achieved an average classification accuracy of 92%, which is a promising result for smartphone-based detection of physical bullying.
1. Introduction
School bullying is a common social problem among teenagers. It affects the victims both mentally and physically and is considered as one of the main reasons for depression, dropping out of school, and adolescent suicide [1, 2]. In view of this, antibullying is an important and timeless topic; one that has been studied ever since the 1960s. New approaches for preventing school bullying become available as sensor technology develops.
School bullying can take various forms, such as cursing and physical violence. According to a survey conducted by the authors in Finland and Indonesia, physical violence is considered as the most harmful to teenagers. Consequently, this paper will focus on detecting physical bullying.
Following the popularity of smartphones, several antibullying applications have entered the market, including Stop Bullies, ICE BlackBox, Campus Safety, and Back Off Bully. They all work in roughly the following fashion.
When a bullying event occurs, the victim or a witness needs to take out a smartphone, run the app, and press a button to send an alarm message. To photograph the event, they must hold the camera toward the bullies. However, this is not convenient for the victim, especially when bullied physically. Moreover, this action could infuriate the bullies and lead to harder bullying.
An application that could autonomously detect a bullying event, without letting the bullies know, would be highly desirable, especially if it came with the capability to send alarm messages automatically.
This paper proposes an algorithm for detecting physical bullying. It could be implemented on a smartphone with a 3D accelerometer and a 3D gyroscope. Once activated, it could run in the background and detect physical bullying automatically. On detecting a bullying event, it would send SMS alarm messages, without requiring any action from the user. A solution of this type requires reliable and automatic classification of physical activities.
As movement sensors have improved considerably in recent years, activity recognition has gained more attention in the literature. Current research focuses mainly on daily-life activities, such as standing, sitting, and walking and on state transitions, such as standing up, sitting down, and stopping walking.
Common motion sensors include 3D accelerometers, gyroscopes, and GPS. Main features in activity recognition include acceleration, inclination angle, GPS speed, and statistic values, such as standard deviation, skewness, signal magnitude area (SMA), and entropy [3–9].
Henpraserttae et al. [3] developed a -NN (-nearest neighbor, ) classifier for activity recognition with an average accuracy of 91%. Detected activities include sitting, standing, walking, and running.
Hache et al. [4] first developed a DT (double threshold) classifier for such simple daily-life activities as starting to walk, stopping walking, standing up, sitting down, and lying down. Their method received an average accuracy of 97%. Having improved the classifier [5], they then applied it to more complex activities, such as stair ascent, stair descent, ramp walking, entering/exiting a house, and starting/stopping a car. The average accuracy was 64%.
Ganti et al. [6] developed an HMM (hidden markov model) classifier to recognize daily-life activities, such as cooking, eating, driving and watching TV. The average accuracy was 65%.
Bicocchi et al. [7] compared four instance-based classifiers and finally chose the -NN () for their classification, because it was much simpler than the others while having similar accuracies. Three accelerometer sensors were fixed on the actor’s/actress’s arm, waist, and leg. Sixteen kinds of daily-life activities were tested, and an average accuracy of 95% was achieved. Furthermore, their idea of recognizing new type of activities other than the predefined classes in the training set in run time is impressive.
Previous work [8] by the authors’ research group implemented a state transition diagram-based classifier for activity recognition, relying on 3-axis acceleration and gyroscope data. They focused on state transitions rather than the states themselves, since irregular movement is more important both for the detection of school bullying and for elderly care [10, 11]. The results showed an average accuracy of 90% for several types of state transition, including standing up, sitting down, sitting up, lying down, starting walking, and stopping walking.
This paper designs a Fuzzy Multithreshold classifier to distinguish physical bullying from normal everyday activities. Such bullying activities as pushing, hitting, and shaking will be compared to nonbullying actions, such as falling, doing push-ups, running, and walking.
The remainder of this paper is constructed as follows: Section 2 describes the architecture of the physical violence detection system and the Fuzzy Multithreshold algorithm for detecting physical bullying; Section 3 presents experimental results using real data; Section 4 provides a discussion and suggests future work; and Section 5 draws the conclusions.
2. Algorithm for Physical Bullying Detection
The hardware architecture of the physical violence detection system is shown in Figure 1.
The accelerometer and the gyroscope are used for movement data sampling. The RAM is used for data and functions storage. The CPU is used for running the algorithm. And the SMS (short message system) module is used for sending alarm messages when a physical bullying event is detected.
Daily-life activities, such as standing up, sitting down, and walking, have regular styles and directions. But physical bullying is different, as hitting, shaking, and pushing may come from any direction and tend to exhibit a great degree of randomness.
Figure 2 shows examples of bullying and nonbullying activities.
(a) Pushing (bullying)
(b) Falling (nonbullying)
(c) Hitting (bullying)
(d) Walking (nonbullying)
As shown in Figure 2, differences between bullying and nonbullying activities are very difficult to detect. So a classification between bullying and nonbullying activities may be more difficult than a classification of daily-life activities.
The authors gathered experimental data by role playing school bullying and daily-life activities. A thorough analysis of these situations suggested that the best recognition method is the magnitude of activities. In view of this, the authors propose a Fuzzy Multithreshold algorithm for detecting bullying activities.
Since physical bullying usually takes place in the horizontal direction, the main features of the algorithm include a combined horizontal acceleration vector, as in (1), and a combined horizontal gyro vector, as in (2). To ascertain the direction, a vertical acceleration vector is also included. Consider
The -axis and -axis were defined as two orthogonal horizontal vectors, while the -axis was the vertical vector. Thus, (1) and (2) represent the combined horizontal acceleration and gyro vectors, respectively, by combining -axis and the -axis vectors.
SMAs of the acceleration and gyro vectors were also considered, but not used, as SMA-based classification accuracy is not as good as that based on horizontal vectors.
Figure 3 shows a flowchart of the proposed physical bullying detection algorithm.
After acquiring raw data from the 3-axis accelerometer and 3-axis gyroscope, the combined horizontal acceleration and gyro vectors, as well as the vertical acceleration vector, are extracted. Then, a sliding window of 8 s (determined experimentally) is applied to these features. At the first step, the algorithm needs to decide whether a fall has occurred, in which case it determines whether the data represent a pushing or a falling movement. Pushing will be classified as a bullying event, whereas falling is a nonbullying event. If no fall is detected in this window, the algorithm continues to a hitting detection module. Hitting and shaking will be classified as bullying events, while running, walking, and doing push-ups will be classified as nonbullying events.
2.1. Detection of Pushing
Firstly, a second-order Butterworth filter is applied to the combined horizontal acceleration vector to smooth out the jitter of the signal (Figure 4). Then, the highest peak PeakAcc in this window will be located. A fall is detected, when the peak value is high enough and the amount of vertical activity exceeds set threshold values before and after the highest peak. Specifically, for the period before the peak, the following relation should be satisfied: where denotes the vertical vector of the raw acceleration signal and represents gravity acceleration. For the period after the peak, the following relation should be satisfied: Finally, if PeakAcc > threshold , then a fall is detected. The parameter is used to distinguish a fall from a single punch or push (but not fall), which usually has a lower peak value.
(a) Before filtering
(b) After filtering
Body posture is determined by the vertical rather than the horizontal acceleration vector. This is because, just before hitting the ground, people tend to brace with their arms, causing a jitter in the horizontal vector. The average vertical vector is more stable both before and after a fall.
Since people often land on their knees after a fall, rather than lying prone, the threshold in (4) was set to the slightly lower value of .
The integration intervals in (3) and (4) were determined experimentally. A key point was that they should not include the falling process.
If the falling movement is not completed in a window, it can be detected in the following one. Since a fall occupies a very short time period (about 1-2 s), a window of 8 s is used, which should contain the entire event.
According to a video recorded during role playing, one main difference between being pushed and falling down becomes apparent the moment the person hits the ground. When pushed, a person usually slides for a short while in the direction of the fall. This rarely happens when a person falls. In order to differentiate between these two cases, integrated values for and GyroHori are calculated as follows: where the period includes the moment when the person hits the ground, and period length is decided experimentally. Several period length values were tested, and the one that best distinguished between pushing and falling was chosen. The variable Int() denotes total variation in speed, whereas Int() denotes total variation of the angle. Int() shows a better performance than Int() in distinguishing pushing from falling. Figure 5 shows Int() box plots of pushing and falling.
There is an overlapping part in the box plots, from which the threshold value of Int(GyroHori) between pushing and falling is selected. However, since samples in this overlapping area are potentially misclassified, another feature needs to be used to improve classification performance. For this purpose, the authors picked the maximum peak of the horizontal gyro vector PeakGyro.
Figure 6 shows the ROC curves (receiver operating characteristic curves) of Int(), PeakGyro, and a comprehensive classification for both. Although PeakGyro alone does not provide a high classification accuracy, a good result is reached when it is used together with Int(). Thresholds are set for Int() and PeakGyro, namely, and , respectively. Samples higher than the thresholds are classified as instances of pushing, whereas samples lower than the thresholds are identified as falls. Figure 7 summarizes the algorithm for pushing detection.
2.2. Hitting Detection
Only when no fall has been detected in the sliding window, the algorithm will continue to detect hitting and shaking. The reasons for this are as follows.(1)If the user has fallen down, there will not likely be hitting or shaking in this window.(2)If the user is pushed, a bullying event has already occurred, and an SMS alarm message will be sent.(3)Detecting a push or a fall (if either has occurred) will take almost half the window time, and the remaining data may not be reliable for hitting detection. Since hitting and shaking are usually continuous actions with a relatively long duration, they can be detected in the next window.
Firstly, a second-order Butterworth filter is applied to the horizontal vector of the acceleration signal (Figure 8). Then, the first and the last peak in the window are located. These should be higher than the threshold to reduce interference caused by weak activities or jitter.
(a) Running (before filtering)
(b) Running (after filtering)
(c) Hitting (before filtering)
(d) Hitting (after filtering)
Then, the integrations of and within the period are calculated as follows:
Two thresholds are set correspondingly, namely, and . For hitting, shaking, and doing push-ups, the integrated values Int() and Int() in (6) mostly exceed the thresholds, while those for walking and running tend to be lower than the thresholds. Since push-ups have different body direction from hitting and shaking, the absolute mean of the acceleration vertical vector within the integration period, as in (7), is used to distinguish push-ups from hitting and shaking. Consider
For push-ups, the absolute mean of is smaller than , whereas for hitting and shaking, it is larger than . The threshold of is determined by the moment when the body and the ground form an angle of 45° (arcsin 0.7).
The experiments showed that when the subjects did not run too fast, the classifier achieved good performance. However, when they ran fast or made a sharp turn when running, these activities had a high probability of being classified as bullying events. A further study was thus conducted on the video recorded during role play. It was found that, in running, the body moves rhythmically and that a corresponding wave form is obvious in the signal; when punched or hit, the body only exhibits a small movement component in the vertical direction. To operationalize this finding, a new parameter was defined in the period within as follows: where Gravity is the local gravity acceleration. For running, the values of Int(AccVert) are larger than the threshold ; whereas for hitting and shaking, the values are smaller than .
Bullying activities, such as hitting and shaking, and nonbullying activities, such as walking, running, and push-ups, can be well distinguished. Figure 9 shows the flowchart for hitting detection.
This algorithm only detects hitting and shaking that last for a sufficiently long period, that is, more than one peak (larger than Thpeaks) within the 8 s window, since hitting and shaking are usually continuous actions.
All the thresholds in this subsection were decided experimentally by inspecting box plots and ROC curves in the same way as in Section 2.1.
2.3. Fuzzy Classification
The thresholds mentioned above were derived purely for optimizing classifier performance. However, in practice, parents would prefer having a lower miss alarm probability at the cost of a higher false alarm probability, to protect their children. To that end, a fuzzy classifier was developed, one that can be easily configured to account for user preferences concerning the balance between missed alarms and false alarms. The principle of the proposed Fuzzy Multithreshold classifier is shown in Figure 10.
In fuzzy classification, samples are not classified crisply into given classes; rather, all input samples are assigned continuous membership values (between 0 and 1) within each class. For example, sample may have % membership value for Class and % membership value for Class . Membership values over the classes need not sum to unity.
By default, a sample is classified to the class with the highest membership value. For example, if the membership value of bullying is higher than that of nonbullying , the activity is classified as bullying. For example, if feature > threshold (or < threshold), the activity will be classified as bullying. When the user wants to reduce the miss alarm probability, the judgment will be , where and . And in this case, if, for example, feature (or ), where , the activity will be classified as bullying. When the user wants to reduce the false alarm probability, the judgment will be , where and . For example, if feature (or ), where , the activity will be classified as bullying.
Moreover, once the thresholds are determined, the classification does not need much computing as -NN or GMM does. So this algorithm can avoid time-consuming training procedures when put into practical use.
3. Experiments
Experimental data were gathered by role playing school bullying and daily-life activities. Three people participated in this role playing, taking turns to act as bullies and their victims. They first stood still for a few seconds to determine the vertical vector of the movement sensor. Once the sensor was fixed on the waist, its direction was not and should not be changed.
Activities such as hitting, shaking, doing push-ups, running, and walking were conducted about 10 times on average, with different speed and power. Activities such as pushing and falling were acted about 20 times on average, with different speed and power from different directions. Totally about 90 sections of activities, that is, about 66 000 feature vectors, were collected.
A 2-fold cross validation method was used for classification. Firstly, half the experimental data were used as the training set and the remainder as the test set. Then, a set of classification results was acquired. Secondly, the training and the test set were exchanged, and another set of classification results was acquired. Neither the training set nor the test set included all the actors, so the classification produced both same-user result and cross-user result, and the final result was the average of them. Table 1 shows averaged classification results.
Among bullying activities, hitting was recognized in 100% of the cases. Also the daily-life activities of running, walking, and doing push-ups were distinguished perfectly. However, 16% of pushing, 17% of shaking, and 21% of falls were misclassified. When the victim was pushed from the front, the bullies were afraid of hurting his hindbrain, so they used slightly reduced force. And when the victims fell while walking fast or running, the fall had a higher probability of being classified as pushing. It was obvious that the detection of falling and pushing depended on the applied external force and the victim’s speed, whereas shaking detection depended on the force employed by the bully. The authors are going to study this issue in more detail in future work.
4. Discussion
Since physical bullying usually takes place in the horizontal direction of the body, horizontal vectors and the vertical acceleration and gyro vectors are included in the SMA feature combining all three axis vectors. A problem with this implementation is that if the sensors are accidentally fixed in an inclined position, classification performance will suffer.
The waist is the best position for the movement sensor because (1) it is the center of the body, and bullying actions usually target the body and (2) the movement sensor can be fixed tight on the waistband, reducing the effect of jitter noise.
In the detection of pushing, experiments were also carried out in which the second highest peak before the maximum was used, that is, the moment when the victim was pushed or tripped. However, classification performance got lower when the activities of pushing and tripping contained extra randomness.
It is possible that additional features based on using magnetic fields could help to distinguish between bullying and nonbullying activities. It was observed that features of a magnetic field vary as another person approaches or departs from the vicinity of the device user. These features could be used to assess more widely the interaction between the victim and the bullies. However, a magnetic field is fragile against interference and difficult to analyze when the user is in a crowd. Numerous experiments need to be performed, before magnetic fields can be used to enhance classification accuracy.
5. Conclusion
School bullying is a common social problem, which affects teenagers in several ways. This paper focuses on methods of detecting physical bullying.
Experimental data were gathered by role playing school bullying and daily-life activities, using accelerometer and gyroscope motion sensors. By analyzing the acquired data, features such as combined horizontal acceleration and gyro vectors and vertical acceleration vector were selected for use.
A Fuzzy Multithreshold classifier was proposed, and its thresholds were determined by box plots and ROC curves. This classifier was able to distinguish such bullying activities as pushing, hitting, and shaking from daily-life activities such as falling, walking, running, and doing push-ups. An average accuracy of 92% was achieved in simulations.
To be convenient for the user, the classifier was designed to be configurable, allowing the alarm thresholds to be set by the user. This way, the user may choose a lower miss alarm probability at the cost of a higher false alarm probability.
The proposed algorithm can be used on smartphones with an accelerometer and a gyroscope to protect teenagers and young children. Once it detects a physical bullying event, a smartphone with this algorithm will automatically send SMS alarm messages to teachers and/or parents.
In future work, the authors intend to involve more test subjects and include more activities in physical bullying experiments to improve the performance of the classifier, especially with regard to the recognition of pushing and falling.
Appendix
Table 2 shows the thresholds mentioned in Section 3 of this paper. The sampling rates of the movement sensors are 50 Hz. The range of the accelerometer is (4 mg/LSB) and that of the gyroscope ±300°/s.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
This work was supported by the Natural Scientific Research Innovation Foundation in Harbin Institute of Technology under Grant no. HIT.NSRIF.2013029, the 973 Program under Grant no. 2013CB329003, and the National Science and Technology Major Project under Grant no. 2012ZX03004003 and partly by the China National Science Foundation under Grant no. 61101123.