- About this Journal ·
- Abstracting and Indexing ·
- Advance Access ·
- Aims and Scope ·
- Article Processing Charges ·
- Articles in Press ·
- Author Guidelines ·
- Bibliographic Information ·
- Citations to this Journal ·
- Contact Information ·
- Editorial Board ·
- Editorial Workflow ·
- Free eTOC Alerts ·
- Publication Ethics ·
- Reviewers Acknowledgment ·
- Submit a Manuscript ·
- Subscription Information ·
- Table of Contents
International Journal of Microwave Science and Technology
Volume 2014 (2014), Article ID 958905, 21 pages
A Multifrequency Radar System for Detecting Humans and Characterizing Human Activities for Short-Range Through-Wall and Long-Range Foliage Penetration Applications
The Pennsylvania State University, University Park, PA 16802, USA
Received 23 December 2013; Accepted 9 February 2014; Published 16 April 2014
Academic Editor: Xianming Qing
Copyright © 2014 Ram M. Narayanan et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
A multifrequency radar system for detecting humans and classifying their activities at short and long ranges is described. The short-range radar system operates within the S-Band frequency range for through-wall applications at distances of up to 3 m. It utilizes two separate waveforms which are selected via switching: a wide-band noise waveform or a continuous single tone. The long-range radar system operating in the W-Band millimeter-wave frequency range performs at distances of up to about 100 m in free space and up to about 30 m through light foliage. It employs a composite multimodal signal consisting of two waveforms, a wide-band noise waveform and an embedded single tone, which are summed and transmitted simultaneously. Matched filtering of the received and transmitted noise signals is performed to detect targets with high-range resolution, whereas the received single tone signal is used for the Doppler analysis. Doppler measurements are used to distinguish between different human movements and gestures using the characteristic micro-Doppler signals. Our measurements establish the ability of this system to detect and range humans and distinguish between different human movements at different ranges.
The ability to detect human targets and identify their movements through building walls and behind light foliage is increasingly important in military and security applications. Expeditionary warfighters and law enforcement personnel are commonly faced with unknown enemy threats from behind different types of walls as well as those concealed behind shrubs and trees. Technology that can be used to unobtrusively detect and monitor the presence of human subjects from stand-off distances and through walls and foliage can be a powerful tool to meet such challenges. Although optical systems achieve excellent angular resolution, optical signals are unable to penetrate solid barriers and foliage cover and therefore are totally ineffective in detecting humans in defilade. However, signals in the microwave frequency range can penetrate barriers to an acceptable degree and are therefore the sensors of choice in detection of targets through optically opaque walls. In this case, the choice of the frequency of operation depends on the application, specifically on the barrier type, target position behind the wall, stand-off requirement, and resolution requirements, all of which are somewhat interrelated. Furthermore, since signals in the millimeter-wave frequency range are able to penetrate light foliage cover to an acceptable degree and can be focused to isolate a single human being, they are emerging as the sensors of choice in detection of targets hidden in foliage. The choice of the frequency of operation depends on the application, specifically on the atmospheric attenuation, stand-off requirements, and resolution requirements, all of which are somewhat interrelated.
Low-frequency microwave signals, less than 5 GHz in frequency, can penetrate building walls made of concrete, brick, or cinder blocks, with reasonably low loss. A noteworthy point is that humans behind walls are located at much shorter range from the radar sensor (typically 6–10 feet); thus portable antennas with relatively wider beamwidths can easily isolate a single human. Millimeter-wave systems typically operate in one of the atmospheric “windows,” which offer low propagation loss. These windows exist around 35, 95, 140, and 220 GHz frequencies. The W-Band of the microwave part of the electromagnetic spectrum ranges from 75 to 110 GHz, thus covering the 95 GHz window. The short wavelengths at these frequencies permit the use of small portable antennas to achieve the required angular resolution in order to isolate a single human.
The antenna beamwidth , that is, the “field of view” of the antenna beam, of a circular aperture antenna (in radians) with a typically used parabolic aperture taper is given by where is the wavelength and is the antenna size . In radar applications, the two-way beamwidth is needed, which takes into account the combined transmit/receive antenna pattern. The two-way beamwidth is given by by reasonably assuming a Gaussian-shaped main beam antenna radiation pattern. The term in the denominator appears due to the fact that we are considering the angle between the half-power points of the two-way, that is, transmit/receive, antenna pattern. As an example, assuming a 3 GHz transmit frequency in the S-Band frequency range (2–4 GHz) corresponding to a wavelength of 10 cm (4 in) and a manageable antenna size of 6 inches, the two-way beamwidth is computed as 0.6 radians or 34.2 degrees. At a target range of 2 m (~6 ft), the azimuth or cross range resolution for a real-aperture radar, given by is computed as 1.1 m (~3.6 ft), which is considered adequate for isolating a single human. In addition, assuming a 95 GHz transmit frequency in the W-Band corresponding to a wavelength of 3.15 mm (~1/8 in) and a manageable antenna size of 15 cm (6 in), the two-way beamwidth is computed as 0.018 radians (~1 degree). At a target range of 100 m (~300 ft), the azimuth or cross range resolution is computed as 1.7 m (~5.4 ft), which is considered reasonably adequate for isolating a single human. While higher millimeter-wave frequencies achieve narrower beamwidths, the W-Band frequency range covering the 95 GHz window is preferred due to the lower cost and more extensive availability of components in this range.
The down-range resolution is solely determined by the transmit bandwidth and is expressed as whereis the speed of light . Therefore, a 500 MHz transmit bandwidth yields a down-range resolution of 30 cm or 1 ft, quite adequate for isolating a single human. Several frequency-modulated waveforms operating over the bandwidth required for achieving the desired down-range resolution can be employed for through-wall imaging applications .
This paper discusses the architecture of the multifrequency radar system and presents data showing that human detection and human activity characterization are possible through different types of barriers. Section 2 provides an overview of noise radar and the micro-Doppler signal analysis. Sections 3 and 4 provide details of the design of the S-Band and W-Band portions of the multifrequency radar system, respectively. Experimental results are shown in Section 5 and conclusions are presented in Section 6.
2. Principles of Noise Radar and the Micro-Doppler Analysis
2.1. Noise Radar
While adequate cross range resolution can be achieved using small size antennas for short-range wall penetration, a suitable modulation scheme must be used to obtain the wide transmit bandwidth of 500 MHz to achieve the desired down-range resolution. Random noise modulation is an ideal candidate for military applications since it possesses several desirable properties, such as covertness, low probability of detection (LPD), low probability of intercept (LPI), immunity from jamming, and resistance to interference, owing to its totally featureless characteristics . Only the basic principles of random noise radar are presented here for the sake of completeness. In a random noise radar, target detection and ranging are accomplished by cross-correlating the target reflected signal with a time-delayed replica of the transmit waveform . The round-trip return time for a target located at a range of is given by
Let represent the transmitted wideband noise waveform whose autocorrelation is given by From (6), we see that the autocorrelation shows a peaked response at zero lag and is zero elsewhere. The reflected signal from the target is delayed by and can therefore be represented as . If a portion of the transmitted noise waveform is captured within the radar and internally delayed by a duration , it can be represented as . The cross correlation of the two signals and shows a peaked response only when and is zero when . By stepping through various internal delays and determining the particular value of at which a peak occurs in the cross correlation response, target range can be determined using
A simplified block diagram of a noise radar is shown in Figure 1, wherein the mixer acts as the cross correlator . Current technological advances permit the implementation of fully digital radar architectures for noise signal generation and processing in the microwave frequency range and thereby achieve a great degree of flexibility [7, 8]. Signals in the millimeter-wave range of frequencies can be generated and processed using up- and downconversion of the digitally generated lower frequency microwave signals.
2.2. The Micro-Doppler Analysis
When a moving target is illuminated with a single tone frequency of corresponding to a wavelength of , it induces a Doppler frequency shift in the reflected signal, which is given by where is the target’s radial velocity with respect to the radar antenna. Mechanical vibration or rotation of structures in a target may induce frequency modulation on the target reflected signals and generate sidebands about the center frequency of the target’s body Doppler frequency . These modulations, which are usually at very low frequencies relative to the body Doppler frequency, are known as the micro-Doppler signatures. A stationary target of course produces no Doppler shift. However, if a stationary target vibrates, rotates, or maneuvers, its structural parts are in motion, and these induce the micro-Doppler modulations. If there are structures with being the radial velocity of the th structure, the composite micro-Doppler signal has frequency components at , which are unique to the specific motional characteristics of the target. Analysis of the micro-Doppler signatures in the joint time-frequency domain can provide useful information for target detection, classification, and recognition.
The micro-Doppler signals are also present in human activity, such as breathing and swinging arms, since each activity involves different types of motions of the chest, torso, and limbs. Figure 2 shows the micro-Doppler signatures of humans behind a wooden wall performing several distinctive activities. The radar stand-off distance was 9 m (30 ft) . There are significant differences in the signatures leading us to believe that not only can we detect concealed humans using radar but we can also identify what they are doing by analyzing their micro-Doppler signatures, which may help us to infer intent. At millimeter-wave frequencies, the Doppler signals occur at much higher frequencies due to the shorter wavelengths. In addition, smaller scale movements can be more easily recorded at these shorter wavelengths.
3. S-Band Radar System Description
A brief summary description of the S-Band through-wall radar system is provided below. A more complete description can be obtained from .
3.1. Baseband Dual-Mode Waveform Generation
In order to both detect humans and characterize their micro-Doppler signatures, a composite waveform is used, consisting of a wideband noise waveform for ranging and a single tone continuous wave signal for micro-Doppler detection. These waveforms are generated at lower frequencies, called baseband, and then upconverted to the desired frequency range of operation. The noise waveform of 500 MHz bandwidth is generated over the frequency range 100 Hz to 500 MHz, while the single tone is located at 300 MHz. An RF switch is used to select either waveform; therefore, the system operates in either the ranging mode (using the noise waveform) or in the Doppler mode (using the single tone). Each waveform is split and one-half is upconverted to the desired frequency range of operation, while the other half of the signal is routed to the receiver for performing the ranging or the micro-Doppler processing with the received and downconverted signal.
3.2. S-Band Radar System Overview Description
A simplified block diagram of the system is shown in Figure 3. The noise source produces a noise waveform with over the 100 Hz to 500 MHz range at an output power of 10 mW, which is filtered to achieve a good spectral shape. After filtering, the waveform passes through a power splitter where one output serves as a reference signal and the other output goes to a switch (SPDT RF Switch) for upconversion. The single tone is generated by a voltage controlled oscillator (VCO), which produces an output of about 4 mW at the desired frequency. This signal is filtered and then split in a similar fashion, where one output leads to the switch awaiting upconversion while the other output terminal goes to the receiver. Depending on user preference, either the noise signal or single tone is selected by the RF switch and sent to an upper sideband upconverter, which is pumped by a high-frequency single tone S-Band local oscillator signal at frequency (in MHz). Thus, the upconverter selects the upper sideband of the mixing process. For the ranging mode, the upconverter output exists over a frequency range between and , while, in the Doppler mode, the output frequency is equal to . The upconverter output is amplified, filtered (again), and transmitted via a transmit antenna.
A two-stage downconversion receiver processor is used in the system. The time-delayed received signal is collected by an identical receive antenna, amplified, and filtered to remove out of band interference and noise. Then, it is downconverted using the same single tone S-Band signal at frequency as the local oscillator, which yields the reflected noise signal over 0 to 500 MHz range in the ranging mode or a micro-Doppler modulated single tone around 300 MHz in the Doppler mode. These signals are separated into different paths via appropriate filters and sent to a data acquisition system and the digitizer. The digitizer also receives the transmitted samples in both the ranging and the Doppler modes. The reference and the received noise signals are digitally cross-correlated to obtain range to target, while the reference single tone and the received micro-Doppler modulated single tone are mixed together after being low-pass filtered and processed to extract the micro-Doppler modulation. The sampling frequency for each channel is 2 GS/s for ranging and 1.25 GS/s for the micro-Doppler signature, more than adequate to satisfy the Nyquist sampling criterion [12, 13].
RF antennas are usually linearly polarized. However, most reinforced building walls contain a lattice of reinforcing bars, or rebars which may be either vertically oriented, or horizontally oriented, or both. Such a structure will affect the propagation of EM waves through it, especially if the rebars are oriented in the direction of the wave polarization. A method to overcome this limitation is to employ a circularly polarized wave, wherein the instantaneous polarization of the wave moves around a circle, thereby allowing most of the wave to pass through with very little loss due to the choice of the wrong polarization. In our system, therefore, we employed helical antennas which are able to transmit and receive circularly polarized signals . They consist of a conducting element wound in the geometrical shape of a helix. The conductors are supported by a central buttress frame, and together they are mounted on a ground plane. To enhance the gain of the antenna and thereby reduce its beamwidth as well as to reduce the beam sidelobes and back lobe, the ground plane can be modified in the shape of a “salad bowl” curved towards the helix, as suggested in .
It is known that targets reflect the oppositely handed polarization when illuminated by a circularly polarized wave. The helices used for the transmit and the receive antennas are oppositely wound so that the transmitted wave is right-hand circularly polarized whereas the receive antenna is left-hand circularly polarized. The helical antennas were operated in the axial mode; that is, the antenna dimensions were comparable to the wavelength, wherein a directional endfire pattern, along the axis of the helix, is achieved. The helical antennas designed for this application had the following dimensions: (a) outside rim diameter of the salad bowl shaped ground plane = 18.8 cm (7.4 in), (b) bottom diameter of the ground plane = 8.89 cm (3.5 in), and (c) overall axial length = 35.3 cm (13.9 in). The designed antenna is shown in Figure 6.
3.4. Wall Construction
A wall support frame was constructed to house different masonry materials in a dry-stack fashion. The frame was designed to support a wall (e.g., brick or cinder block) that was 2.44 m (8 ft) tall × 2.44 m (8 ft) wide. In addition, the frame had an adjustable width for wall thicknesses of 10.2, 20.3, or 30.5 cm (4, 8, or 12 in, resp.). The structure stood a foot above the ground (adding additional height to the wall) on castor wheels enabling the wall to be mobile. Figure 7 shows the constructed wall with 20.3 cm (8 in) thick cinder blocks.
To collect radar data, the antennas were mounted on a wooden stand that positioned the antennas approximately 1.37 m (54 in) above the ground and about 1.83 m (6 ft) from the front of the wall. Care was taken to align the antennas properly since poor alignment could negatively influence the results. Coaxial cables of adequate length were connected to the antennas from the radar system (allowing some separation). Targets, such as metallic trihedral corner reflectors and humans were located behind the wall at various distances.
4. W-Band Radar System Description
A brief summary description of the W-Band foliage penetration radar system is provided below. A more complete description is provided in reference .
4.1. Baseband Multimodal Waveform Generation
In order to both detect humans and characterize their micro-Doppler signatures, a composite waveform is required, consisting of a wideband noise waveform for ranging and a single tone continuous wave signal for the micro-Doppler detection. These waveforms are generated at lower frequencies, called baseband, and then upconverted to the desired frequency range of operation. The noise waveform of 500 MHz bandwidth is generated over the frequency range of 1.1–1.6 GHz in the L-Band frequency range, while the embedded single tone is located at 1.1 GHz. Both signals are summed together, upconverted to the desired frequency range at W-Band, and transmitted as a composite multimodal waveform. Thus, the system operates simultaneously in both the ranging mode (exploiting the noise waveform component) and the Doppler mode (exploiting the single tone component). Just prior to waveform summation in the transmit chain, one-half of each signal is split and routed to the receiver for performing the cross correlation operation with the received and downconverted signal. Our system was designed in two main sections, a low-frequency L-Band section and a high-frequency mm wave section. Both the L-Band and mm wave sections can then be further subdivided into transmit and receive chains.
4.2. W-Band Radar System Overview Description
A simplified block diagram of the system is shown in Figure 8. The L-Band and the W-Band transmit and receive chains are clearly demarcated. The low-frequency noise source produces a noise waveform over the 1.1 to 1.6 GHz range at an output power of 6.25 mW, which is filtered to achieve a good spectral shape. The single tone output power at 1.1 GHz is 5 mW. Thus, the total power of the composite multimodal waveform is 11.25 mW. The W-Band transmit chain accepts the signal from the L-Band transmit chain and upconverts it to W-band via a mixer followed by a high-pass filter (HPF), which discards the lower sideband. The mixer affords good suppression of the W-Band local oscillator source leakage. The SSB upconversion is possible because the L-Band signal is offset in frequency from baseband; that is, it is created from 1.1 to 1.6 GHz. This scheme avoids the necessity to use an expensive in-phase and quadrature (I/Q) SSB upconverter. Figure 9 illustrates the frequency spectrum of the signal after upconversion and filtering. After upconversion, the composite signal power is reduced to 4 mW due to conversion loss of the mixer. The multimodal signal at W-Band is then transmitted using a transmit antenna.
Once the signal reflects off of an object, the W-Band receive chain captures the backscattered signal through the receive antenna. The received signal is amplified using a low noise amplifier and downconverted back to the 1.1–1.6 GHz frequency range. The signal is then sent to the L-Band receive chain for further processing. The L-Band receive chain takes this signal and prepares it for the final downconverting stage. This process consists of amplifying and filtering the signal before downconverting to baseband. Down converting both the Doppler and noise waveforms to baseband simultaneously is possible because the single tone is placed at the beginning edge of the band. After downconversion to baseband, the Doppler and noise waveforms are separated by splitting the signal and filtering appropriately. Since the Doppler signal is located in the range of DC to a few kHz, a low-pass filter (LPF) is used to band-limit the signal and to avoid aliasing unwanted signals components when the signal is digitized. The noise waveform contains a DC offset created by the single tone mixing with itself so a DC blocker in addition to a LPF is used to prepare the noise waveform for digitizing. A copy of the transmitted signal is needed as a reference to the matched filter. A copy of the noise waveform is sampled from the L-Band transmit chain and downconverted using an identical mixer and local oscillator as is used for the received signal. The signal is then filtered and attenuated before being digitized. Once the Doppler and noise waveforms are available at baseband, they are digitized using two separate digitizers. The Doppler signal is digitized using a low sample rate digitizer since these frequencies are quite low. Both the received and reference noise waveforms are digitized using high-speed digitizer with the sample rate set to 1 Gs/sec to satisfy the Nyquist sampling criterion [12, 13]. Both digitizers are connected to a laptop and interfaced with LabView for processing and data saving.
The component layout of the W-Band radar is shown in Figure 10. The W-Band component tray fits atop the L-Band tray in the fully packaged system. The antennas shown in Figure 11 are smaller pyramidal horn antennas for closer range measurements, while long-range field measurements used a larger circular dielectric horn lens antenna.
Two different antennas were used, depending upon the target range considerations. For close range measurements, pyramidal horn antennas of aperture size 1.0625 inches × 0.875 inches, shown in Figure 11(a), were used. The one-way beamwidths in the principal planes for this antenna are computed as 8.4 degrees × 10.2 degrees. In this antenna, the feed waveguide aperture is flared in both dimensions to achieve a higher gain over the operating waveguide bandwidth . These antennas achieve a cross range resolution of 1.7 m (5.4 ft) at a target range of approximately 14.7 m (48 ft). Beyond this range, it is not possible to isolate a single human; therefore, larger size antennas were used for longer range applications. For longer range measurements, circular dielectric horn lens antennas of 15.24 cm (6 in) aperture diameter, shown in Figure 11(b), were used. In this antenna, a dielectric lens is integrated into a dielectric loaded horn antenna to improve the antenna efficiency . The principle involved in geometrical optics lens design is to collimate the rays from the primary source by refraction at the surface of the lens. These antennas had a one-way beamwidth of approximately 1.5 degrees, achieving a cross range resolution of 1.7 m (5.4 ft) at a target range of 91.4 m (300 ft).
4.4. Foliage Cover Description
In addition to unobstructed long-range measurements, data were also collected to investigate the radar system’s ability to detect targets through light foliage. To do this, we aimed the radar at a Border Forsythia (Forsythia × intermedia) shrub of approximate dimensions 2 m × 2 m × 2 m and placed different targets behind it. This shrub has an upright habit with arching branches and grows to 3 to 4 m high. Data were collected from targets behind the bush when it had leaves and also after the leaves fell off to see the effect the leaves have on the system’s ability to identify targets behind it. A diagram of experimental setup is shown in Figure 12 and photographs of the foliage penetration measurement setup are shown in Figure 13.
To collect radar data, the antennas were mounted on a wooden stand that positioned the antennas approximately 1.37 m (54 in) above the ground. Care was taken to align the antennas properly using sighting scopes since poor alignment could negatively influence the results. Targets, such as metallic trihedral corner reflectors and humans, were located at various distances.
5. Experimental Results and Data Analysis
5.1. Background Subtraction
A major problem in through-wall radar is the existence of large peaks in the reflected response due to direct antenna coupling as well as the reflection from the wall itself. These signals can obscure the target reflections, as can be seen in Figure 14. In order to overcome this limitation, background subtraction was used  for the S-Band radar, which resulted in the excellent suppression of constant nontarget induced responses and significant enhancement of the target-induced response, which is also shown in Figure 14. The residual clutter, which is not completely suppressed, is due to mutual interactions between different reflectors.
5.2. Distance Correction
Direct antenna coupling also obscured the low target reflections from longer ranges for the W-Band radar, as can be seen from the correlation plot in Figure 15(a) for the data acquired from a human target under unobstructed conditions at a range of 700 feet (213 m). In order to overcome this limitation, background subtraction was used  and the results are shown in the correlation plot in Figure 15(b). The residual clutter, which is not completely suppressed, is due to mutual interactions between different reflectors. In addition, there are still background peaks of comparable amplitudes to that of the human. These peaks are much closer to the radar. However, the radar range equation tells us that the received power from a target falls off as , which is called the spreading loss. The spreading loss effect is more pronounced in the case of the long-range radars, where the maximum target distance is several orders of magnitude compared to the minimum target distance. Thus, if the same target is placed at twice the distance from the radar antenna, the power actually received at the radar is reduced by a factor of . A straightforward method is to weight the data in order to compensate for the increasing signal attenuation as a function of range caused by the material attenuation and spreading loss . Therefore, in order to compensate for this roll-off and equalize the target response at different ranges, an inverse distance correction of was implemented to adaptively enhance the return from distant targets. Figure 15(c) shows the distance-corrected correlation plot, wherein the peaks from the human target can be more clearly detected and identified.
5.3. Human Detection and Human Movement Tracking through Wall
For the S-Band though-wall radar, a human was located at a distance of 1.22 m (4 ft) behind a 10.2 cm (4 in) thick brick wall with the antennas located at a distance of 1.83 m (6 ft) in front of the wall. Thus, the distance between the antennas and the human was about 3.05 m (10 ft). We note from Figure 16 that the background subtraction technique is able to detect the human quite clearly.
In addition, background subtraction can also be used for detecting moving targets. Subtraction of successive frames of the cross correlation signals between each received element signal and the transmitted signal has been shown to be able to isolate moving targets in heavy clutter . To accomplish this, a total of 10 previous correlation scenes which were aggregated for averaging were also subsequently used to perform frame by frame subtraction. The plots shown in Figure 17 clearly demonstrate the ability of the algorithm to track a moving human.
5.4. Target Detection through Foliage
For the W-Band radar foliage penetration experiments, we used two targets, a corner reflector and a human. The radar was located at a stand-off distance of 30 m (98.4 ft) from the bush and each target was placed 2 m (6.6 ft) behind the bush. Correlation data were averaged over 100 looks to reduce the effects of noise. Figures 18(a) and 18(b) show baseline data for the bush with no leaves and corner reflector, respectively. The average correlation value, which is proportional to the received power, of the bush with no leaves is 0.09, while it is 5.0 for the unobstructed corner reflector. When the corner reflector was placed behind the bush, its average correlation value dropped to 0.92, as seen in Figure 18(c). This corresponds to a two-way RF signal loss of 7.4 dB for the bush with no leaves. We note that the reflection from the bush in this scene is consistent with the baseline bush data. Data were also taken when the bush had leaves on it and the corner reflector was placed behind the bush. The correlation plot in Figure 18(d) shows that the reflection from the bush has doubled to 0.18 (due to the leaves), while the response from the corner reflector is reduced to 0.12. This corresponds to a two-way loss through the bush with leaves of 16.2 dB.
In addition, data were also collected for a human target. Correlation data taken for the human behind the bush with no leaves is shown in Figure 19. This plot shows that the response of the human behind the bush with no leaves is about 0.04. This is 23 times less (13.6 dB less) than the response of the corner reflector for the same setup. When the loss through the bush with leaves is taken into account, we infer that detection of human is not possible under these conditions. However, we will show later on that the human can be detected behind a fully foliated bush by using the micro-Doppler characteristics of the human movement activity.
5.5. Human Activity Recognition Approach
Since different human activities result in different micro-Doppler signatures, a technique was developed and implemented for automatic classification of specific human activities, more fully described in , and hence not repeated. Since the radar micro-Doppler signals are generally nonlinear and nonstationary, conventional Fourier-based approaches are not optimal for their analysis. The basis of our approach is to decompose the micro-Doppler signals using the empirical mode decomposition (EMD) into their intrinsic oscillatory modes called the intrinsic mode functions (IMFs) . The faster oscillations in the signal are present in the lower-indexed IMFs while the slower oscillations reside in the higher-indexed IMFs. These IMFs are components of the original signal and each IMF is orthogonal to all of the other IMFs. Each IMF comprises signal components that belong to a specific oscillatory time scale. The energy as a function of the IMF index provides us with a unique feature vector for human activity classification.
Classification of signals, such as the micro-Doppler signatures, requires a unique feature vector for each signal. EMD readily provides a feature vector by the calculation of the energy of each IMF component or the inner product of the signal with itself. When the EMD process is conducted on the micro-Doppler signals, the collection of IMF energies provides us with a vector that is unique to the movement that caused the Doppler frequency shift.
Support vector machines (SVMs) have proven to be an effective alternative to traditional classification techniques, such as the Bayesian classifiers and artificial neural networks (ANNs) . The primary advantages of SVMs over other methods are their ability to generalize and relative ease of implementation. The classifier is optimized to produce a model that is based on the training set feature vectors and their associated known class label. Using this model, the test set can be accurately classified using only their feature vectors, without knowledge of the class label.
The classification is performed using an SVM with a Gaussian kernel. The constrained optimization problem is formulated as where is the weight vector that defines a linear hyperplane separating the two classes of data, is the constant offset of the hyperplane, is a measure of the error of any misclassification for the th class, and is a penalty parameter that allows the classifier to tolerate some errors. The vector function maps the feature vectors into an -dimensional space. The parameters and are the feature vectors and their associated class label (±1), respectively. For the specific problem of classifying the micro-Doppler signals that arise from human motion, the values are the energy feature vectors extracted using EMD.
SVMs were originally developed to solve the binary classification problem; therefore, modifications must be made in order to extend the binary problem to a multiclass problem. Multiple methods have been proposed to tackle this problem. Because of its intuitiveness and its ability to be easily adapted for additional classes, the one-against-all (1-a-a) method was chosen for the experiments . The five human activity classes tested are as follows: (1) noise, that is, no human present, (2) breathing, (3) swinging arms, (4) picking up an object on the ground from a standing position, and (5) transitioning from crouching to standing.
5.6. Human Activity Classification Results for S-Band Through-Wall Radar
Experimentally observed time-frequency plots of unobstructed (without wall) human activities are shown in Figure 20 for the S-Band through-wall radar, wherein clear differences are observed. With the presence of a wall between the radar and the human, it was noted that the signal-to-noise ratio (SNR) was degraded and the micro-Doppler features were distorted, as shown in Figure 21. This indicates that the feature vectors for through-wall recognition of human activities are expected to be different from those without a wall.
Three cases were examined for activity classification for the S-Band through-wall radar: (1) direct transmission without a wall barrier, (2) transmission through a 10.2 cm (4 in) thick brick wall, and (3) transmission through a 20.3 cm (8 in) thick cinder block wall. For the direct transmission case, the person was located 3.35 m (11 ft) away from the radar antennas. For the brick wall case, the person was located 1.52 m (5 ft) behind the wall with the wall 1.83 m (6 ft) away from the radar antennas. For the cinder block case, the signals were very weak, so the distances were shortened. The person was located 91 cm (3 ft) behind the wall and the wall was located 61 cm (2 ft) away from the radar antennas. The five different motions listed above were considered for classification. The training set consisted of data from five of the six test subjects and was used both for training the classifier and for the cross-validation. The cross-validation set also consists of data from five of the six test subjects, but these data were used only for cross-validation and not used for training the classifier. The test set consists of data from one of the six test subjects and these data were used neither for training the classifier nor for cross-validation. Classification results are shown in Table 1. The average accuracy when combining the results of all six of the test subjects is 63.9%, 49.8%, and 64.1% for no barrier, brick wall, and cinder block wall, respectively. Although the classification accuracies appear to be low, it must be borne in mind that different individuals perform physical activities quite differently based on body shape and cultural factors.
5.7. Human Activity Classification Results for W-Band Foliage Penetration Radar
Five different motions were investigated for the W-Band foliage penetration radar. These include (a) no activity (for reference), (b) breathing, (c) swinging arms, (d) picking up object from ground, and (e) transitioning from crouching to standing. Experimentally observed time series and time-frequency plots of unobstructed (without foliage cover) human activities at a range of 30 m (98.4 ft) are shown in Figure 22, wherein clear differences between different activities are observed. The time series plots are just the raw data versus time recorded by the system when the specified activity takes place, while the time-frequency plots show the Doppler frequency as a function of time, with the higher amplitudes signifying faster movement. From Figure 22(b), it is evident that breathing produces the Doppler frequency shifts on the order of a few tens of Hz. Figure 22(c) shows that the arm swinging motion is periodic every second or so. The time-frequency plot also shows that there are multiple components of the body moving while the arms are swinging. The Doppler frequencies lower than 100 Hz correspond to the larger, slower torso swaying while the smaller, faster Doppler response from the arms swinging goes up to 900 Hz. In the picking up object motion in Figure 22(d), there are two distinct high-frequency pulses that correspond to the person first bending over and then standing back up. These pulses last about 1 second, and within each pulse there are multiple components in the Doppler signal relating to different parts of the body. The higher frequency lower amplitude component corresponds to the motion of the head and reaches frequencies up to 800 Hz. This is because the pivot point is at the waist, the head is the fast moving object when bending over and standing back up, and the head is smaller in physical size than the torso and shoulders. The lower frequency higher amplitude components correspond to the high RCS torso, arms, and shoulders moving slower because they are close to the pivot point. These motions produce the Doppler frequencies up to 400 Hz. For the crouching to standing motion in Figure 22(e), there is one high-frequency pulse that lasts about 1 second and tapers off into lower frequency swaying. The lower frequency swaying is caused by the human being slightly unstable from the process of standing from the crouch position. This motion does not have a pivot point so all body parts move at the same time with similar speeds, making it difficult to identify different body parts based on speed. The highest Doppler frequencies reached in this motion are about 300 Hz and the low speed swaying causes the Doppler frequencies of <100 Hz. Figure 23 shows just the time-frequency plots of the same activities at a range of 90 m (295 ft), which are very similar to the 30 m measurements, except for a reduction in amplitude. Note that the Doppler frequencies and the associated time durations are also nearly identical. The radar’s ability to detect the human Doppler signal through light foliage was also tested. The test setup was the same as Section 4.4 and is as shown in Figures 12 and 13. In this scenario, the received signal contains Doppler signatures of both the wind-influenced foliage and the human activity behind it. For this case, the background Doppler data are very important. This is because, unlike the ranging data, the Doppler background cannot be averaged or subtracted out. In this case, filtering out the dominant frequencies produced by the foliage is needed. When filtering is performed, it consequently filters out some of our desired signals, but in many cases there is enough information remaining to identify human motion. Figure 24(a) shows the time series and time-frequency plots of the background Doppler data from the foliage. As seen in time-frequency plot, the majority of the background Doppler is present in the frequencies less than 50 Hz, which is much lower than the Doppler from the human activity (except for the breathing). For this reason, a high-pass filter with a cutoff frequency of 50 Hz was applied to reduce the effect of the foliage. Figure 24(b) shows the corresponding background Doppler data plots after the filter is applied, from which we note that the response from the foliage is reduced by a factor of 10–20 dB. An example of how effectively the filtering works is shown in Figure 25 for a human picking up an object behind the bush. The filtered time series and time-frequency plots in Figure 25(b) are almost identical to the free-space measurement shown in Figure 23(d). Similar results were obtained for swinging arms and transitioning from crouching to standing. Unfortunately, since the breathing Doppler signals are of the same order as those caused by the swaying of the bush, it is not possible to detect human breathing while standing still behind a bush.
All five different motions listed above were considered for classification. The training set consisted of data from five of the six test subjects and was used for both training the classifier and for the cross-validation. The cross-validation set also consists of data from five of the six test subjects, but these data were used only for cross-validation and not used for training the classifier. The test set consists of data from one of the six test subjects and these data were used neither for training the classifier nor for cross-validation. Classification results are shown in Table 2. The average accuracy when combining the results of all six of the test subjects is 79.2%, 79.6%, and 94.3% for human targets at ranges of 30.5, 61, and 91.4 m (100, 200, and 300 ft, resp.) from the radar antennas, respectively. These are good classification accuracies. The reasoning behind the high accuracy at 91.4 m (300 ft) is because, at the shorter distances, the human target will move outside of the antenna beam for portions of the motions, especially for picking up an object and transitioning from crouching to standing. At the longer distances, the entire body is within the antenna beam, but the illuminated area is still small enough to isolate the human target. As stated earlier, different individuals perform physical activities quite differently based on body shape and cultural factors.
To the best of our knowledge, this is the first reporting of the ability to classify different types of human activity behind opaque walls and foliage cover. While the results obtained thus far are quite encouraging, more research and system development are needed to improve the classification accuracies in the presence of barriers and to include additional movements and gestures. We are currently working on expanding the range of human activities as well as the variety of humans for additional data collection.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
This work was supported by the U.S. Army ARDEC Joint Service Small Arms Program (JSSAP) under Contract no. W15QKN-09-C-0116. The authors appreciate helpful comments provided by E. Beckel, W. Luk, and G. Gaeta of ARDEC.
- W. L. Stutzman and G. A. Thiele, Antenna Theory and Design, John Wiley & Sons, Hoboken, NJ, USA, 3rd edition, 2013.
- D. N. Keep, “Frequency-modulation radar for use in the mercantile marine,” Proceedings of the Institution of Electrical Engineers B: Radio and Electronic Engineering, vol. 103, no. 10, pp. 519–523, 1956.
- F. Ahmad and R. M. Narayanan, “Conventional and emerging waveforms for detection and imaging of targets behind walls,” in Through-the-Wall Radar Imaging, M. G. Amin, Ed., pp. 157–184, CRC Press, Boca Raton, Fla, USA, 2010.
- L. Turner, “The evolution of featureless waveforms for LPI communications,” in Proceedings of the IEEE National Aerospace and Electronics Conference (NAECON '91), pp. 1325–1331, Dayton, Ohio, USA, May 1991.
- B. M. Horton, “Noise-modulated distance measuring systems,” Proceedings of the IRE, vol. 47, no. 5, pp. 821–828, 1959.
- R. M. Narayanan, “Through-wall radar imaging using UWB noise waveforms,” Journal of the Franklin Institute, vol. 345, no. 6, pp. 659–678, 2008.
- B. Ferguson, S. Mosel, W. Brodie-Tyrrell, M. Trinke, and D. Gray, “Characterisation of an L-band digital noise radar,” in Proceedings of the IET International Conference on Radar Systems (RADAR '07), pp. 1–5, Edinburgh, UK, October 2007.
- E. K. Walton, “Digital noise radar prototype development,” in Proceedings of the 30th Antenna Measurement Techniques Association Annual Symposium, pp. 23–26, Boston, Mass, USA, November 2008.
- V. C. Chen, “Analysis of radar micro-doppler signature with time-frequency transform,” in Proceedings of the IEEE Signal Processing Workshop on Statistical Signal and Array Processing (SSAP '00), pp. 463–466, Pocono Manor, Pa, USA, August 2000.
- P.-H. Chen, M. C. Shastry, C.-P. Lai, and R. M. Narayanan, “A portable real-time digital noise radar system for through-the-wall imaging,” IEEE Transactions on Geoscience and Remote Sensing, vol. 50, no. 10, pp. 4123–4134, 2012.
- S. Smith and R. M. Narayanan, “Ranging and target detection performance through lossy media using an ultrawideband S-band through-wall sensing noise radar,” in Radar Sensor Technology XVII, vol. 8714 of Proceedings of SPIE, pp. 1–12, Baltimore, Md, USA, April 2013, 871408.
- H. Nyquist, “Certain topics in telegraph transmission theory,” Transactions of the American Institute of Electrical Engineers, vol. 47, no. 2, pp. 617–644, 1928.
- C. E. Shannon, “Communication in the presence of noise,” Proceedings of the IRE, vol. 37, no. 1, pp. 10–21, 1949.
- J. D. Kraus, “The helical antenna,” Proceedings of the IEEE, vol. 37, no. 3, pp. 263–272, 1949.
- A. R. Djordjević, A. G. Zajić, and M. M. Ilić, “Enhancing the gain of helical antennas by shaping the ground conductor,” IEEE Antennas and Wireless Propagation Letters, vol. 5, pp. 138–140, 2006.
- K. A. Gallagher and R. M. Narayanan, “Human detection and ranging at long range and through light foliage using a W-band noise radar with an embedded tone,” in Radar Sensor Technology XVII, vol. 8714 of Proceedings of SPIE, pp. 1–12, Baltimore, Md, USA, April 2013, 871402.
- M. J. Maybell and P. S. Simon, “Pyramidal horn gain calculation with improved accuracy,” IEEE Transactions on Antennas and Propagation, vol. 41, no. 7, pp. 884–889, 1993.
- A. D. Olver and B. Philips, “Integrated lens with dielectric horn antenna,” Electronics Letters, vol. 29, no. 13, pp. 1150–1152, 1993.
- M. Piccardi, “Background subtraction techniques: a review,” in Proceedings of the IEEE International Conference on Systems, Man and Cybernetics (SMC '04), pp. 3099–3104, The Hague, The Netherlands, October 2004.
- D. J. Daniels, “Radar for non destructive testing of materials,” in Proceedings of the IEE Colloquium on Measurements, Modelling and Imaging for Non-Destructive Testing, pp. 9-1–9-3, London, UK, March 1991.
- H. Wang, R. M. Narayanan, and Z. O. Zhou, “Through-wall imaging of moving targets using UWB random noise radar,” IEEE Antennas and Wireless Propagation Letters, vol. 8, pp. 802–805, 2009.
- D. P. Fairchild and R. M. Narayanan, “Micro-doppler radar classification of human motions under various training scenarios,” in Active and Passive Signatures IV, vol. 8734 of Proceedings of SPIE, pp. 1–11, Baltimore, Md, USA, April 2013, 873407.
- N. E. Huang, Z. Shen, S. R. Long et al., “The empirical mode decomposition and the Hubert spectrum for nonlinear and non-stationary time series analysis,” Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, vol. 454, no. 1971, pp. 903–995, 1998.
- C. Cortes and V. Vapnik, “Support-vector networks,” Machine Learning, vol. 20, no. 3, pp. 273–297, 1995.
- R. Rifkin and A. Klautau, “In defense of one-vs-all classification,” The Journal of Machine Learning Research, vol. 5, pp. 101–141, 2004.