Abstract
The similarity between a shock response spectrum (SRS) and a target shock specification is essential in evaluating the success of a qualification test of a space component. Qualification testing facilities often utilize shock response databases for rapid testing. Traditionally, the comparison of two shocks (SRS) depends on visual evaluation, which is, at best, subjective. This paper compares five different quantitative methods for evaluating shock response similarity. This work aims to find the most suitable metric for retrieving an SRS from a pyroshock database. The five methods are the SRS difference, mean acceleration difference, average SRS ratio, dimensionless SRS coefficients, and mean square goodness-of-fit method. None of the similarity metrics account for the sign of the deviation between the target SRS and database SRS, making it challenging to satisfy the criteria for a good shock test. We propose a metric (the weighted distance) for retrieving the most similar SRS to a target SRS specification from a shock database in this work. The weighted distance outperforms the mean square goodness-of-fit and other metrics in database SRS retrieval for rapid qualification testing.
1. Introduction
Small electronic components such as chips and crystals used in satellite-electronic boards are susceptible to the pyrotechnic shock events during a rocket launch. Pyrotechnic (explosive) shocks may lead to the failure of the satellite’s critical systems and components. Component failure modes may be cracks, fractures, seals failures, relay and switch chatter, and structural deformation [1]. We can guarantee that a component will retain its functional performance after exposure to a dynamic environment by doing a qualification testing (QT). In this case, we are considering a satellite attached to a rocket body and exposed to pyrotechnic shock. The QT aims to match the shock severity from a laboratory shock simulation to the shock generated during the satellite’s actual separation from the rocket body.
In mechanical shock characterization, there are many tools, such as the energy spectrum (Fourier transform), shock response spectrum (SRS), temporal moments, and wavelets for comparing signal features [2]. However, most of these tools’ starting point is the acceleration-time history, which is not usually available as a shock test specification. Shock tests are typically specified using the shock response spectrum (SRS). Comparing the SRS calculated from the acceleration measurements on a test article to a specified target SRS forms the basis for QT. The velocity-time history, Fast Fourier Transform (FFT), wavelets, or temporal moments are used to validate the measurements [2, 3].
The SRS is a measure of the peak response of a series of single-degree-of-freedom (SDOF) systems as a function of the systems’ natural frequencies. A digital filter models the SDOF at each natural frequency. There is an equal number of digital filters per octave between the upper and lower frequency limits. The SRS was first described in a thesis on the transient oscillations in elastic systems by Maurice Biot [4]. Since then, it has found applications in the characterization of aerospace components’ mechanical shock, upon sudden impact, or in the explosive (pyrotechnic) separation of spacecraft rocket stages [5].
The SRS is, first and foremost, a visual tool. To pass the QT, the SRS profile of a shock simulated in the laboratory must match or surpass the target SRS profile. When this happens, we assume the two shocks are equal in severity. The satellite (component) must not experience any physical or functional failure after the shock test. The first QT criterion is that, at least 50% of the SRS magnitudes should exceed the nominal test specification [1]. The second criterion is intuitive. The SRS specification is bounded by statistically derived lower and upper tolerance limits that define the range of acceptable SRS for the simulated shocks [6]. When tolerance limits bound the nominal test specification, 100% of the SRS magnitudes should exceed the nominal specification’s lower limit. Our recent testing experience suggests a client preference for 100% of the SRS magnitudes above the nominal specification. In pyrotechnic shock testing, it is difficult to match a laboratory-simulated shock to a target SRS profile. A pyroshock typically has high acceleration amplitudes that span a wide frequency bandwidth (up to 10 kHz in the near-field).
When there is a high demand for QT, it is beneficial to maintain a database of shock responses and the experimental conditions used in obtaining such SRS. Provided the testing machine has proven repeatability in the SRS measurements, we can retrieve the operating conditions for a future shock test from a database [7, 8]. The problem here is that retrieving the operating parameters for a future shock test from a previous shock test database is not well described in the available literature. For a large database, a simple visual evaluation of SRS similarity is insufficient. There is a need for a quantitative measure of SRS similarity.
The purpose of this paper is to propose a suitable metric for retrieving an SRS with a similar profile to a target SRS specification from a pyroshock database. First, we evaluate the suitability of different metrics used in other shock test applications for our intended purpose. The metrics include SRS difference, mean acceleration difference (MAD), dimensionless SRS coefficients ( and ), average SRS ratio (ASR), and the mean-squared (RMSE) goodness-of-fit metric [7]. The shock testing applications include a new shock machine’s characterization, demonstration of shock attenuation [9], and different simulation methods [9]. Apart from the mean squared goodness-of-fit metric [7], we did not find any instance where the other metrics were useful in extracting a similar SRS from an SRS database. In general, we did not find any publication with a detailed explanation of the metric used for database retrieval.
In the following sections, we describe the experimental setup and data acquisition method. Next, we discuss the different metrics listed above for comparing two shocks. Next, we introduce the weighted distance as a metric for database SRS retrieval and illustrate its application using an example. The paper concludes with a discussion of our results and a summary of our findings.
2. Experimental Setup and Data Collection
The data used in this work were measured using three ENDEVCO 2225 accelerometers attached to an interface jig on an air gun shock machine. The test article was mounted on the interface jig. The shock measurements were carried out along three orthogonal axes, as shown in Figure 1. The test article is an aluminum structure used in the vibration testing of 1 to 3 U CubeSats. The shock machine produces a shock by the mechanical impact of a projectile and the shock table. The projectile is propelled by air pressure and impacts the XY plane of the shock table directly beneath the point where the test article is mounted. Usually, we measure the acceleration-time histories at three positions corresponding to the X, Y, and Z axes. The data sampling speed was 100,000 samples per second. The accelerometer signals were amplified using a SHOWA 4035-52 (large input) charge amplifier. The amplified signals were obtained on the control PC through a NI 9222 module with a maximum sampling capability of 500 kS/s. SRS computation was at an octave spacing of 1/48 from 10 Hz to 10 kHz and a 5% damping ratio (dynamic amplification Q = 10).
For data with a zero shift error, the SRS is corrected using the wavelet-based zero shift correction program described in [10]. A zero shift may sometimes be undetectable by visual inspection of the acceleration-time history. However, it manifests as a constant maximax acceleration in the SRS’s low-frequency region. The velocity-time history may also indicate a zero shift if the net velocity change is not 0. Zero shift occurs predominantly in piezoelectric accelerometers due to spurious charge outputs. The spurious charges may be caused by saturation in the accelerometer’s transducer or the signal conditioner. It may also be due to cabling problems in the measurement setup [11]. Zero shift correction is carried out on a case by case basis depending on the shock test’s purpose. Figure 2 shows the velocity-time history and the positive and negative SRS of an acceleration-time history before and after zero shift correction using the wavelet method.
(a)
(b)
(c)
(d)
After correction, the velocity-time history shows a net-zero velocity, and the SRS reflects the correct slope. Accepting corrected shock data exhibiting zero shift requires engineering judgment. A shock data exhibiting zero shift is still useful for evaluating the SRS in the high-frequency region (>1 kHz). According to [10], the correction algorithm is accurate within 4 dB of the actual acceleration measurement. However, our policy is to repeat a test whenever the data exhibits a zero shift.
3. SRS Comparison
We can evaluate the similarity between two or more shocks quantitatively by comparing their SRS. A visual distinction of 2 SRS is sufficient in some applications to explain the two shocks’ differences. However, a quantitative description of the similarity or difference between two SRS may be necessary for other applications. Below is a brief description of some of the methods that describe the similarity between two SRSs quantitatively.
3.1. Visual Evaluation
The SRS specification with a given lower and upper limit is a type of control chart. The control chart concept was first introduced by Shewhart [12] as a method of quality control. In the control chart, acceptable measurement values fall within tolerance limits derived empirically.
The data that lies outside the tolerance limits are out-of-control and are therefore of unacceptable quality. Typically, the upper and lower limits for an SRS specification are for natural frequencies below 3 kHz and when the natural frequency is above 3 kHz [1]. Figure 3 shows the SRS from 3 shock measurements for a shock test specification.
3.2. SRS Difference (DIFF)
The simplest method to evaluate the similarity (or difference) between 2 SRSs is subtracting one from another (equation (1)). The SRS difference is the absolute value of the difference in acceleration of the 2 SRS (in dB) at each natural frequency. This method was used in [9] to show SRS attenuation for different kinds of absorber materials. One of the SRS data is a reference, while the other is the measured SRS. For example, in Figure 3, the difference between the upper limit and the SRS specification is 6 dB at all natural frequencies below 3 kHz and 9 dB otherwise:
3.3. Mean Acceleration Difference (MAD)
In equation (2), the MAD is a ratio of the absolute difference between a reference SRS and a measured (or modeled) SRS for natural frequencies in the measurement bandwidth. The MAD gives a single scalar value to measure the similarity (or difference) between 2 SRSs. In contrast, the SRS difference method shows a pointwise comparison. Suppose the reference SRS is the same as the measured SRS. In that case, the MAD is 0 and tends to positive infinity as the SRS difference becomes more pronounced. This method was used in [13] to compare SRS measurements:
3.4. Average SRS Ratio (ASR)
In [14], ASR is the ratio of measured SRS to a reference SRS. In equation (3), is the number of successive measurements or simulation trials. According to the authors, there was poor repeatability between successive tests below 1 kHz, so the computation only covers natural frequencies above 1 kHz. The measurement system could accurately measure near-field pyroshock with frequency content of up to 10 kHz. For our application, we require no frequency control above 5 kHz:
3.5. Dimensionless SRS Coefficients
Recent shock publications express that the similarity between two or more SRS is in terms of a dimensionless coefficient [15–19]. The two dimensionless coefficients used frequently are the mean SRS coefficient, , and the maximum SRS coefficient, . The two SRS coefficients are calculated using equations (4) and (5). In [19], the dimensionless coefficients are also called attenuation rates, and was calculated using the formula for the MAD in equation (2):
3.6. Mean Squared Goodness-of-Fit (RMSE)
The mean-squared goodness-of-fit method involves calculating the root-mean squared-error between a measured SRS and a reference SRS. The RMSE is similar to the SRS difference calculation. In the RMSE, a single scalar quantity (in dB) represents the difference between the two SRS. In contrast, the SRS difference can only be shown visually as a pointwise difference between two SRS. This method was used in [7] for rapid shock testing. Equation (6) shows the formula for calculating the mean square deviation in dB, where is the number of natural frequencies in the SRS’s computation:
Table 1 shows a summary of the metrics discussed in this section.
4. Application: Retrieving an SRS from a Database for a Shock Test
Pyrotechnic shock testing facilities often carry out large volumes of shock qualification tests with various SRS profiles. Usually, the SRS measurements from these tests and the associated shock machine operation conditions form a shock-test database. For a future test, the test engineer retrieves an SRS profile similar to the target SRS specification from the database as well as the associated shock machine operating conditions [7, 8, 20, 21]. This section compares the effectiveness of the SRS comparison methods discussed in section 3 for retrieving an SRS from a database for a shock test. The comparison is limited to purely quantitative (nonvisual) methods.
The database comprises 324 channels of recently measured SRS at the QT facility of the Center for Nanosatellite Testing (CeNT). Section 2 describes the data acquisition process. The target shock specification is the ISO 19683 [22] recommended shock level. The upper and lower limits of the ISO 19683 are +6 dB and −6 dB, respectively (Table 2). By default, the SRS is computed automatically at a 1/48 octave spacing between 10 Hz and 10 kHz, corresponding to 480 data points (natural frequencies). However, we used a total of 324 natural frequencies between 100 Hz and 5 kHz. The SRS below 100 Hz are typically not considered in practice, and the 5 kHz upper limit is due to the upper limit of the example SRS specification in Table 2. The test bandwidth is within the maximum usable bandwidth of the accelerometer according to the 1/5th transducer resonance rule of thumb [23]. The database SRS was evaluated based on two criteria. The first criterion is that 50 percent of the measured data points shall exceed the SRS specification [1, 6]. The second criterion is that 100% of the data points shall exceed the lower limit.
Table 3 shows the results of the MAD, ASR, dimensionless coefficients, and RMSE as a metric for retrieving a similar SRS to the SRS specification in Table 2 from a small database of 10 shocks. The analysis was extended to a bigger dataset of 324 channels to see the metrics’ effect on the database size. Tables 3 and 4 show the analysis results. Figures 4 and 5 show the top three SRS selected for each metric when the database size is 10 and 324.
(a)
(b)
(c)
(d)
(e)
(f)
(a)
(b)
(c)
(d)
(e)
(f)
4.1. Discussion
The results in Tables 3 and 4 and Figures 4 and 5 reflect the following points:(i)The MAD ranges from 0 (for entirely similar SRS) to positive infinity. Here, the biggest MAD is 159.8%. The lower the MAD is, the more similar the database SRS is to the target SRS.(ii)When the MAD is used for database SRS retrieval, some of the best results do not meet the required minimum percentage of data points above the target SRS specification. For example, 67% of the data points for SRS6 (Table 3) and 70% of the data points for SRS100 (Table 4) fall below the target SRS specification for the MAD.(iii)The ASR and reflect a similar pattern. Theoretically, the measured SRS is entirely similar to the target SRS when the ratio (ASR ) is 1. A higher ratio implies that the measured SRS is higher than the target SRS.(iv)Table 3 shows that 67% of SRS6, 67% of SRS 4, and 48% of SRS1 (retrieved for the ASR ) fall below the target SRS. In a bigger dataset (Table 4), the ASR and have an approximate value equal to 1. However, Figure 5 shows that the SRS selected using this method significantly falls below the target SRS specification. The reason is that the ASR excludes the SRS’s low-frequency portion (<1000 Hz) from the computation. Similarly, for , the SRS magnitudes at the high-frequency portion (>1000 Hz) are several orders of magnitude higher than in the lower frequencies. Therefore, the sum of the measured SRS magnitudes may exceed the target SRS magnitudes.(v)The considers only the peak SRS magnitude (without considering the frequency at which it occurs). Shock transients typically have instantaneous high magnitude accelerations. Even when the peak amplitude is the same (SRS2), the two SRS profiles may be entirely different. A useful metric should account for all the SRS magnitudes at all the natural frequencies. The has the worst performance of the metrics compared.(vi)The retrieved SRS from the RMSE computation lies between the specified lower and upper tolerance limits. However, only 33% of SRS4 in Table 3 and 30% of SRS100 in Table 4 falls above the target SRS specification.(vii)The MAD and RMSE are based on the absolute difference between the SRS magnitudes of the target SRS and the measured SRS. The drawback here is that the absolute difference does not account for the sign of the deviation between the 2 SRS. Therefore, it is challenging to satisfy the criterion that at least 50% of the measured SRS magnitudes should exceed the target SRS. Most of the database SRS with a small RMSE has more than 50% of the SRS magnitudes falling below the target SRS at the lower frequencies (<1000 Hz).
4.2. Improved Mean Squared Error Scheme for SRS Database Similarity Estimation
It is clear from section 5 that we need a metric to choose a preferred database SRS that satisfies the test criteria. The metric must account for the sign of the deviation of the SRS magnitudes. The new metric is an improvement on the RMSE computation. To overcome the limitations of the RMSE, we introduce weights, , and a correction factor, , to the RMSE computation. The weights and correction factor are combined in equation (7) to form the weighted distance (weighted RMSE), . We can calculate the weighted distance over the entire frequency spectrum (from 10 Hz to 10 kHz) or across any frequency range of interest:where
If we have an SRS database with N entries, is the normalized RMSE for each SRS . Normalizing is carried out by dividing the calculated RMSE for each SRS with the entire database’s RMSE range. Mathematically, becomes undefined if . However, this is highly unlikely in a database comprising different shocks. For mathematical consistency, equation (8) is valid in all instances, where . The weight, , is the percentage of negative deviations from the target SRS for each database SRS. In equation (8), is the ratio of the RMSE for each SRS to the sum of RMSE for all the SRS in the database. If , then the weighted distance is determined by . However, if is small, is a coefficient chosen to prioritize the selection of an SRS in the database with more positive deviations from the target SRS . The database SRS are ranked using the weighted distance based on the following conditions:(i)Suppose several SRS in the database has equal RMSE. In that case, the SRS with more positive deviations from the target SRS is the preference.(ii)If two SRS have an equal-weighted distance, the SRS with the lower RMSE is preferred.
The aim is to define a metric such that the desired boundary of the distance is [0, 1]. Similarity decreases as the weighted distance approach a value of 1. Note that the value of may slightly exceed 1. However, since our primary interest is to select values closer to zero, the values of closer to 1 does not impact the selection. Therefore, we automatically assign 1 as the value of for every instance where is greater than 1.
Table 3 and Figure 4 show the result of the weighted RMSE. The most similar database SRS to the target SRS is SRS5 . The next most similar SRS are SRS3 and SRS8 . The initial database SRS is then ranked from minimum to maximum using the weighted distance. Considering the criterion on the minimum percentage of data points above the target SRS specification, Figure 5 shows that the weighted RMSE outperforms other metrics in retrieving an SRS from a database. The top three SRS retrieved using this method have 100% of their data points above the target SRS specification. There is no significant difference in the percentage of data points above the specification when the octave spacing increases from 1/48 to 1/12. Table 5 summarises the effect of increasing the octave spacing.
It is also possible to use the weighted distance for SRS specifications based on three-axis accelerometer readings. In this case, three-axis measurement refers to independent measurements taken on the X, Y, and Z faces of an interface jig using three single-axis accelerometers (Figure 1). Additional weights may be applied per axis if a statistical (or empirical) relationship exists between the three measurements. We did not apply a different weight per axis since the SRS measurements are independent. If an SRS database comprises 108 shocks (N) and each shock has SRS three measurements corresponding to X, Y, and Z axes (324 channels), the weighted distance is calculated for each axis and averaged over the three axes for each test. Equation (9) shows the computation of the weighted distance for each test, . Figure 6 shows that all of the data points between 100 Hz and 5 kHz for the three axes SRS are above the SRS specification:
5. Conclusion
We evaluated five SRS similarity metrics for suitability in retrieving an SRS with a similar profile to a target SRS specification from a shock test database. All evaluated metrics are calculated from the absolute deviation between the SRS magnitudes. The SRS’s absolute deviation does not account for the sign of the deviations of the database SRS from the target SRS. Among the metrics, the mean square goodness-of-fit (RMSE) algorithm shows the best results in retrieving experimental conditions for a previous shock test from an SRS database. However, the RMSE algorithm may sometimes retrieve a database SRS that does not comply with QT criteria.
We introduced an improved goodness-of-fit metric that meets the minimum percentage of data points (50%) that should exceed the specified target SRS magnitudes. We demonstrated the effectiveness of our newly improved metric, the weighted RMSE , by retrieving the most similar SRS to a target shock test from a small database and a big database. We also extended the method to the 3-axis SRS measurements from the 108 shock tests.
In the future, we hope to utilize a deep learning framework to extract features for similarity estimation between two or more shock transients. In the current database retrieval scheme, the acceleration-time history, SRS, and experimental conditions of past shock tests are always available ahead of time. A deep learning method could help determine the experimental conditions for a given shock transient if the experimental conditions are unknown or if the record is lost.
Abbreviations
| : | Normalized RMSE for each database entry |
| : | Scalar multiplier |
| : | RMSE ratio for the jth SRS in the database |
| : | Natural frequency (Hz) |
| : | Number of natural frequencies (points) in the SRS calculation |
| : | Number of successive trials in a shock simulation |
| N: | Total number of SRS in the database |
| : | Database entry number |
| : | SRS measured in an experiment or a simulation |
| : | Target SRS of a shock specification |
| : | % negative deviation from |
| : | Weighted distance for the jth database SRS |
| : | Weighted distance for triaxial SRS measurements. |
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.