Abstract
This study proposes the first fully deep learning-based structural response intelligent computing framework for civil engineering. For the first time, from the data side to the model side, the structural information of the structure itself and any loading system is comprehensively considered, which can be applied to materials, components, and even structures, system and other multi-level mechanical response prediction problems. First, according to the characteristics of structural calculation scenarios, a unified data interface mode for structural static characteristics is formulated, which preserves the original structural information input and effectively reduces manual intervention. On this basis, an attention mechanism and a deep cross network are introduced, and a structural static feature representation learning model PADCN is proposed, which can take into account the memory and generalization of structural static features, and mine the coupling relationship of different structural information. Then, the PADCN model is integrated with the dynamic feature prediction model Mechformer and connected with the designed general data interface to form an end-to-end data-driven structural response intelligent computing framework. In order to verify the validity of the framework, numerical experiments were carried out with the steel plate shear wall structure as the carrier, in which a data augmentation algorithm suitable for the field of structural calculation was proposed to alleviate the problem of lack of structural engineering data. The results show that the deep learning model based on this framework successfully predicts the whole-process nonlinear response of specimens with different structures, the simulation accuracy is better than that of the fine finite element model, and the computational efficiency exceeds the traditional numerical method by more than 1000 times, achieving a qualitative improvement. It is proven that the intelligent computing framework has excellent accuracy and efficiency.
1. Introduction
Deep learning is the product of a new round of scientific and technological revolution and industrial transformation. It permeates all aspects of human social activities and production activities. Intelligent machines and equipment will definitely replace human large-scale manual labor and work in extremely harsh environments. In the field of civil engineering, artificial intelligence technology is deeply integrated into the entire life cycle process of architectural planning, design, construction, and operation, profoundly changing the development of civil engineering, and comprehensively improving the level of mechanization, automation, informatization, and intelligence [1]. The use of machine learning in the architectural planning phase is a new approach. Machine learning is at the heart of artificial intelligence, improving optimization algorithms based on data and previous experience through machine learning of the existing surrounding environment, geological conditions, objective factors, and needs of big data such as human and traffic behavior, combined with virtual reality situation reproduction technology, to create a new model of planning and design, and to avoid possible errors in reality, providing a greener building environment to achieve intelligent planning [2].
The whole world is working hard on the industrial transformation of the new generation of information technology, modern manufacturing, and producer services. Internationally, in 2012, Germany launched the “Industry 4.0 Plan” focusing on “smart factories.” In December 2016, the United Kingdom published “Artificial Intelligence: Opportunities and Implications for Future Decision Designation” [3]. France released its Artificial Intelligence Strategy in March 2017. Japan has designated 2017 as the first year of artificial intelligence to promote the construction of a “super-intelligent society 5.0.” In 2018, the United States released a national strategy for artificial intelligence. China is one of the countries with the earliest and fastest AI action in the world. In July 2015, the State Council issued the “Guiding Opinions on Actively Promoting the “Internet” Action,” which clearly listed “Internet + AI” as a key action. On July 20, 2017, the State Council issued the “New Generation Artificial Intelligence Development Plan,” which clearly pointed out that artificial intelligence is the core technology of a new round of scientific and technological revolution and industrial transformation. Compared with other industries, although both steel (steel structure) and concrete (concrete engineering) are products of industrialization, the degree of mechanization, automation, intelligence, and informatization of infrastructure is still relatively low. Artificial intelligence technology will permeate all aspects of human social activities and production activities, and human large-scale manual labor and work in harsh environments will be replaced by machines or robots. In the field of civil infrastructure, artificial intelligence technology deeply integrates the whole life cycle of civil infrastructure planning, design, construction, and maintenance, and profoundly changes the development of civil engineering [4].
Building Information Modeling plays an extremely important role in the design phase. BIM technology is a data-based tool for engineering design, construction, and management. By applying BIM technology, virtual reality, cloud computing, and other technologies, it can eliminate design problems such as mistakes, omissions, and defects, reduce the cost of simulation analysis and optimization calculation of design schemes, and effectively to shorten the construction period, improve the visualization level of the design results, and significantly improve the design efficiency and quality [5]. For example, the data from the Shanghai Center project survey show that the use of BIM technology can eliminate 40% of engineering changes, eliminate 90% of drawing errors, reduce rework by 60%, and shorten the construction period by 10%, greatly improving project benefits.
2. Basic Research Fields of Artificial Intelligence
Four basic research areas: natural language processing, computer vision, speech recognition, and cross-cutting areas.
2.1. Natural Language Processing
Natural Language Processing (NLP) is the processing of human-specific natural language by a computer as a medium, so that computers can “process” and “understand” natural language like humans [6]. In the field of civil engineering, NLP has shown great application prospects from basic semantic similarity, and dependency syntax analysis to applied human-computer interaction, report analysis, etc. It can be transformed into unstructured risk information using NLP. The construction organization plan information is transformed into structured information, so as to mine the tacit knowledge (such as dangerous objects, dangerous locations, accident causes, and accident types) of the daily documents of civil engineering construction projects [7].
In 2016, Tixier et al. proved that the use of NLP can eliminate reporting errors caused by manual information analysis. Using the NLP system can automatically scan and quickly analyze a large number of unstructured reports with an accuracy rate of over 95%. We obtained a large number of reliable structured data sets from the information report database, so as to extract new safety information and improve project safety management; in 2018, Wang Fei et al. reviewed the development of natural language processing driven by deep learning and believed that deep learning promoted natural language advances in processing and natural language processing to provide broader application prospects for deep learning; in 2019, Kim et al. proposed an NLP-based knowledge management system for construction accident cases, as shown in Figure 1. In this system, the information retrieval model can be used to query accident cases that are more than 97% related to user intentions, and the information extraction model can be used to automatically analyze tacit knowledge in accident cases to achieve efficient risk management; in 2020, Li Zhoujun et al. proved that static and dynamic pretraining technology combs new pretraining methods including BERT and XLNet and gives the future development direction.
Based on the current research status, the research depth and application scope of NLP are still relatively low [8]. First of all, it is manifested in the poor generality of the thesaurus in the construction field, which leads to the low quality of file preprocessing, which will affect the text data segmentation and part-of-speech tagging in the NLP process. Procedures adversely affect; then there is a limited formulation of information extraction rules, that is, in the field of civil engineering; it is difficult to obtain all project data (such as project contracts, etc.), making it difficult to develop all possible rules for information extraction; In addition, NLP, the deep learning training model, is related to local languages. The same model cannot process text information in different languages, so effective transfer learning cannot be carried out. Finally, the current NLP is mostly used in the construction stage, while the application in the design, maintenance, and other phases is carried out in the subsequent phases. It leads to low efficiency and quality of document management in the whole life cycle of civil engineering [9].
2.2. Computer Vision
Computer vision uses an imaging system instead of the visual organ as an input sensing means, and an intelligent algorithm instead of the human brain as a processing and analysis hub, extracting symbolic and digital information from images and videos for target recognition, detection, and tracking, and finally. It enables computers to “observe” and “understand” the world through vision like humans [10]. Computer vision has received a lot of research in the field of civil engineering in concrete crack detection, structural damage identification, construction site safety monitoring, etc. and has a very broad application prospect [11].
In 2011, Zaurin et al. proposed the use of computer vision for health monitoring of bridge structures, combining images and videos with computer vision technology to detect, classify and track different vehicles, and use sensor data to determine the standardized response of bridge structures; in 2015, Seo summarized the computer vision-based construction site safety and health monitoring methods, divided the previous computer vision research into three categories: target monitoring, target tracking, and action recognition, and proposed a general framework for computer vision-based safety and health monitoring; in 2018, Han Xiaojian et al. used computer vision technology to conduct research on crack detection on concrete surfaces and established a deep convolutional neural network crack recognition model that can automatically locate cracks and obtain crack widths from images, with a recognition accuracy of over 98%; in 2020, Zhou et al. proposed that a noncontact automatic identification method of vehicle load based on computer vision technology and deep learning algorithm is proposed, and 8624 vehicle image data sets are established and deep convolutional neural network training is carried out. Finally, transfer learning is combined with the general features extracted by ImageNet, and the recognition degree under the reinforcement learning strategy can reach up to 98.17%; in 2020, Song Yanfei and others proposed a three-dimensional model reconstruction method of the space grid structure using binocular stereo vision technology and image recognition technology, and passed the actual test of the grid model to verify the feasibility of the method [12].
With the development of software and hardware technologies such as parallel computing, cloud computing, big data, and deep learning, computer vision technology has been continuously improved, but there are still many technical challenges and application problems at this stage [13]. For example, the research of computer vision in structural health monitoring is still in its infancy. How to reduce errors caused by hardware factors, algorithm factors, environmental factors, etc. is an important research direction in the future. How to improve the application efficiency and reliability of computer vision is a follow-up research focus; in addition, computer vision has achieved good results in detecting whether construction workers wear safety helmets, but how to trigger the alarm system and human-machine coupling in later applications needs further research.
2.3. Speech Recognition
Speech recognition is a process in which the computer recognizes and understands the input speech signal and converts it into text output, so that the computer can have the “hearing” function like a human being.
In the building environment, voice recognition can be used for garage switches and voice password locks; in the home environment, voice recognition can be used for remote sensing of home appliances; in addition, voice recognition can also be used for keyword retrieval, number voice query, etc. In future application research, speech recognition can provide assistance for building intelligent installation, such as building route voice navigation and robot human–computer interaction, and can also provide assistance for the effective identification of life after disasters [14].
In the field of civil engineering, there are few related researches and applications of speech recognition at present, and the research difficulties mainly focus on noise processing, robustness, and speech model. First, various noises often appear when inputting speech signals, and improving the noise processing is an important part of improving the accuracy of speech signal recognition; second, the existing speech signal recognition systems generally relies on the environment. High, different environments will lead to large differences in the recognition accuracy of speech signals, and enhancing the robustness of the speech recognition system will help to achieve the practical application of the system; finally, during speech interaction, semantics, speech rate, and emotion will all be affected. It affects the real meaning of speech, so the optimization of the speech model is also a difficult research point.
2.4. Cross-Domain
Interdisciplinary fields refer to numerous interdisciplinary subject groups, reflecting the trend of scientific research toward comprehensive development, with high complexity, breadth, and diversity. The intersection of artificial intelligence and civil engineering can greatly improve the engineering quality and work efficiency of infrastructure projects.
In 2015, Tang Hesheng et al. established a model for predicting the yield strength of rectangular concrete columns based on artificial neural networks, analyzed the key factors affecting the yield performance of concrete columns, and used Garson sensitivity analysis to prove the rationality of the model; in 2019, Ding Yang proposed that taking the process of mass concrete pouring as an example, a prediction model for the internal temperature of concrete hydration and heat release was established to provide a basis for monitoring, prediction, and early warning of subsequent maintenance. A fire monitoring method based on YOLO-BP neural network is proposed. The accuracy rate of using this method to monitor the fire in the repair stage of ancient buildings is 93.9%. In 2021, Zhao Yannan et al. proposed a tree structure intelligent form-finding based on BP neural network. The method can be used to intelligently locate the lower-level hierarchical nodes, so as to realize the intelligent form-finding of the overall geometric shape of the tree structure [15].
With the deep integration of industrialization, informatization, and intelligence, the traditional civil engineering industry is facing profound changes. The key to promoting the intelligent development of the entire life cycle of civil engineering is to comprehensively carry out the technology research and development and practice of intelligent design, intelligent construction, and intelligent maintenance, and strengthen the construction of the interdisciplinary system of artificial intelligence and civil engineering. In addition, in the construction of the interdisciplinary system of artificial intelligence and civil engineering, civil engineering should be adhered to as the main body, artificial intelligence as the auxiliary, and artificial intelligence technology should be used to support and promote the intelligent development of civil engineering throughout the life cycle.
3. An Intelligent Computing Framework for the Whole-Process Response of Civil Engineering Structures Based on Deep Learning
3.1. Structural Static Characteristics Unified Data Interface Mode
The structure is complex and diverse. In order to establish an end-to-end general computing framework, it is first necessary to solve the problem of how to organize multiple types of structures in an orderly manner, so as to facilitate the unified processing of subsequent deep learning programs. In order to meet the requirement of fidelity structure construction of original information, this study introduces the concept of a feature module, which can decompose, organize, and classify the static features of the structure by referring to the assembly idea of “from part to whole” of fine finite element technology [16].
A feature module corresponds to a certain structural property of a structure and contains many aspects of the structure. Taking the steel plate shear wall structure as an example, an embedded steel plate feature module can be set, and the feature information such as the width, height, and thickness of the steel plate can be recorded in the module. In order to consider the repeatability of construction, define two types of static feature modules: variable-length static feature model and fixed-length static feature model, and specify that any feature module is only one of the two [17]. For example, the opening feature of the steel plate embedded in the steel plate shear wall structure is a typical variable-length static feature module, because one or more openings may be set, and each hole can have its own geometric information. Frame top beams are usually a fixed-length static feature module because there is usually only one top beam. Further, the sub-features in the feature module are divided into dense features and sparse features: dense features mean that the feature value type is a continuous real number (or integer), the value size is comparable, and algebraic operations can be performed. The sparse feature means that the feature value type is discrete, and it represents the category or dummy feature. For example, whether the steel plate shear wall structure has out-of-plane constraints can be 0 to indicate no, 1 to indicate yes; the shape feature under the opening feature module Use 0 for a circle, 1 for a square, and so on. These numbers are not comparable in size and have no operational meaning. In the deep learning model, they will be converted into one-hot representation to avoid the ambiguity of discrete features under the Euclidean space distance metric, and ensure that the distances between discrete features are the same and equal [18].
3.2. PADCN Model
According to the definition of variable-length and fixed-length feature modules, combined with the characteristics of actual structural calculation and analysis, two functions that need to be realized by the structural static feature learning model can be summarized: (1) the internal sub-feature sequence of the variable-length feature module can be reasonably integrated; (2) the coupling relationship between each feature module and each other can be mined. Because of the influence of various factors such as material and geometric nonlinearity, each structure usually does not satisfy the simple superposition principle. Therefore, structural static feature learning is a complex feature processing scenario: there are both sequence features and general fixed-length features, and there is an interactive relationship between the two [19].
The joint feature problem is rare in computer vision and natural language processing, the two main research fields of artificial intelligence, but it is very similar to the recommendation system scenario, one of the core businesses of the Internet industry. In order to recommend a product to a user, the following three categories of information are generally collected and processed: user portrait, product information, and user behavior sequence. Among them, user portraits and product information can be regarded as fixed-length feature modules, and user behavior sequences correspond to variable-length feature modules, and it is also necessary to mine the deep interaction between the user side, the product side, and the user behavior sequence. In this way, the deep learning model of the recommender system can be used to process the static features of the structure [20].
First, we consider the sub-feature sequence integration problem of variable-length feature modules. The purpose of the integration is to form a fixed-length vector, similar to the embedding vector in the recommender system, which contains the main information of the feature module. We adopt the standard multi-head attention mechanism to preprocess the sequence features, including the self-attention mechanism and cross-attention mechanism, as shown in Figure 2. Taking the steel plate shear wall stiffener feature module as an example, the self-attention mechanism can explore the interaction between the stiffeners, because the staggered stiffeners work together rather than independently to delay the buckling of the plate; the cross-attention mechanism is in order to introduce the interaction between the structures in advance, such as the intersection of the feature module of the embedded steel plate and the feature module of the stiffener, it can be expected to integrate the information of the plate into the stiffener expression in advance, which is more helpful for the subsequent extensive cross-learning. After the attention mechanism is implemented, the representation vector of the variable-length feature module is obtained using average pooling and dimensional transformation through a single-layer feed-forward neural network [21].
After the variable-length feature module completes the preprocessing, all feature modules are converted into feature vectors of certain dimensions, which facilitates the mining of coupling relationships. One of the mainstream deep learning models in the recommender system field is introduced into the deep and cross-network, which is connected with the attention mechanism preprocessing layer to form the PADCN model, which realizes the complete structural static feature representation learning, as shown in Figure 3. The structure of the DCN model is derived from thinking about the effectiveness of traditional recommendation system algorithms (such as SVD and FM): first, the memory of user habits, that is, a user must have fixed preferences for a long time. For example, if user A likes electronic products, then as long as the recommendation model can provide popular and latest products such as computers and mobile phones, the possibility of A’s positive feedback on the recommendation results is definitely not low; the second is the generalized exploration of user preferences. There are no related products in the browsing footprint, and it is also very likely that you like digital peripherals, such as game figures. In conclusion, a high-quality model needs to take into account the two attributes of memory and generalization at the same time. This is also applicable to structural response simulation: the hysteretic responses of structurally similar components are usually not much different, which belongs to memory; and if the model training set contains artificially developed central circular openings. For specimens with holes and central rectangular openings, after learning, it is expected that the model can simulate the mechanical response of a uniform matrix with multiple openings, which belongs to generalization [22].
The method of DCN to take both into consideration is to divide the model into two parts, as the name suggests, one is the cross-network part, which is used to memorize historical patterns. Finally, the two are spliced together for output [23]. The deep network part is a common feedforward neural network, which will not be described in detail. In the intersecting network part, the features themselves are constantly intersected, and then the higher-order interactions are mined; at the same time, the residual connection structure is used to ensure the complete transmission of the initial information.
Expanding and analyzing the above formula, it can be found that what the first layer learns is the polynomial of the input feature itself , and the polynomial contains all terms less than or equal to the order, and there is no information loss of any order [24]. In addition, the complexity of each layer of the cross-network part realizes the automatic upgrade of the coupling relationship, which effectively avoids the common combination explosion problem in pairwise cross-action learning.
So far, the unified data interface formulation and representation learning of structural static features have been completed. Then, by splicing the representation vector learned by the PADCN model with the input of the Mechformer model, an end-to-end structural intelligent computing framework based on deep learning can be formed, which comprehensively covers the data side and the model side, and can fully consider the static and dynamic characteristics of the structure, so as to realize the whole-process mechanical response prediction of different structures.
3.3. Numerical Test
This section will use the steel plate shear wall structure as the carrier to verify the effectiveness of the structural intelligence computing framework based on the PADCN-Mechformer model.
3.3.1. Data Preparation
The initial training data are mainly derived from experimental reports in historical documents and generated by means of high-precision and precise finite element technical parameter analysis. However, in practical applications, the initial data volume may not be able to support the training requirements of large-scale deep learning models due to limitations such as high experiment costs and low computational efficiency of finite element models [25]. Therefore, data augmentation algorithms need to be considered.
In the field of computer vision, operations such as rotating, cutting, scaling, and coloring the input image are performed to increase the amount of data. These methods are obviously not suitable for real-valued sequence fitting problems in structural calculations. With the richness of language meaning, the field of natural language processing can use synonym interchange, back translation (that is, translating a sample to another language with an existing model, and then translate it back), based on mask replacement, random deletion, and insertion. Methods such as language noise, in which back-translation can be analogized to the use of fine finite element models for data generation, and the addition of noise can provide inspiration for structural response data augmentation algorithms.
Noise addition methods based on random deletion and insertion cannot be directly applied, because unlike the flexibility and robustness of language (many sentences can be understood by humans even with a large number of typos), the structural response curve once the amplitude points are randomly deleted, etc. data, or randomly interpolating a point that deviates significantly from the loading trajectory, may cause a noticeable change in the shape of the hysteresis curve, impairing the model training process [26].
Corresponding to the random deletion method, a piecewise proportional downsampling algorithm based on amplitude points is proposed for the structural response curve. The flowchart is shown in Figure 4, and the schematic diagram of the results is given on the right side by taking Lubell’s SPSW2 test song as an example. The principle of the algorithm is that the test curve usually contains a large number of data points due to the high sampling frequency of the equipment, and the step size between two adjacent points is very small, so reasonable downsampling will not change the overall shape of the response curve and the key mechanics it reflects. At the same time, samples of different step lengths can be obtained to enhance the robustness of the deep learning model. The reason for the segmentation is that the hysteresis curve is highly nonlinear. Generally, the plastic segment is longer, the elastic–plastic transition segment is the second, and the elastic segment is the shortest. The plastic transition is the most prominent and is the key region to control the shape of the curve. If the entire half-ring is directly sampled uniformly, most of the sampling points fall in the plastic segment, and the elastoplastic transition segment is not fully described, resulting in serious curve distortion. Therefore, it is necessary to set the segmentation ratio, roughly frame the range of each segment, and give each segment a reasonable sampling ratio. After testing, the more suitable subsection ratio is {0.1, 0.15, 0.25, 0.5} (the elastic-plastic transition section is further divided into two parts: elastic transition and plastic transition), and the sampling ratio of each section is {0.05, 0.4, 0.25, 0.3 }. After the preliminary sampling is completed, it is recommended to refine the sampling curve. Some segments may have a small amount of data, repeated sampling occurs, and the data needs to be deduplicated. In addition, due to test measurements and other reasons, the maximum point of the displacement amplitude of the hysteretic half-ring of some test curves may not correspond to the point of the maximum load amplitude. The previously extracted displacement time history amplitude data and load time history amplitude data are corrected for relevant points.
Using the above algorithm flow, multiple new hysteresis response curves can be generated by setting different sampling points and trying to change the segmentation ratio and sampling ratio. The parametric analysis results show that for most of the response curves with more than 3000 data points, only 20% of the data points can reproduce the original curve with a high degree of coincidence. Therefore, the sub-scale downsampling algorithm based on amplitude points can generate a considerable number of new samples.
3.4. Simulation Accuracy Metrics
At present, the evaluation of the accuracy of the structural response curve simulation is usually based on the qualitative observation of the researchers, and most of the literature will claim to be “good fit.” In order to more reliably compare the accuracy of simulation results with experimental curves, it is necessary to establish relevant quantitative indicators. In structural performance analysis, mechanical indicators such as ultimate bearing capacity and overall (or average) energy consumption are generally used. Both of these two indicators have important physical meanings, but they also have certain limitations: the former is a single-point indicator, suitable for the design stage, and obviously not explanatory for the accuracy of the whole process simulation; the latter can be regarded as a certain sense. For the mean value index, since the number of hysteresis loops experienced by the simulation and the test is the same, the overall energy consumption is determined by the average energy consumption of each cycle. Therefore, this index lacks the control of the variance index and is also not comprehensive. For example, there may be a situation where the final average energy consumption of the hysteresis curve is similar because some segments consume too much energy and some segments consume less energy, but the simulation accuracy is actually low. Therefore, a more comprehensive metric needs to be explored.
3.5. Framework Verification
Through experimental data collection, fine finite element generation, and the expansion of a series of data augmentation algorithms in Section 3.1, the length of the loading regime sequence was controlled between 200 and 3000 data points. We divide 512 samples as a training set, 256 samples as a validation set, and 256 samples as a test set, of which test samples and their data augmentation in the training set account for 58.3%.
The deep learning-based structural whole-process response intelligent computing framework proposed in this study is universal in the whole process and is suitable for any structural computing and analysis tasks: as shown in Figure 5 (1) First, we establish a database according to the data interface mode specified in Section 1. This step can digitize the inherent properties of the target object “what you see is what you get,” almost without manual experience preprocessing; (2) the user specifies the reference value of each physical feature, and the program automatically adopts the reference value scaling method proposed by the author to carry out unified de-dimensioning and normalization; (3) the program uses the PADCN-Mechformer model for training and testing; (4) a deep learning model that can be used for actual structural response prediction is obtained, with high accuracy and far superior to traditional numerical methods computational efficiency.
4. Conclusion
This study proposes an end-to-end structural response intelligent computing framework based on deep learning, including structural static feature data interface mode, data augmentation algorithm, and core deep learning model. It is suitable for multi-level mechanical response prediction problems such as materials, components, and even structural systems.
Data Availability
The dataset can be accessed upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest.
Acknowledgments
The authors thank the Key Project of Natural Science Research in Anhui Province, Study on Water Restoration Technology of Microbial Filter Bed (No. KJ2016A389).