Abstract
The purpose of the study is to improve the structure restoration and design of ice sculptures. This study discusses the concept and artistic expression of ice sculpting based on the relevant theoretical knowledge of visual sensing technology and three-dimensional reconstruction technology. Based on this, the ice sculpture structure restoration and design model based on multivision sensors and three-dimensional reconstruction technology is constructed, and the model is compared with the manual test. The results show that the ice sculpture structure restoration and design model based on a multivision sensor and three-dimensional reconstruction technology is feasible and has high efficiency and stability compared with artificial ice sculpture restoration. In the experiment of structure reconstruction and restoration of an ice sculpture, the manual group 1 spends 237 minutes, the manual group 2 spends 252 minutes, and the time spent by the model group is very stable, which is about 139 minutes. Due to the difference in manual group personnel, their carving time is also different. As the carving continues, the possibility of errors made by the two manual groups increases. Therefore, ice sculpture structure restoration and design based on multivision sensors and three-dimensional reconstruction technology will provide a reference for the study in the field of ice sculpture structure restoration and design.
1. Introduction
In recent years, the art of ice and snow has developed rapidly in China. With the progress of the times, the ice sculptures that can only be seen in the northern region are introduced to warm areas. Over the years, people in Heilongjiang developed the art of ice sculpting in the season of ice and snow. The simple and easy-to-get ice cubes in Heilongjiang are no longer just used to store perishable food but also have become a tool for people to have fun. With the progress of the times, people’s pursuit of spiritual and cultural life is getting higher and higher, and this unique artistic activity has also developed rapidly. At the same time, this once popular entertainment in Harbin has also become international. The unique charm of the combination of craftsmanship and lighting art attracts many artists. Ice sculpting is no longer limited to its simple carving at the beginning but has developed into a new artistic behavior combined with a variety of artistic systems. Countless fine works of art appear on the ice show, and many scholars want to reproduce those hard-to-make-permanent works of art.
Rafi and Ekram [1] proposed a simple low complexity block tree coding, which reduces the overall memory and complexity of image coding, and its memory requirements and complexity are far less than the other types of coding algorithms. This makes the image encoder applicable in memory-constrained and real-time visual sensor networks [1]. Tai Jesse et al. successfully reconstructed the three-dimensional anatomical structure of the male pelvic floor through three-dimensional reconstruction technology, including the bladder, the urethral sponge, prostate, and bone [2]. Bharath and Hote believe that three-dimensional reconstruction is helpful for the planning and prognosis of all length enhancements of the pulmonary vein and provide the three-dimensional reconstruction of pulmonary vein angiography images of children in March, showing the structure of primary pulmonary vein stenosis [3]. Ivan et al. [4] pointed out that digital reproduction of works of art belonging to cultural heritage is one of the innovations in the field of exhibition and introduced mobile devices based on visualization. After the processing of the image geometry is completed, a more accurate three-dimensional model is generated, and similar exhibition cases are studied [4]. Yang et al. proposed a three-dimensional reconstruction algorithm based on the single pipe weld ray image, which can accurately and effectively reconstruct the weld three-dimensional based on a single two-dimensional pipeline weld ray image, and realize the accurate identification of weld nodules at the root of the pipeline [5]. Zhang et al. argued that three-dimensional reconstruction technology is conducive to a better understanding of important anatomical structures, and it is applied to the misdiagnosed Morgagni’s hernia (MH) patient, and the disease is successfully detected [6]. Wu et al. proposed a bridge vibration measurement method based on the three-dimensional reconstruction, which can effectively overcome the limitation that fixed reference points and target points must be coplanar in homography correction [7]. Hu et al. [8] designed a prediction model for brain image processing and brain disease diagnosis. Under the premise of ensuring energy consumption, the model has higher accuracy than other models, a more obvious denoising effect, and the best segmentation recognition effect, which provides the experimental basis for feature recognition and prediction diagnosis of brain images [8]. Liu et al. described the latest method for screening atrial fibrillation from single-lead ECG waves and discussed the potential factors that may contribute to screening atrial fibrillation from single-lead ECG waves [9]. Wan et al. constructed a brain image fusion digital twin diagnosis and prediction model based on a semisupervised support vector machine and improved AlexNet, which can provide high precision, good acceleration efficiency, and good segmentation and recognition performance with low errors, and provide the experimental basis for brain image feature recognition and digital diagnosis [10].
The above literature shows that the three-dimensional reconstruction technology is extremely suitable for the reconstruction and restoration of the structure, but there is no relevant research on the three-dimensional reconstruction technology for the structural restoration and design of ice sculptures. Since the nature of the ice sculpture is not easy to preserve, the concept and artistic expression of ice sculpting are discussed based on the relevant theories of visual sensing technology and three-dimensional reconstruction technology. On this basis, the restoration and design model of ice sculpture structure based on multivision sensors and three-dimensional reconstruction technology is constructed and compared with the restoration and design of the ice sculpture structure by artificial experiment. The three-dimensional reconstruction technology is innovatively applied to the structure and reconstruction of ice sculptures, and the application fields of visual sensing technology and three-dimensional reconstruction technology are expanded, providing a reference for the research in the field of structural recovery and design of ice sculptures and making beneficial contributions to the rapid development of ice sculpting.
2. Visual Sensing, 3D Reconstruction, and Ice Carving
2.1. Visual Sensing Technology
Visual sensing technology can solve many problems in many fields. Compared with biological visual systems, visual sensing technology can be applied to harsh environments where many organisms cannot adapt. Its advantages of fast and accurate intuitive results make it brilliant in many fields. With the rapid development of various sensing technologies, its important role in various fields is gradually reflected. The current visual sensing technology mainly includes the following contents [11].
Laser scanning imaging technology can estimate the distance between objects by measuring the time required for the emitted laser to return to the sensor after the object is reached. By the oscillating mechanism used, two-dimensional and three-dimensional scans of the surrounding environment can be realized by scanning in multiple directions. Compared with other sensing technologies, laser scanning imaging technology has a wide range of measurements, which can be applied in mild indoor environments and outdoor environments with more unstable factors. And laser imaging technology has small measurement errors and a high sampling rate, which makes it useful in a dynamic imaging environment. But at the same time, laser scanning imaging technology also has many limitations. For example, haze, rainfall, and other weather conditions will affect the accuracy of its scanning. In addition, due to the characteristics of the laser, the power consumption of the scanning imaging technology based on it should not be underestimated, and the cost of its equipment is high. These factors cause certain restrictions on the wide range use of laser scanning imaging technology. At present, the application of laser scanning imaging technology is embodied in feature extraction, and points or lines are extracted point by point. In the process, there are two methods, namely, the use of image processing means to deal with the data points in the two-dimensional image or dealing with the data points in the space; semantic annotation and segmentation are needed to accurately segment points to each specific entity to identify specific entities in space. This process takes a lot of time with human beings, so the workload is reduced and the efficiency of semantic segmentation is achieved by combining with deep learning technologies, such as the recurrent neural network and the convolutional neural network, to achieve the efficiency of semantic segmentation. The entity is monitored to accurately locate in three dimensions, and the contour of the entity is described from the data point of the entity. This technology can be applied to the front-end AI field to promote the development of intelligence. In the current situation, mobile laser scanning technology is widely used in building modeling and other fields. It plays an irreplaceable role in the three-dimensional modeling of various urban environments [12, 13].
In the large dynamic range optical imaging, the so-called large dynamic imaging system generally includes image information acquisition, information processing, information storage, and image display. The dynamic range above 80 dB enables it to obtain more complex images. Large dynamic range optical imaging can be achieved by modulating the incident light with various methods based on the detector and processing the obtained data. Similar to laser scanning technology, large dynamic range optical imaging technology can also be combined with deep learning technology to make the processing process more intelligent and convenient. In addition, large dynamic range optical imaging can also be used for image fusion and tone mapping. As the core of large dynamic range optical imaging, image reconstruction includes image fusion, reconstruction, and tone mapping. It is noteworthy that deep learning technology can also be applied to image fusion and tone mapping. At present, large dynamic range optical imaging is applied to multiple scenes, such as smartphones, cameras, and microscopes, which play an important role in cutting-edge fields like biology and aerospace [14].
The traditional imaging technology revolves around the radiation intensity and spectrum, and the recognition ability of image information is limited. As a new optical imaging technology, polarization imaging and sensing technology revolve around the polarization information of radiation. In polarization imaging and sensing technology, the relative depth of the entity can be restored utilizing polarization and shadow constraints. This method gets rid of the limitations of traditional reconstruction methods such as light sources and noise. It can well describe the shape, material, and other characteristics of the detected object and has a more accurate recognition ability. Therefore, polarization imaging and sensing technology are applied to biology, showing good adaptability [15].
Ocean acoustic tomography is the use of acoustic wave diffusion phase and amplitude of the relationship and through tomography to achieve image generation. Through the layered imaging of an object, marine acoustic tomography can measure the seawater flow and seawater temperature in a large area for a long time and predict it to a certain extent to complete the monitoring of marine environmental changes. On this basis, ocean acoustic tomography can be used to observe the problem of global warming. It can successfully observe the change of ocean current temperature by monitoring the trend of ocean current movement and its temperature in a large basin, and the rate of global warming can be obtained by comparing the data. Compared with in situ measurement, marine acoustic tomography can collect and aggregate information in large-scale ocean space, and its monitoring time is longer. In addition, marine acoustic tomography also has the advantages of all visual sensing technologies, which can be used in special environments where organisms cannot adapt [16].
2.2. 3D Reconstruction Technology
At present, 3D modeling is applied to various fields such as architecture and medicine and plays an important role. The 3D modeling combined with multivision is developed in this case [17]. The following will discuss the three-dimensional modeling of multivision.
2.2.1. Camera Model
The camera model is constructed based on several coordinate systems, as shown in Figure 1.
Through the photosensitive chip, the camera can capture the light signal of the entity, and the corresponding image can be converted to the digital format by processing. Each pixel is converted to its corresponding element and stored in the matrix. The principle of camera imaging is shown in Figure 2. After the entities in the real world are projected to the two-dimensional plane, the corresponding image information is obtained.
This perspective projection model can simply explain how the camera projects the entity in the real world to a two-dimensional plane, convert the physical coordinate position of the entity to the coordinate point in the camera coordinate system, and then transform the point in the camera coordinate system to the coordinate point in the plane. After the size and position are adjusted, the position point in the image coordinate system is obtained, completing the imaging.
2.2.2. The Multiview Geometry
In the study of multiple images captured from multiple perspectives, epipolar geometry is introduced, which is a theory to describe the relationship between multiple images from multiple perspectives. Through this theory, the position angle of the camera can be accurately found out, and then, the midpoint of the coordinate system is estimated, which is an important means for reconstruction technology [18–21]. The relationship between the two views is shown in Figure 3.
The view is that the camera projects the same point at two different angles. The polar plane of the two cameras intersects with the phase plane, and each polar line is the image projection displayed on the other camera when the camera projects the object. In the geometric projection of the two views, it is necessary to note that one point on the obtained image corresponds to a polar line on the other image, and if the multipoint is in the same plane, there may be a corresponding relationship to map it to another view.
Here, select corresponding points to complete the matching in the two planes as and and obtain the following equation.
Here, , , and and are the abscissa and ordinate, respectively; according to the projection relationship, equation (2) can be obtained. is the camera internal parameter matrix, and are the camera motions, and are parameters, and and are planes. The least square method is used for optimization, as shown in the following equation. is the camera internal parameter matrix, and are the camera motions, and are parameters, and are planes, and is a parameter. The observation equation given here is shown in the following equation. is the coordinate point of this projection position, is the pose, and is the road sign. The observed error is shown in the following equation. is the error, is the coordinate point of this projection position, suggests the result of quadratic projection, means the projection function, and indicates a 3D point. Then, the optimization function can be obtained as follows. is the pose, is a 3D point, and are parameters, is the generated data, is the error, means the projection function, , and .
2.2.3. Camera Calibration
For the commonly used camera, its parameters are usually fixed and will not change in a long time. If these internal or external parameters can be calibrated in advance, its calculation process can be reduced a lot. The camera calibration can be divided into two categories according to whether the reference is used or not. The first category is a more traditional calibration method, which is used to detect the landmarks on the plane for calculating the camera parameters. Different calibration results can be obtained according to the different calibration tools used, which depends on the accuracy of the calibration tool [22]. The commonly used calibration tool is similar to the chessboard, and its calibration structure is shown in Figure 4.
Through this chessboard calibration tool, the camera is used to shoot it from different angles, and then, the image is processed to obtain the coordinate value at the chessboard intersection. Under the assumption that the image is normal, the internal and external parameters are calculated. Then, the least square method is used to calculate the real distortion, and the camera calibration is completed. The other is the self-calibration of the camera. This kind of calibration does not require the use of calibration tools, and the internal and external parameters of the camera are automatically obtained by analyzing and calculating similar points between images at different angles. Due to the lack of calibration tools, the accuracy of this calibration method is poor. But the self-calibration of the camera can complete the calibration without tools because of its characteristics, and it has high flexibility and is not restricted by the use environment. The specific self-calibration method can be divided into the vanishing point, the line constraint method, the solving equation method, and the hierarchical method. The method of the vanishing point and line constraint has the characteristics of easily deducing the equation. At the same time, due to its harsh use restrictions, there may be some errors in the parameters calculated. The solving equation method has a relatively perfect basic theory, but its use is restricted because of its large numerical fluctuation and its being prone to algorithm failure. The hierarchical method can calculate the initial value of some algorithms, but it may be unable to converge due to the instability of the value [23].
2.2.4. Salient Object Detection
For organisms, it is almost instinctive to notice the more visible areas of vision. They can get more information by more detailed analysis of these obvious areas. How to make computer vision have similar capabilities and use it to explore the characteristics of the region in the image to solve some complex visual problems is the focus of relevant scientific researchers. With the development of deep learning, this problem is preliminarily solved. From the beginning of artificial features to the independent processing of deep learning networks, the processing model based on a deep learning network can extract more specific information from images and capture specific entities in images more accurately. Therefore, this image processing model based on a deep learning network is becoming an important part of 3D reconstruction technology. For example, the excellent ability of image segmentation and generation of the convolutional neural network makes it greatly developed in this field. The typical convolutional neural network can be divided into the convolution layer, the pooling layer, and the fully connected layer. The convolution layer extracts the feature points in the image, the pooling layer reduces the parameters by an order of magnitude, and the corresponding results are outputted from the fully connected layer. Its specific structure is shown in Figure 5.
For a long time, the convolutional neural network is one of the cores in the field of image recognition. Its stable learning ability enables it to have good applications in object recognition, behavior cognition, attitude estimation, neural style conversion, and natural language processing [24].
2.3. Research on Ice Carving
Ice carving, as the name implies, is a kind of behavior of carving with ice as the raw material. The sculpture of ice can be said to be “crisis-ridden.” If one is not careful, the sculpture may collapse at any time. As a kind of sculpture art, the production of the ice sculpture focuses on the technical level and artistic and cultural literacy of the sculptor. The material of ice itself is almost transparent. Although this feature makes it difficult to show its three-dimensional image, it is because of this transparent material that the ice carving can show its exquisite and penetrating artistic beauty [25–27]. Ice sculpture works usually focus on the light problem of the place to fully reflect this feature, as shown in Figure 6.
Under the colorful light mapping, the beauty of ice sculpture works is reflected incisively and vividly. Nowadays, the art of ice sculpting is developing rapidly. In addition to Harbin, many cities in warm regions have built cold storages for the development of ice sculpture culture for the storage and exhibition of ice sculptures. This shows that the ice sculpture culture breaks through the environmental temperature limit and is developing rapidly. Generally, as a kind of sculpture art, the type of ice sculpture is also the same as that of an ordinary sculpture, which is divided into relief, through sculpture and other forms. Two different types of ice sculpture (floating and round sculpture) are shown in Figure 7.
Due to the transparent material of ice, it is the most suitable for carving the sculpture whose beauty is reflected from the inside. As a commonly used means in traditional carving, the advantages of ice sculpting are the obvious spatial sense, the flexible artistic expression, the less required materials, and the relatively light carving weight. When the ice is carved, special attention is paid to its ice thickness, carving accuracy, and so on. In addition, perfect ice sculpture works should be equipped with the corresponding lighting, so that the artistic beauty of the ice sculpture is fully reflected. For the ice sculpture, the shape is the first link for the ice sculpture to be reflected in the eyes of the viewer. It needs to have a beautiful outline, a harmonious overall structure, and an obvious artistic expression. Attention also needs to be paid to the rules of the line for a good ice sculpture, and they can be soft stretch or peak turn, just like a piece of music, reflecting the high melody of the ice sculpture. Besides, the production of the ice sculpture also needs to be combined with its place. The position, distance, and height of the viewer are all factors that need to be considered. On this basis, the aesthetic feeling of the ice sculpture is reflected. At the same time, art and culture rely on the support of communication. Human civilization is the product of cultural communication, and this communication is the unique cultural heritage of mankind, which is the carrier of the formation of human society, contributing to the cultural exchange between mankind. It has great significance for the development of the country and the nation. It is this communication that links Harbin and ice sculpting together. It shows the customs of Harbin and reflects the historical and cultural connotations of the whole northern region. This cultural connotation led to the local economic development, tourism, and other industries being based on this. The spread of ice sculpture culture makes the image of Harbin more specific around China and brings great international influence [28]. As an open and inclusive Chinese city, Harbin has rich cultural conservation and advanced and diversified ideas, which are spread worldwide under the influence of ice sculpting.
2.4. Modeling of Ice Carving Structure Restoration and Design
The design and restoration of ice sculptures are based on three dimensions. The beauty of ice sculpture can be viewed from different angles such as front, side, and back, and each angle should be carefully carved. Ice sculpture develops with the development of modern science and technology, which is the driving force. The multivisual sensors and three-dimensional reconstruction techniques are used to design and restore the ice sculpture structure. The specific model is shown in Figure 8.
First, the camera image is determined, the camera is positioned, and the parameters of the image camera are reconstructed. On this basis, the feature points are matched, and the relationship between these feature points is established. The feature points, namely, the image segmentation, are completed by CNN (Convolution Neural Network). Second, the surface reconstruction of the corresponding contour obtained by observing the ice sculpture from different perspectives and the vertebral structure formed by the projection center is carried out. Finally, the mapping and texture improvement is conducted. Based on the above, the camera position should be calculated if the image is taken, and some points with obvious characteristics are selected for matching. Then, the connection between these points is established, and all the points in a graph can correspond to the points in another graph. The parameters of a camera are reconstructed by triangulation, as shown in Figure 9.
After the basic parameters of the camera are obtained, the error caused by its operation process is shown in Figure 10. The obtained parameters need to be optimized to eliminate the error caused by the optimization. This error reflects the difference between the projection points of a three-dimensional entity on a two-dimensional image and the projection points.
The entity segmentation is needed before the reconstruction of the ice sculpture model. The traditional manual annotation method is not used here because of its low efficiency. Therefore, the concept of deep learning is introduced to complete image segmentation by the convolutional neural network, and the three-dimensional geometry can be constructed through the contour of the ice sculpture. First, the image is segmented after the corresponding contour lines are obtained when ice sculptures are observed from different perspectives. These contours form a vertebral structure with the center of the projection, as shown in Figure 11.
Thus, the visual shell of the entity can be reconstructed, and the three-dimensional reconstruction of the ice sculpture can be realized by combining the relevant camera parameters. Since the ice sculpture is a closed geometry with a smooth surface, the light projected onto the contour curve formed by the image constitutes the shell of the entity to ensure that the projection image of the shell is consistent with that of the entity. The visible line on the contour of the object with different angles is obtained by mapping projection, and then, the complete shell generated by the intersection of the vertebral structure is used to restore the boundary, obtaining the complete polygon model of the ice sculpture entity [29].
2.5. Test and Results of Ice Carving Structure Restoration and Design Model
2.5.1. Test Results of the Model
The established model is tested to ensure its feasibility, and the restoration test of the same ice sculpture is taken. The control groups adopt manual restoration. The skilled ice sculptors are selected for ice sculpture restoration to avoid the influence of manual errors and other factors. After that, the completion time of each group is counted to evaluate the feasibility of the model. The results of ice sculpture restoration between the established model and the manual operation are shown in Table 1.
The specific proportion of time consumption between the established model and manual group 1 is shown in Figure 12.
The proportion of time consumption between the established model and manual group 2 is shown in Figure 13.
The above figures show that the model is efficient for the restoration of ice sculptures. Compared with the manual groups of about 4 hours, the model only needs half of the time spent by the manual groups to complete the restoration. Besides, manual group 1 spends 237 minutes, manual group 2 spends 252 minutes, and the model group spends 139 minutes, which is relatively stable. Because of different personnel in the manual groups, the carving time is also different. With the progress of carving, the possibility of errors in manual carving is gradually increasing. Therefore, the model of ice sculpture structure restoration and design based on multivision sensors and three-dimensional reconstruction technology in this study is feasible. It can greatly shorten the recovery time of ice sculpture and provide a high quality ice sculpture. The restoration of ice sculptures will replace the manual structural restoration of ice sculptures on many occasions.
2.6. Actual Restoration Results of the Model
Based on the above ice sculpture structure design and restoration model, the ice sculpture structure is restored and designed, as shown in Figure 14.
The figure shows that this model is feasible for the structural restoration and design of ice sculptures. Compared with the manual restoration, the ice sculpture structure restoration and design model based on multivisual sensors and three-dimensional reconstruction technology can design and restore the existing ice sculpture entities accurately, and its recovery rate and quality have great advantages. Although its innovation and refinement are poorer compared with manual restoration, its efficient and stable characteristics are still worthy of attention.
3. Conclusion
At present, the research on three-dimensional reconstruction technology is becoming more and more popular, and its powerful structural reconstruction function makes it applicable to medical, biological, and other fields. However, the application of related technologies in the field of structural restoration and reconstruction of ice sculptures is relatively rare. Since an ice sculpture is not easy to store, the ice sculpture structure restoration and design based on multivisual sensors and three-dimensional reconstruction technology are studied. The results show that the constructed ice sculpture structure restoration and design model based on multivisual sensors and three-dimensional reconstruction technology in this study is feasible. Compared with manual restoration, the ice sculpture design and restoration by the proposed model are more efficient and stable. In addition, the restoration and design model of the structure of ice sculpture based on multivision sensors and three-dimensional reconstruction technology is not limited by time and space and can be used for structure restoration and design of ice sculpture under harsh conditions. And it avoids the damage to the ice sculpture structure caused by the possible errors and emergencies of manual carving. The shortcoming of this study is that the actual test of the model does not evaluate the exquisiteness of the model and the manual restoration of ice sculpture, and a complete evaluation system will be established in the future, which will be the future research direction in the follow-up study. The ice sculpture structure restoration and design model based on multivisual sensors and three-dimensional reconstruction technology can quickly and efficiently complete the design and restoration task of ice sculptures, which has a certain contribution to the reproduction of ice sculpture art. In addition, it also enlarges the application field of three-dimensional reconstruction technology and multivisual sensing technology.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare no conflicts of existing interest.
Acknowledgments
This project is the 13th Five Year Plan Project of Humanities and Social Sciences Research of the Education Department of Jilin Province “Development and Application of Digital Ice and Snow Sculpture Equipment in the Field of Ice and Snow Sculpture” (No. JJKH20211374SK).