Abstract
This study proposes a cyber-physical interconnection method for computational design and robotic construction in a wooden architectural realm. It aims to provide a highly efficient, flexible, and adaptive design-construction approach by continuously updating digital models and physical operations according to the locally sourced materials. A perception-modeling system to scan the source materials on-site and send their data simultaneously to design software was developed by using physical sensors and computational technologies in an innovative manner. The data was used for architectural programs to generate design outcomes and guide the robotic construction. The novelty of this study is to establish a real-time, bidirectional interaction mechanism between digital design and physical construction. The design outcome is no longer a fixed, predefined geometric model but a dynamic, data-driven model which would be updated by material conditions on-site. The construction robot is able to make synchronous adjustment automatically in coordination with the dynamic design. The success of the iterative perceiving-simulating-updating loop was demonstrated by building two pavilions.
1. Introduction
Robotic fabrication in the architectural field experienced a big leap since the beginning of the 21st century. Thanks to the expansion of the open-source program ecosystem and the development of programmable robotic machines (Kuka [1], ABB [2], etc.), an increasing number of digital fabrication projects are being conducted in which the designers are fully involved in the design-construction process from the beginning until completion. In most of these projects, digitalization technologies enable the design ideas to develop virtually by analyzing and managing a large quantity of data through different types of simulations.
In 2015, Menges introduced the concept of cyber-physical construction in architecture, which involves construction processes that intensively integrate the realm of physical production with the virtual domain of computation [3]. This concept helps robotic fabrication to become more intelligent, as various levels of predictions and new operation tests for a complex system are simulated before the system is applied in the real world [4].
In order to further develop this method, this research proposes an on-site robotic construction workflow that is based on real-time physical sensing and computational analysis instead of relying on explicit modeling and inalterable instructions. The construction environment of the architecture using local source materials is complex and dynamic. Natural materials vary in sizes and shapes; hence, they are difficult to predict and to be suitably used. Our on-site robotic construction system is endowed with the perception of its current environment and conditions of source materials. With the feedback information collected by the robot, the virtual plan is updated dynamically and physical robotic actuation is adjusted in coordination with the virtual plan in reverse. This research proposes another design-construction workflow; it is a recursive, informational data-driven paradigm, instead of a static, closed top-down decision-making process. It helps on-site robotic construction to remain optimal in terms of adaptiveness and feasibility.
In order to clarify the subject of discussion, this article aims to achieve the following objectives:(1)A review of robotic fabrication and digital twin method applications(2)An extension and realization of the perception-modeling system with sensing and transfer technologies so that the system is able to capture and store source materials’ geometry in advance, assemble the components in Rhino/Grasshopper, and instruct on-site construction with real-time site information(3)A redefinition of the novel design-construction workflow and data management, with a robust and integrated informational interaction of human-robot collaboration based on a digital twin model.
2. Background and Related Works
It was not until recently that the multifunctional industrial robots had their own test scenarios in the field of on-site architectural construction. In order to obtain remarkably efficient and accurate but automated fabrication output from robots, either of these experimental approaches tends to emerge as a series of parallel research fields to realize the novel architectural design-construction discipline, and, at the same time, a brand new concept of managing source data and operations over various fields as well as different phases in the design-construction workflow is raised.
2.1. Robotic Fabrication for Wooden Architecture
The application of digital technologies in wooden construction, including early joinery customization and the CNC process, started in the 1980s [5]. In less than 40 years, Mjøsa Tower, the tallest prefabricated timber frame high-rise building in the world, has been constructed with a maximum height of 85.4 m, demonstrating the potential of wood as one of the most sustainable universal construction materials. Due to the flexibility of its materialization nature, wood has also become one of the preferred robotic fabrication materials and has been used in more than 80% of the most projects in the current research field [6]. In earlier studies, two main aspects of robotic wood fabrication research were reported. Fabio et al. completed a series of large-scale robotic fabrication projects [5, 7], in which the wooden structural frame of the house was processed and assembled on-site. Alvarez et al. [8] and Schwinn et al. [9] from ICD University of Stuttgart introduced an integrative approach to build a sophisticated timber pavilion. However, both researches used processed timber components which is delicately produced and organized, and passively accepted errors occurred and accumulated during both fabrication and assembling processes. On the other hand, research studies also shed light on using unprocessed wood as the local components in a novel way, which may immensely improve the freedom and effectiveness of the system. Monier et al. developed a method for optimizing the use of native irregular shape wood pieces [10], although the method is still in the digital model phase. Schindler et al. worked on the potential of 3D scanning and the postprocessing of wooden branches by using CNC fabrication [11]. Obuchi et al. and their research pavilions [12, 13] from T-ADS in the University of Tokyo explored a series of raw material assembly logic by accumulating regular and irregular local components, pushing the limit of raw materials as proper constructible elements, and using the latest digitization technology and a sophisticated working process. Self et al. and their Wood Chip Barn minimized the preprocessing of the wooden elements, replacing wooden trunks with the pretreatment of the joints. The trunks were later 3D-scanned, all geometric information was collected to generate the final form, and part of the main structure was assembled beforehand and then transported to the site [6]. Nevertheless, for all of these unprocessed wooden construction researches, neither a fully integrated procedure from design to construction phases nor a synthesized data flow management method can be found.
2.2. From Human-Robot Collaboration to Perception-Modeling Fabrication
The component assembly task is considered to be of great significance in on-site construction. The main challenges of this task are handling complicated geometries and obtaining high production flexibility [14], which in turn reflects in the proportion of human and robot involvement in this process. With the ultimate objective of achieving a construction automaton, to indicate a clear boundary between human and robot is inevitable, which increases an additional layer to the system complexity. Research studies have been conducted on various methodologies and frameworks for human-robot collaboration, realizing a highly flexible and adaptable construction system. Almeida and Sousa operated a robotic arm to cut timber components and then assembled the pavilion with human effort [15]. Larsson et al. proposed a human-in-the-loop fabrication system for small-scale construction by using irregular wooden branches, a leverage digital scanning technology, and a 2.5 D CNC machine [16, 17]. It can be concluded that the division of labor between robot and human is relatively clear in the above cases; yet human involvement increases alongside the geometrical complexity, thereby resulting in a reduction in the effectiveness. By reconsidering such an increasing number of scenarios, the cases of implementing robotic perception-modeling fabrication intend to realize human involvement at the beginning of each informational input loop. Decision taken by the process of feeding physical environmental perception to the robot becomes possible with the support of the latest edge-computing technologies. Brugnaro et al. inspect the robotic fabrication of the woven bamboo structure in their robotic softness research, in which the real-time information is continuously shared with the decision maker [18]. Wu and Kilian conducted a research study on setting up a vision system to suggest assembly positions of the desired topological connections through real-time processed image information [19]. Jeffers discussed a method that a virtual model is built for persistent storage of material data and next built action can be computed accordingly [20].
In these cases, the robotic fabrication process includes interference either from the IoT devices or directly from humans to achieve a more adaptable system attributed by the real-time informational update and adjustment, avoiding the gradually accumulated errors, while a master management method about the workflow has not yet been summarized or proposed. In order to develop this concept further, the distribution and management of the appropriate human-robot relationship for the whole lifecycle should be addressed prior to the description of the construction.
2.3. The Digital Twin Paradigm
Digital twin (DT), as a novel paradigm for realizing interaction and integration between the physical space and the virtual space, has been developed to manage complicated systems for the relevant academic circles and enterprises [21]. This method is considered to be able to provide an integral and material-behavior-driven construction paradigm for complex conditions in robotic fabrication process [22], aimed at bringing geometric and detailed design, structural simulation, 3D scanning, and on-site construction together.
The concept of the digital twin was first introduced by Grieves, who stated it as the “Conceptual ideal for product lifecycle management (PLM)” at the University of Michigan in 2002 [4], defined as “a set of virtual information” that describes the physical object of different scales. Due to the rapid development of IoT devices, the cloud-computing technology, and the machine learning algorithm, a multifunctional robotic arm is conferred with the capacity of retaining its digital twin, which receives real-time information through sensors, has access to the collected big data cloud, and triggers its physical twin to take actions after analyzing various design disciplines in the virtual world. This method was tested in the early space capsule prototype [23] and can be continuously developed in the manufacturing realm. In 1991, an idea called “simulation control” was proposed, which outlines a prototype system in which simulation is integrated into a CIM environment for manufacturing [24]. Thereafter, the concept and exploration of DT have seen rapid progress thanks to the advances in technology. The digital-twin-driven approach includes the following three parts: the physical layer, digital data exchange, and the virtual layer. The physical layer refers to the real production environment. The virtual layer refers to the digital model of the physical resources for carrying out both the manufacturing simulation and the real-time optimization of the systems. Digital data exchange refers to the data interaction between the physical layer and the virtual layer; it collects the real-time data from the physical layer and the feedback from the virtual layer and vice versa. Nowadays, the focus areas of digital twin applications are different from the product lifecycle, layout planning, process design, maintenance, PPC, etc. [25], and the inclusion of the digital twin model optimizes multiple aspects. Boschert and Rosen illustrated that the digital twin uses essential information to bridge over creation, production, operation, and disposal phases, thus resulting in a system available for succeeding in the phases in which almost everything within the system, including material geometry, structural performance, and raw data from fabrication, can be monitored and tracked [21]. Malik and Bilberg proposed a digital-twin-framework-based model that supports the design, development, and operation of human-robot collaboration in the manufacturing practice, illustrating the use of digital twin in the human-robot task allocation and the workplace setup [14]. Harper et al. in ABB focused on the development of digital twin standards in which the on-site virtual support can be achieved [26].
Taking advantage of the digital twin model, the negotiation and blurry boundary between the virtual and the physical space are erased, thus leaving a fully integrated cyber-physical construction environment for a real-time information feedback loop. The vision of the digital twin application is always emphasized; yet practical use is seldom tested or realized, especially for those robotic fabrication researches in architectural field.
3. Methodology
This research presents a digital twin framework (Figure 1) of the overall design-construction process, ensuring a smooth interconnected informational exchange back and forth linking the virtual and physical space. The framework consists of four phases: in the preparation phase, unprocessed wood branches are selected as source materials; the silhouette is selected and developed as the measuring method. In the material perception-modeling phase, the parameters and mathematical formulation of the real-time image acquisition and shape reconstruction system are calculated; corresponding programs are developed. In the dynamic design phase, material organization and the corresponding overall geometries are evolved using matching algorithm and positioning algorithm. In the flexible construction phase, robotic assembly is achieved using the executing program, monitoring program, and adjusting program. The digital twin framework enables up-to-date material information of the physical space to be reflected in the digital model with a high level of precision and synchronization, and the dynamic modifications of the digital model also provide an almost real-time change in the robotic assembly.
3.1. Preparation Phase
3.1.1. Material Selection
Wood is considered to be a low-carbon, environmentally friendly material, and its unique texture is aesthetically favorable for architectural design. Straight tree trunks are widely used as they can be cut into standardized pieces for the building industry. However, small tree trunks and branches with diversified shapes are typically abandoned as they are difficult to use systematically due to their variability.
However, we believe that wood branches with natural variability demonstrate great potential in nonlinear architecture with complex geometry as they offer more approaches and possibilities in design. In the manufacturing processes, taking advantage of material variability rather than fabricating variable components out of standardized materials decreases processing operations, improves fabricating efficiency, and transforms scrap materials into good account.
Our objective was to develop a real-time material information acquisition system that features variability recognition and shape awareness. Therefore, we used unprocessed wood branches collected from the trees on the construction site on the campus of Tianjin University. These branches were cut by the City Sanitation Department and were considered low valued and were to be chipped or burned if not used (Figure 2).
This study conducted pavilion-scale design-construction experiments. Consequently, we selected branches with lengths of 35–245 cm. Their diameters ranged from 4 cm to 20 cm, thick enough to be strong and thin enough to be easily used.
3.1.2. Measuring Method Decision
The existing measuring and modeling methods can be categorized into active and passive methods. The active methods primarily include moiré fringe, time-of-flight (ToF), structured lighting, and triangulation, whereas the passive methods primarily include shape from texture (SFT), shape from shading (SFS), multiview stereo (MVS), and silhouette methods. These methods rebuild objects with mathematical and physical principles. To be more specific, moiré fringe uses the interference of wave to deduce the shape; ToF measures the time taken by an object to travel a distance through a medium; SFT, SFS, and structured lighting mainly use perspective principles; and triangulation, MVS, and silhouette use the methods of deformations with triangular positioning. The attributes of each method are presented in Table 1 [14–18]. The “+” and “−” signs indicate strong and weak performances.
The perception-modeling system developed in this study was based on the silhouette method. We selected the silhouette method for the following reasons. 2D silhouettes from different views of the scanning material are able to rebuild a 3D digital model by Boolean intersection, and hence the method is feasible. Second, the hardware setup is convenient and suits for the environment of construction site. Third, the software is easy to operate and can be compatible with the popular design software Rhino/Grasshopper.
Based on the silhouette method, we introduced a simple algorithm to improve the speed and efficiency of the obtained outlines from wood branches and rebuild their visual replicas. The optimized perception-modeling system could quickly and easily build 3D digital models of all of the tested branches. The system could be embedded in the end effector of a construction robot for small-scale material components or fitted to a particular holder for large-scale components.
3.2. Material Perception-Modeling Phase
3.2.1. Parameters and Mathematical Formulation
The parameters of the perception-modeling system include camera angle, the size of the provided branches, the distance between the camera and the testing branch, the angle between two adjacent cameras, and the number of pixels in the corresponding direction. These parameters should be considered as a related set, and they would determine the precision and the cost of the system.
In this study, the radius of the branch range is set as R, the distance between the camera and the branch center is d, and the camera angle is set as α. Hence, a relationship is formulated between these parameters as follows:
Then, the number of pixels in the corresponding direction of the camera is set as n, and the resolution of the reconstructed model is set as γ. Then, a relationship is established between them using (1):
In regard to the error analysis, the uniform errors of the system are affected by two factors, as shown in Figure 3: one is the angle between two adjacent cameras, and the other is the radius of the projection on the plane of the camera group. In order to establish the error, the study sets the former as δ, the latter as r, and the inherent error as e. Then, a relationship is formulated between these parameters as follows:
An approach direction is that the performance of the equipment should be improved as much as possible, which means e < γ. Finally, the corresponding equipment configuration and the working range can be determined by solving the above equations.
For relatively smaller branches with a section radius of less than 10 cm, the perception-modeling system is embedded inside a Kuka KR60/HA Robot. Considering the robot arm span and the analysis of the above formular, the length of 165 cm was chosen as a safe threshold for the scanning branch. Actually, the threshold is a reference value with multiple idealizations. The measurement limit is related to the shape of the branch, the position it is placed at, the lens parameters of the camera, and the position of the shot. The limit value is estimated from these parameters through a ternary function. Maintaining a balance between convenience and accuracy, this research finally uses d = 100 cm, α = 0.3π, δ = 0.444π, and n = 720. As shown in Figure 4, the robot would capture nine images from different perspectives as its end effort moves and shots in relevant positions.
For larger components exceeding the robotic scanning range, another strategy that controls several cameras to capture material images from different perspectives at the same time is applied by using a particular holder. With a similar calculation method, this study considers d = 300 cm, α = 0.3π, δ = 0.667π, and n = 720. Six cameras are fixed to a 600 cm long frame to shot the branch at the same time.
3.2.2. Programs
A calibration program for correcting dimensions is prepared before scanning the branches. The program establishes the mapping relationship between the contour pixels of the testing object in the measuring location and spatial points in the modeling software. The scaling parameter and the rotation parameter are calculated and examined. Meanwhile, scripts for reading information from multiple cameras are also developed. This calibration program only executes once at the beginning of the whole workflow and does not require recalculation once the branch is scanned.
After running the calibration program, the system begins to perceive physical branches and rebuild their 3D models virtually. Two programs are developed. One is an external program that captures different views of the branch images and extracts related data for the visual presentation. The other one is an interface program that synchronously receives the data and converts it into the corresponding digital model in the design software Rhino/Grasshopper.
The external program and the interface program run simultaneously. The external program extracts smooth contours of the branch, calculates three-dimensional coordinates of the points in the contours, and generates a file containing the data of all the points. The interface program receives array data, maps the spatial positions of the points, shapes outlines from different views, and generates the corresponding digital model by using the algorithm of the Boolean intersection (Figure 5).
The perception-modeling system developed in this study is endowed with flexibility, reliability, and usability. Firstly, flexibility implies that the material perception programs are applied in modular organizations, where each camera and its corresponding software layer can work independently. Therefore, researchers can add or reduce camera numbers to achieve different accuracy requirements. Additionally, the system displays a high tolerance for faults and hence is considered highly reliable. It considers the unstable operation of the sensor: when some cameras are broken, the other cameras can still work and maintain the system’s function despite a reduced accuracy. Lastly, usability implies that the system is designed for easy operation, low cost, fast running, and high efficiency on most devices. The system has been optimized through appropriate image abstraction techniques that need less computational capacity, so that most computers are capable of running the program without considering the speed issue.
3.3. Dynamic Generative Design Phase
Source materials are recognized, digitalized, and analyzed in the perception-modeling phase and would be well organized in the design phase. They actively participate in generating the overall geometry of the architecture, and the outcome remains dynamic with the updating of the provided materials. This study provides two approaches to generate the dynamic design outcome based on the source materials.
3.3.1. Matching Algorithm
The first approach focuses on the matching algorithm; it generates geometry by optimizing the organization order of the provided branches. Based on connection rules, i.e., each fork of the branch should be connected with a fork of another branch; it firstly gets the results of different combination possibilities. If none of the combinations can achieve the rule, more source branches should be provided until at least one option is resultful. After geometric simulations, material effectiveness of all resultful combinations is calculated and the most effective way is selected. Specifically, if the distance between two adjacent forks is too small, the extra parts beyond the intersection will not bear weight and will be ineffective. The total lengths of the ineffective parts of different combinations are calculated and their results are sorted. Finally, structural performance of generative geometry is simulated and optimized by finite element analysis in Grasshopper (Figure 6).
3.3.2. Positioning Algorithm
The other approach of generative design focuses on positioning algorithm, and the overall geometry evolves by scanning and assembling the source branches iteratively. The outcome is updated by continuously mirroring the current branch and putting it in a suitable position. A guiding curve is defined by the designer from an aesthetic perspective. The branch is assembled along the curve by anchor determination script and orientation determination script. The operation continues until no more branch is provided. In this approach, the overall geometry forms gradually, and the final outcome appears only after the last source branch is captured (Figure 7).
3.4. Flexible Robotic Construction Phase
After the material perception-modeling phase and the dynamic generative design phase, a flexible construction system is developed using real-time robotic communication techniques. The “flexible” term implies that the robotic fabrication system can react to new information acquired during the actual fabrication process rather than merely executing the previously compiled machine code.
3.4.1. Executing Program
Robotic executing program is developed in Grasshopper/KukaPrc. They output synchronous serial commands for the effector actuation in coordination with the dynamic design. Once the design model calculates the position of a branch, the robotic arm would move it to the corresponding location for assembly. In addition, a pause function and a forced-termination function are added to the program in order to prevent accidents.
3.4.2. Monitoring and Adjusting Program
The monitoring and adjusting program is developed using Grasshopper Python to quickly validate the assembly operation before its output in the real world. A monitoring module made of Kinect device is embedded in the end effector of the robotic arm to perform real-time skeletal detection. Quokka, a software tool for connecting Kinect with Grasshopper, is edited to use the Kinect as a 3D scanner. With the monitoring module, points and vectors could then be extracted from the centreline throughout the geometry of the branch, and then the actual status of the branch could be compared with the digital simulation. Adjusting codes are written in Grasshopper Python for modifying the assembling location if the difference between real geometry and digital geometry exceeds the threshold value.
4. Experimental Results and Discussion
Based on the above-discussed cyber-physical system, this research constructed two experimental pavilions to verify the feasibility. The processes and results are explained in the following sections.
4.1. Experimental Case 1: “Robotic Knows” Pavilion
The first experimental pavilion is called “Robotic knows.” The pavilion is devised using a physical-virtual-physical procedure. Information extracted from the physical materials is processed to drive the digital design geometry and the digital model and, in turn, decides the robotic actuation in the physical space.
4.1.1. Physical to Virtual: Material Perception and Design Generation
In this pavilion, the abandoned “Y”-shaped branches with the length ranging from 75 cm to 245 cm are used, and their digital replicas are automatically built using the perception-modeling method developed in this study. The valuable data of the scanning branch are distilled, whereas the useless data that does not affect the design result, such as slight unevenness on the branch surface and the bending amplitude of each fork, are discarded. The valuable data include the following parameters: three-dimensional coordinates of the four points—P0, P1, P2, and P3—of the branch, the length between every two points, and the length and the thickness of each fork (Figure 8).
Then, the digital replicas of all the branches are used to generate the architectural design in Rhino/Grasshopper platform. Multiple screening is used to choose the organization method for all branches and to generate the corresponding architectural geometry. There are four levels of system screening: load-bearing screening, arrangement order screening, material effectiveness screening, and structural performance screening. In load-bearing screening, branches are sorted by volume, and the three largest branches are selected as the main supporting structure. In arrangement order screening, feasible arrangement orders are kept while incorrect arrangements are eliminated. As shown in Figure 9, for any two branches B and C, fix their P1 to P2 or P3 of branches A and check if they can intersect or not and then remove all the disjoint combinations. For the next layer, branches D and E connecting with branches B, repeat the logic. In material effectiveness screening, the total amount of useless material of every feasible arrangement is calculated and ranked. For all possible combinations, the intersection trajectory of branches B and C is an ellipse; choose the position that is closest to the central axis of branch A to ensure the overall stability, cut the useless parts of branches B and C, and repeat this process until all branches are linked to each other. Calculate the total cutting lengths of different options according to the information list of all the branches and rank all possibilities according to wasted materials. The top five options are selected. In structural performance screening, a plug-in of Grasshopper called “Millipedes” is used, and scripts are written in GH Python to test and optimize the structural performances of the five options. The best one is selected and optimized. The initial design sets three load-bearing branches from an empirical point of view. Nevertheless, the simulation results show that the shear force at the grounding points can be very strong and exceed their beating capacities. Therefore, modifications are proposed, and more small supporting branches are added. The upper part of the geometry is also regenerated to make the force distribution more uniform (see Figure 10).
4.1.2. Virtual to Physical: Real-Time Data Flow and Robotic Construction
Once the geometry is generated and optimized, data is transmitted from Grasshopper to the robot arm for creating a direct link between the digital design and the physical construction. Robotic construction includes the following two parts: localization and motion planning. In the localization part, a reference location is used to make the robot “know” its position. The results show that the position accuracy can be achieved within 2 cm. In the motion planning part, a plug-in called KukaPrc is used to control the robotic actuation. The program is written for computing the movement data based on the digital model (Figure 11).
The last step is joint designing for feedback error correction. At present, two methods are available for removing the errors: reducing errors by higher precision control and designing structures with larger error redundancy to directly absorb the errors. Considering the complexity and reliability, this experiment adopts the latter strategy. A special node with universal joints, three-way pipes, and jackscrews is designed to fix the major error. The universal coupling is very flexible and can freely adjust the distance and the angle of the two connection points. Before positioning the branches, the cardans are fixed on each branch. Each pipe with a universal joint is fixed with a fork, and there are three universal joints in each node that are connected by prefabricated metal pipes instead of the traditional shaft. The branch is autonomously moved to the corresponding position by the robot, whereas the cardans are manually welded, and the node is manually locked by the architects. The newly designed joints ensure structural stability and local adjustment (Figures 12 and 13).
4.2. Experimental Case 2: “Robotic Eye-Brain-Hand Coordination” Pavilion
The second experimental pavilion is referred to as the “robotic eye-brain-hand coordination.” The pavilion is evolved by cyber-physical iteration with sensor-actuator feedback. The perception-modeling system is embedded in the end effector of a robot and is considered as the “eye” (the sensing end) of the iteration system, which reads the characteristics of the scanned branch and transfers the data to Rhino/Grasshopper on a PC. The “brain” (the calculation end) analyzes the data to calculate the position of the scanned branch. The “hand” (the action end), i.e., the robotic end-effector, assembles the branch autonomously without programming its motion code in advance. The “eye-brain-hand” program loop operates continuously, gradually evolving and unfolding the overall geometry of the pavilion (Figure 14).
The source materials of this pavilion are “”-shaped branches with lengths ranging from 35 cm to 100 cm. The shape and size of the source material are different from those of the former pavilion for testing the adaptability of the perception-modeling system. The results show that it works well.
The positioning algorithm is programmed according to the aesthetic tendency, structurability, and structure performance. First, a guiding curve is defined by the designer. The space function and site conditions would affect the designer’s conception. The site of this pavilion is flat, where there are no roads or obstacles. Hence, a decreasing spiral guiding curve is defined, expecting that a closed shield is formed with an entrance hole. Then, the number of joints and joint positions are set based on the structurability and structure performance. Before the actual construction, testing experiments were conducted, and the results showed the following logic ensured feasibility and stability: the position of the current branch is determined by itself and the previous one. When assembling branch (N + 1), its midpoint G of the longer fork is found and then one-third of points near the branch attachment points A and B of branch N are computed; the midpoint of line AB coincides with the point G, and the longer fork in branch (N + 1) is passed through points A and B so that the joint position of branches (N + 1) and branch N is known. At this time, branch (N + 1) can be rotated along the AB axis and its rotation angle is determined by the tangent direction of branch (N + 1) to the nearest point of the guiding curve so that the assembly location of branch (N + 1) is determined (Figure 15).
The monitoring programs and adjusting programs are developed to prevent the problems before they even occur. Codes are written in Grasshopper and KukaPrc to allow real-time rectifications. In the digital model, the closest point selected each time is compared to the former one. If the current closest point grows in the opposite direction of the generating surfaces, the former point replaces it. This loop continues until the generating surface moves beyond the guiding curve. If there is any problem during the construction process, the data are fed back to the digital model in Grasshopper for a real-time adjustment (Figures 16 and 17).
(a)
(b)
The end effector of the robotic arm is custom-designed and is supposed to hold the branch firmly and be available for the different surface qualities of all the provided wood branches. First, an electronically controlled claw is used. By using the AFMotor library in Arduino, the opening and closing states, running speed, and rotation direction of the gripper can be controlled in real time. However, such a claw has the following limitations: its opening and closing range is only 0–90 mm, and its maximum force weight is only 16 kg, so it is considered suitable for grabbing only light and thin branches. In order to improve the grabbing ability, the pneumatic parallel open/close claw is finally selected and modified. The size between the parallel plates is adjusted to match the diameter of the branch. The contacting surface of the plates is coated with a special rubber material for increasing the coefficient of friction so that the branch is not twisted. The features of this developed pneumatic claw are as follows: (1) wide opening and closing range, (2) greater grabbling ability, (3) being suitable for branches with different sizes and weights, and (4) the capability of being connected to the GH function and controlled by the generative digital model.
4.3. Discussion
The main advantages of the digital twin paradigm presented by the two pavilions are as follows.
First, the design-construction system becomes more adaptive and flexible. The traditional approaches usually present design by using static modeling and are not able to manage the challenges of dynamic conditions and the changing demands on the construction site. In the digital twin paradigm, generative design with updating information can optimize the output in real time in response to the source materials and unpredictable changes on the site.
Second, collecting and sending the data simultaneously improves the efficiency of handling complex material information and allows better use of each resource. Significant benefits of reduced managing time and cost can be achieved by the virtual experimentation of the generating and assembling process.
Lastly, the digital twin model is continuously updated with all the evolutionary changes and modifications in the physical system, and design and construction are no longer disconnected and rather collaborate in a digital loop, offering a reasonable design and achieving a more efficient and precise actual realization.
Although these two pavilions focus on wood branches, the digital twin paradigm for generative design and robotic construction is applicable to other local source materials. By using the cyber-physical system, material scanning can be combined with a digitally arranged and robotically assembled continuous real-time workflow. Similar to the presented experimental cases, the final design and appearance emerge only after analyzing and responding to the geometric variability of the material.
5. Conclusion and Future Work
In this research work, a real-time perception-modeling system is explored for digitizing wooden branches of variable shapes, and the generated data is deployed later in both generative design and robotic construction phases. Using this system, material data were obtained and transformed from the physical space to the digital space. The data were subjected to geometrical optimization and structural simulation. In coordination with the digital model, the robot simultaneously outputs synchronous serial commands for the effector actuation. The following contributions of this research can be concluded.
First, a stable, robust, and easy-to-use material scanning method is explored. By implementing this method in perception-modeling phase, the unique geometrical information of each unprocessed wooden branch can be digitalized and abstracted so that all of the data is stored in a master digital twin model. This component information is later used in both generative design phase and robotic construction phase for different purposes.
Second, this research focuses on reusing unprocessed wooden branches as the basic components for two on-site constructed pavilions, in which the solutions are oriented toward the source material and the changing environment instead of a predefined workflow. The system is established with the capacity of integrating all data from preparation to construction phases and being updated according to real-time data.
Third, two practical construction cases are built and studied under guidance of the digital twin paradigm. All of the process data is stored and extracted back and forth from a master model, ensuring the data consistency and providing excellent efficiency in dealing with complicated geometry cases. Besides, the developed paradigm can reversely benefit the traditional discrete construction system with the method of adaptive process that copes with inevitable construction errors by feeding real-time information and making alternative decision iteratively.
In this report, a series of potential improvements are presented. First of all, the material sensing part can be improved. Different types of sensors are installed, and the analysis algorithms can be optimized. Therefore, the system is able to perceive and utilize other kinds of material information, such as strength and toughness. Second, the cyber-physical iteration part can be further optimized to deal with physical construction errors. In the robotic arm assembly part, errors inevitably exist between the real and the ideal environments, which may degrade the performance. Further improvement can be achieved by developing a global monitoring system together with the local joint scanning scheme, which provides necessary overall geometric modifying indications towards the ideal form. According to the difference between real geometry and digital geometry, a new strategy should be generated in real time for adjusting the assembling location of the next component.
The theory and method of this study can be used in future research. The digital twin paradigm opens up the possibility of integrating material variability by using highly customized analysis, calculation, and fabrication process [27]. Irregular timber panels or trunks can be analyzed for geometric fitting in large-scale constructions.
Data Availability
The data used to support the findings of this study are included within the article.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This study was funded by Tianjin Science Foundation (agreement no. 19JCQNJC07300).