Abstract
A venation skeleton-driven method for modeling and animating plant leaf wilting is presented. The proposed method includes five principal processes. Firstly, a three-dimensional leaf skeleton is constructed from a leaf image, and the leaf skeleton is further used to generate a detailed mesh for the leaf surface. Then a venation skeleton is generated interactively from the leaf skeleton. Each vein in the venation skeleton consists of a segmented vertices string. Thirdly, each vertex in the leaf mesh is banded to the nearest vertex in the venation skeleton. We then deform the venation skeleton by controlling the movement of each vertex in the venation skeleton by rotating it around a fixed vector. Finally, the leaf mesh is mapped to the deformed venation skeleton, as such the deformation of the mesh follows the deformation of the venation skeleton. The proposed techniques have been applied to simulate plant leaf surface deformation resulted from biological responses of plant wilting.
1. Introduction
Realistic modeling of plant leaves has a long history in computer graphics. This is partly due to their either beautiful or colorful images, and partly they have a strong visual effect on the audience. Many techniques have been proposed for modeling the geometry or shape of leaves. Most of these methods, however, just describe the shape of leaves formed in normal natural conditions and do not account for the shapes formed under stress, for example, curled or withered leaves.
Additionally, there has been a great deal of previous work on simulating motions of plant, including plant growth, motion in the wind, and so on. But less work has focused on modeling leaf surface deformation and simulating subtle behaviors of plant, such as wilting of leaves suffering from insufficient water supply. This paper presents a venation skeleton-based deformation method for plant leaves and aims to develop an approximately kinematic model of leaf for simulating motions of plant leaves, especially wilting. Firstly, an initial leaf skeleton is exacted from scanned image of leaf. The leaf skeleton plays two roles: it is used to generate a venation skeleton for later deformation; a geometric mesh for the leaf surface is also constructed from the leaf skeleton. Furthermore, a subdivision scheme is applied to generate a detailed triangular mesh from the initial mesh, and each vertex in the mesh is mapped to its nearest vertex in the venation skeleton. Then the venation skeleton is deformed interactively to desired shape. Lastly, the detailed mesh is deformed according to the deformed venation skeleton. Applications of our approach to simulate wilting plant leaves with realistic results illustrate the flexibility and effectiveness of our model.
2. Background and Related Work
2.1. Venation Patterns
Both the outline and venation system of a leaf are essential in the recognition of plant species. Various venation structures can be found in plant kingdom. It is believed that venation patterns correlate closely with the taxonomic groups of plants and the shapes of leaves. Hickey [1] has given a classification for the leaf venation patterns, in which pinnate venation and actinodromous venation are two common found categories (see Figure 1). Pinnate venation characterized by a single primary vein (the midvein) being attached by several secondary veins; the primary vein originates at the base and extends toward the leaf tip. Whereas in actinodromous venation, three or more primary veins diverge radially from a single point. Primary veins support sequences of secondary (lateral) veins, which may branch further into higher-order veins. The secondary veins and their descendants may be free ending, which produces an open, tree-like venation pattern, or they may connect, forming loops characteristic of a closed pattern.
(a)
(b)
Although the interrelationships between topological or geometric properties of the various leaf venation patterns and functional aspects are far from being well understood, it is believed that the leaf venation system brings various functional properties. More information about this can be found in [2].
2.2. Model Deformation and Motions of Plant Leaf in Computer Graphics
Some researchers have endeavored to generate curled shapes of plant leaves. Prusinkiewicz et al. [3] provided a detailed representation of combining interaction and parameterized algorithms for realistic plant modeling and scene creation involving plants, including curled leaves. Mündermann et al. [4] proposed a method for modeling lobed leaves; curled surface of leaves could be generated by using free deformation in their framework. Recently, Hong et al. [5] proposed an interactive method for modeling curled leaf surface. But it could involve excessive manual interactions for generating a desired curled shape of leaf by using their method. Studies on curvature of plant leaves from biophysical perspective have raised the question of what role, if any, genes play in controlling the curvature of leaves [6]. Yet there are some researchers study waved or wrinkled pattern in leaves with physical analysis [7]. But these may go beyond our focus in this paper.
In aspects of modeling motions of plant, most work has been done on modeling dynamic motions of tree in the wind, such as the work demonstrated in [8]. Based on the fact that plant growth is generally influenced by gravity and tropisms, Jirasek and Prusinkiewicz [9] proposed a biomechanical model for creating curved plant branches by using physically based modeling. Hart et al. [10] extended their idea in modeling plant growth considering physical properties of the plant. Note that none of the above models simulated leaves behaviors. Wang et al. [11] had simulated physically the growth of a plant leaf; the physical model used in their simulation is the governing equations of fluid mechanics—the Navier-Stokes equations. But they just tested their model in 2D.
Much work has been done on surface deformation. And many techniques, such as multiresolution mesh representations [12], skeleton-driven global free-form shape deformations [13], and differential deformation [14, 15], have been developed to help artists deform object shapes. But none of these techniques has been tested by modeling leaf deformation.
3. Overview of the Proposed Method
Figure 2 gives an overview how our proposed method works. The modeling processes include constructing the venation skeleton from a leaf image, deforming the leaf surface, and so forth. These schematics label the transition processes in uppercase letters, A, B, and so forth. The interactive simulating system that we propose tries to strike a pragmatic balance between processes that can be automated and those that seem to require interaction to achieve the desired level of realism. It has primarily been designed to support our experimentation with interactive animation of leaf motions.
Each process will be detailed in the following sections.
4. Generating Venation Skeleton
The venation structure of a leaf plays a major biological role in determining the leaf surface shape and controlling its deformation; therefore, we use it to control the deformation of a leaf blade. To generate the venation skeleton, currently we consider an interactive method. In Figure 2, process A and C illustrate the steps for generating venation skeleton of a leaf.
4.1. Extracting a Leaf Boundary
We use a representation of a leaf skeleton consisting of two boundary curves and a midvein curve which consists of feature points, as shown in the left of process A. These boundary curves can be reconstructed from feature points in the boundary of a leaf, while these feature points can be extracted automatically from a scanned digital image by using a standard edge detection algorithm, or obtained by using a 3D digitizer.
To meet the need of interactive design, the leaf skeleton can also be constructed automatically with a parametric method, in which the length of midvein, the width of leaf blade, and the number of feature points are all initialized with parameters.
4.2. Generating Venation Skeleton
The skeleton was originally introduced by Blum [16] for 2D shapes in order to provide a symmetry-based shape representation for shape perception and recognition. Recently the skeleton in 3D has been studied in connection with a research on shape organization [17] and shape manipulation. Practical extraction of the skeleton of a 3D shape is usually based on 3D Voronoi diagram techniques [18].
For our needs, we develop an interface for generating interactively venation skeleton from a leaf skeleton. As Figure 2 shows (from process A to process C), the leaf skeleton can be obtained from a scanned image; a venation skeleton is then generated from the leaf skeleton, and each vein is segmented. The result of process C demonstrates a generated venation skeleton consisting of one midvein and four secondary veins; the black vertices string segmented each vein into several line segments. The process of generating venation skeleton involves several manual interactions including defining the start point and end point for each vein, and specifying parameters for segmenting each vein. Note that the venation skeleton is not unique. It can be created according to actual needs. Users can decide how the midvein crosses the leaf skeleton, and how many secondary veins are attached to the midvein.
5. Leaf Surface Meshing and Banding
5.1. Constructing Leaf Surface
We have constructed a leaf skeleton with two boundary curves and a midvein curve as shown in Figure 2 (process A). To mesh the void area within these boundary curves, we employ Delaunay triangulation scheme, because it can deal with the problem of existing concave area in the leaf blade, which is difficult to render directly by simple polygon. For example, lobed leaves often have irregular silhouette characterizing by a number of concave outline. When using Delaunay triangulation, we can use directly the feature points in the midvein and silhouette, or extract a series of points from the midvein curve and silhouette curves with a fixed interval. The mesh resulted from process B of Figure 2 is generated from the result of process A by using Delaunay triangulation scheme.
The initial mesh of a leaf surface generated by using Delaunay triangulation is generally irregular and rough. It is necessary to refine the mesh for later deformation. Currently, we use a simple method to subdivide the initial mesh. Generally each triangle can be divided into four triangles. To meet users’ requirements of interaction, we provide two parameters for the subdivision: one is the times of iteration steps of subdivision, the other is the minimal distance of an edge in the mesh. The iterations timeis used to control the number of iterations in the subdividing, while using the constraint of the minimal distance for an edge is to avoid generating too short edge. The lower figure of process D in Figure 2 illustrates a mesh subdividing the initial mesh shown in the upper figure at the same process with iteration steps and minimal distance being specified 2 and 0.5, respectively.
5.2. Banding
The banding is to attach all the vertices in the subdivision mesh to the initial venation skeleton of the leaf. The banding is based on the distance of each vertex to the venation skeleton. In other words, each vertex in the mesh is banded to the nearest vertex in the venation skeleton. Figure 3 gives an example of banding a leaf mesh to its venation skeleton which consists of a midvein and two secondary veins.
6. Venation Skeleton-Driven Leaf Surface Deformation
Skeleton-based methods have been extensively used for mesh deformation in computer animation and computer modeling. In this method, two or more “bones” meet at each joint, to control shape deformation. This allows intuitive control, naturally describing the way in which many objects, for example, animals, deform the muscles and other tissues follow motions of the underlying bones.
Traditional skeleton-based methods commonly require a tedious process of weight selection to obtain satisfactory results. Note that the natural deformation of plant leaves is different from the deformation of humans’ or animals’ organs, and spontaneous motions of a leaf blade are relatively simple. The most regular motions of plant leaves which can be seen by our naked eyes are curling and wilting. So we can constraint the movement of vertices in the venation skeleton for simulating the movements of leaves. The major goal of our approach is to develop an approximately kinematic model for simulating these motions of plant leaves.
For our needs, we restrict the movement of each vertex in the venation skeleton by rotating it around a fixed vector. Figure 4(a) illustrates how a venation skeleton works. For convenience, the venation skeleton includes a vein only, which consists of four segments. The light black vertex serves as the root node. Note that the movement of a leaf blade is always downward during its wilting, as such each joint in the skeleton segment does spherical movement. Take vertex , for example, vector will align gradually vector during wilting, in which is a downward vector reverse to -axis, while vector is perpendicular to the plane which contains vector and . To the movement of vertex , it can be looked as is rotated round vector .
(a)
(b)
To obtain a motion sequence of a vertex in a skeleton, the simplest method is to rotate the vertex with a fixed angle, such as θ in Figure 4(a), and the angle is given commonly by the users. But a conventional method is using inverse kinematics [19]. We have mentioned before that a vertex in the skeleton will do spherical rotation during the wilting of its corresponding leaf surface. As Figure 4(a) illustrates, we can calculate the new position of vertex by using the following parametric equation: where . Further, the motion sequence of can be obtained by increasing parameter . This may simplify the rotation operation. Note that vector needs to be recalculated when vertex is repositioned, but it is always downward.
And also, all child segment of a vertex in the venation skeleton will follow the movement of the vertex; this can be done by passing a displacement and rotation angle to its child vertices when the vertex is being rotated. Figure 4(b) gives results of rotating four vertices in a skeleton successively, in which the blue is the rotated vertices while the red is the base point.
We have detailed the mechanism for controlling movement of the venation skeleton and methods for constructing the leaf mesh in our approach. The last step is to deform the leaf surface according to the deformed venation skeleton. This process can be illustrated as process F in Figure 2.
Firstly all the vertices in the subdivision mesh of the leaf are banded to the initial venation skeleton. Then the initial venation skeleton is deformed by using the method described in Section 6.1. For example, at process E, we can generate the shape of the venation skeleton shown bottom in Figure 2E from the initial skeleton shown top in Figure 2E (with different number of joints in each vein). Lastly the position of each vertex in the mesh is recalculated according to the new coordinate of its banded vertex in the venation skeleton. The left-bottom figure in Figure 2 shows the resulted mesh.
The right-bottom figure in Figure 2 illustrates the rendering result. The texture mapping is calculated before the deformation, and no need to remap after the deformation.
It needs to be noted that the number of joints in each vein in the venation skeleton will influence the effects of deformation. It is easy to imagine that the larger the number of joints is, the smoother the deformed surface will be. And large deformation needs large number of joints. But larger number of joints in the venation skeleton also means more computation and more difficult controlling over the deformation. In our experiments, we find that pleasing visual effects of the simulated wilted leaves shape can be achieved when the number of joints in each vein is not smaller than 4. Users can also obtain a satisfactory result by interactive experiments.
Constraints and collision detection are usually the common issues in surface deformation. For constraint, we have stated that each vertex in the venation skeleton can rotate around a fixed vector. Additionally, the rotation needs to meet some extra constraints. For example, the leaf surface will be always drooped during its wilting. In simulating the effect of wilting leaf surface, a vertex in the leaf mesh could not be rotated after it had reached the maximal drooped distance. When simulating curling of a leaf, it needs to avoid overlap of the leaf surface. This can be done by keeping the included angle of two adjacent line sections on each vein being larger than a predefined angle.
Collision detection and response is usually the most time consuming process for the overall simulation. Currently we just consider a collision detection to avoid self-intersect in the deformation. During deforming a leaf mesh, each handling currently vertex needs to be checked if its movement will pierce some triangle in the mesh. Piercing means that there is intersection in the triangle mesh. If no piercing occurs, no response is made. If there is an intersection, then we calculate a maximal displacement from the precalculated displacement for the vertex to move to avoid intersection, and correct the displacement of corresponding vertex in the venation skeleton.
7. Applications and Discussion
We implemented our algorithm for venation skeleton-driven leaf surface deformation in C++ on a PC with a 2.8 GHz Pentium D processor and a NVIDIA GeFore 7900 GS graphics card, and used OpenGL to render the results. In this section we report the modeling results.
Firstly we simulate wilting effect of a plant leaf. The plant used in this experiment is cucumber. Figure 5 shows a photo of a real wilting cucumber in greenhouse. In our experiment, we first simulate the wilted effect of a cucumber leaf. As shown in Figure 6, Figure 6(a) is the initial shape of the simulated leaf surface while Figures 6(b) and 6(c) are two deformed results corresponding to two different levels of wilted leaves. The initial leaf surface is created interactively from a scanned image within 3 minutes, with 187 vertices and 286 triangles. The iterations time for subdividing the leaf surface is specified to 0. In this experiment, the venation skeleton consists of a midvein and two secondary veins parameters for segmenting these veins are 20, 4, respectively, and the total time for simulating the leaf wilting is 8 seconds with 22 frames.
(a)
(b)
(c)
We then use the venation skeleton model to simulate the process of a cucumber wilting. Figure 7 demonstrates the simulated results, in which Figure 7(a) is the initial shape, Figure 7(b) simulates a slight wilting leaves, and Figure 7(c) simulates an acute wilting one. This initial plant is created within 5 minutes by using an interactive interface that we have developed for designing crop structure. The number of vertices and triangles of this model are 4082 and 6150, respectively. We use three instances of leaf surface in the cucumber model, and the venation skeleton in each instance is different from each other, but with the same number of veins (three). The venation skeleton is deformed automatically by rotating the vertices in the venation skeleton downward from the boundary to root of the leaf, by using (1), and the above leaves start wilting later than the lower leaves do, whereas the speed of wilting can be adjusted by modifying parameter t. This simulation consists of 300 frames and runs for approximately 2 minutes.
(a)
(b)
(c)
The second application example is simulating a watermelon leaf wilting. Watermelon leaf is a typical lobed leaf. We use a venation skeleton shown in Figure 8(a) to control its deformation of the leaf blade. The venation skeleton consists of one midvein and four secondary veins, and the parameter for segmenting the midvein is specified to 30; parameter for segmenting secondary vein is 10. The initial shape of the leaf is shown in Figure 8(b), which contains 239 vertices and 316 triangles, while Figure 8(c) demonstrates three wilting effects, respectively. We do not apply subdivision to the mesh of the leaf surface (the iterations time is 0), but the results are plausible.
(a)
(b)
(c)
To test the influence of the iterations time for subdividing leaf surface to the simulation of leaf wilting, we make an experimental comparison, and the simulated results can be seen in Figure 9. It can be concluded that the larger the iterations time for subdividing, the smoother the simulated wilted leaf surface.
(a)
(b)
(c)
The above application examples demonstrate that the proposed venation skeleton-driven approach for simulating wilting of leaf surface is effective and flexible. All of these simulations are running in real time. It can generate realistic effects of wilted leaves similar to natural shape. Currently generating the venation skeleton is manual and interactive in our framework, and controlling the motions of leaves in the scale of a plant is still simple. In fact, leaves wilting may be a natural response for plant adapting to the environment basing on their inner state. An attractive area for future work might involve combining our dynamic modeling technique with physiological model of the leaf. In addition, we just consider a single plant or a leaf in our framework. It is desirable to simulate the motions of plant leaves in an ecosystem scale.
Compared to previous related work on modeling shape of plant leaves, in our knowledge, our method is a first attempt to model plant leaves wilting in computer graphics. The proposed approach cannot only generate wilted leaves shape, but also simulate the wilting process of a plant. This provides an intuitive mechanism for animating subtle motions of plant.
8. Conclusion
We have presented a model for modeling wilted leaf surface and simulating motions of plant leaves. This model deforms a leaf surface by driving a venation skeleton which is embedded into the geometric mesh of a leaf. The venation skeleton can be created from any polygonal mesh of leaf surface, whereas the polygonal mesh can be captured from real leaves, which makes it easy to create highly realistic leaf appearance models. Currently generating the venation skeleton is manual and interactive in our framework, and with a parametric method.
We have demonstrated our model by simulating wilting of a cucumber plant and a watermelon leaf. But it needs to be noted that motions of plant leaves would result from a series of complex reasons, which are difficult to be revealed and simulated, as such mechanism of motions of plant leaves is not easy to model. The leaf deformation model presented in this paper is an example of a model that provides intuitive control for simulating of some motions of plant leaves. An exciting area for future work is the development of a framework for virtual agronomic experiment for broader classes of plants.
Acknowledgments
This work is supported by National High Tech R&D Program of China under Grant no. 2007AA10Z226, Beijing Natural Science Foundation of China under Grant no. 4081001, and the National 11th Five-year Plan for Science & Technology of China under Grant no. 2006BAD10A07.