Abstract
Freehand sketching in a real-time 3D styling system presents a different way to create a 3D model from the traditional CAD system. Strokes are the most common and useful objects in the expression of a stylist’s intention by using the gesture commands and in the construction of geometries. Not only are they sketched freely as drawing on a paper with a pen, but also they can be converted automatically into spline curves and then can be used to form a surface. Therefore, it is necessary to investigate the features of stroke, the extraction and identification of these features, and the applications. Here, the features from a single vector pressure stroke have been discussed in detail according to their applications in an automotive freehand styling system in the early stage of conceptual design. The features have been divided into three parts: the input features, the calculable feature, and the added features. Over a hundred features have been defined and classified with the geometry or performance attributes. The features proposed have been applied in the identification of gesture commands, the expression of art vector stroke by pseudoantialiasing method, and a novel method to generate a spatial curve directly by the pressure of stroke in AutoSketch system.
1. Introduction
Strokes are not only used in the generation of geometries but also applied in many related manipulations to assist sketching, which make the freehand sketching system more natural and suitable for creative design. The most direct application is that the strokes are converted into images in a raster graphics system, for example, in Painter or Photoshop, or transformed into curves in a vector graphic system, for example, TEDDY [1], I Love Sketch system [2], or AutoSketch [3] (Figure 1), which is quite different from the curve generations in the traditional CAD systems. A multistroke combination method has been proposed by Li et al. [4–6]. In recent years, more features have been applied into AutoSketch system for the requirement of real-time 3D sketching [7, 8], especially the pressure information of digital pen that has been used to creature the spatial curve and express the style of stroke, in which more features from the strokes are calculated [9]. Strokes can be drawn like art sketching on a paper with a pencil or marker pen.
For the sketched strokes, one issue is to determine the geometries, for example, the curves from the strokes such as straight lines, arcs, or parabolic curves, and the geometry shapes such as triangles, rectangles, and circles [10–13]. Most of them are regular or standardized geometries and there are many methods to identify them, in which the segmentations are quite common methods to find the primitives from the strokes using features such as geometrical or velocity features [14, 15]. Another issue is to determine the meanings of gesture command sketched. Many gesture command strokes go with the sketching system. Some gesture strokes are designed for specific manipulations in I Love Sketch system (Figure 2). A scratch-out stroke means to delete an object, a roll stroke stands for an undo manipulation, and a small lasso stroke is to select a curve or set a subcoordinate system. The gesture command strokes in AutoSketch support manipulations such as select, undo, redo, and delete (Figure 3). Similar gesture strokes have also been presented in Feng et al.’s gesture-based handwriting mathematics editing system [16] (left) and Song’s ISID [17] (right) systems (Figure 4).
Almost all the definitions of gesture command stroke depend on the techniques of identification [18]. The methods can be summarized into three aspects: the template matching techniques, the statistical analysis techniques, and the neural network techniques. However, it is necessary to dig out the implicit features from a single vector pressure stroke, which may be applied in many applications. Here, only the vector stroke has been discussed because the raster stroke is not easily obtained in a vector CAD system and is bad in real-time performance [19].
A single vector pressure stroke is obtained easily in a vector graphic system with a sequence array, which consists of a series of points. The data is smaller than that of raster stroke relatively, which has a good real-time performance [20].
The features discussed here have been investigated and implemented into AutoSketch system for the identification of standardized geometries and gesture commands. There are five parts of related research works for the development of AutoSketch system: the combination of multistrokes, the identification of sketched gesture commands, the standardization of freehand sketched geometries, the transformation of curves or surfaces from 2D to 3D, and the application of pressure information. This work related four projects: a novel algorithm from 2D sketch to 3D surface model in automotive styling (Science Fund Project of State Key Laboratory of Advanced Design and Manufacturing for Vehicle Body (SKLVBDM2006001)), study on the key technology of integration of 2D hand sketching and 3D automotive styling (State “863” Project (2008AA04Z110)), research on input method of 3D freehand sketching in automotive styling (Science and Technology Plan Projects of Jinhua City (2011-1-045)), and 3D automotive freehand styling system based on the pressure of digital pen and the combination of strokes (Science and Technology Plan Projects of Zhejiang Province (2013C31085)).
These features can be classified into three parts: input, calculable, and added ones.
2. Input Features
The input features are obtained from the output of a digital pen (Figure 5). They are put into an array with the data format of , and its size is (Table 1 and Figure 1). Each point of a stroke contains the information of coordinates, pressure, azimuth angle, and elevation angle. The values of coordinates are within the screen of computer in pixel, where the width is and the height is .
There are only the first four input features that have been used in AutoSketch system until now. The azimuth and elevation angles will be applied to investigate the characteristics of the stylist’s sketching in the future researches. Therefore, the following discussions are not related to the angles.
3. Calculable Features
Calculable features are those that can be calculated out directly from the input ones. There may be different calculable features according to the real applications. Generally, these features are, according to our researches, classified into six parts: extreme value, coordinate, length and width, angle, performance, and auxiliary features.
3.1. Extreme Value Features
There are two extreme value features, and (Table 2). The functions of min. and max. are used for finding the minimum and maximum values from a set of parameters, respectively. The parameter will be replaced by the parameter , , or as needed. The extreme values are used for the determination of the container of an object.
3.2. Coordinate Features
A series of points contains the coordinates in - plane. Some special coordinates are listed in Table 3.
A container is a minimum rectangle or box including an object. stands for a simple horizontal container (Figure 6, left), which is represented by two diagonal points; another is a more general one (right), which is smaller than the horizontal one and is represented by four points and a rotation angle (refer to Table 5).
The center of box container , the centroid of convex , and the centroid of stroke are different (Figure 6). If two segments intersect each other, their containers must be overlapped (Figure 7), and there must exist a minimum box and the cross point of two segments must be within the minimum box as well.
There are four cases in the calculation of cross point: the first segment is vertical, the second segment is vertical, two segments are collinear, and both are in general position. The knee point and the inflection point are different. The former is a sharp turning and its tangent line does not cross the stroke. The latter is a smooth turning and its tangent line crosses the stroke. The axial knee point or is a turning point along the -axis or -axis, respectively.
Two continuous and unparallel segments can be used to form an arc (Figure 8). The arc center and radius are certain. can be used as the radius of curvature of the point .
The convex hull of stroke is a convex polygon containing all the points of stroke [21]. There are quite many methods to find the convex hull, such as Gift-Wrapping, Graham-Scan, or QuickHull method. Gift-Wrapping is simple and fast enough for the point set of a stroke; therefore, it is applied to this work. At first, the point with the minimum is the start point; secondly, the segments from the start point to the rest of the points are set up; thirdly, the angles of these segments with the -axis are calculated out and then sorted; fourthly, the point with the minimum angle is added to the convex hull set; fifthly, the last three points are linked to two vector segments; if the second segment turns right, the last point in the convex hull is deleted; otherwise, it is as the current start point and then the second step is repeated until all the points are checked.
3.3. Length and Width Features
The lengths and widths are geometric features. Their positions are shown in Figure 9 and the definitions are listed in Table 4. The length of segment and the length of stroke are very useful. The average radius of arc stands for the curling degree of stroke and the difference of radius reflects the vibration of it. The dimensions of container , , and stand for the size of geometry. The length between both ends of stroke is a span of stroke and the difference of two values is also a span between two points.
3.4. Angle Features
The angles are also geometric features. Their size stands for the position relationship between two segments (Figures 8 and 9). The value of azimuth angle , , , or is from 0 to 360°, which can be divided into eight zones, that is, east, west, north, south, northeast, northwest, southeast, and southwest, as shown in Table 5. However, the rotation angle of segment is not the real value in degree; it is a direction with the values of <1, 0, and >1, standing for turning left, collineation, and turning right, respectively. The azimuth angles , , and are useful for some special applications. is used to judge the general direction of stroke from start point to the end point. For example, the shapes of undo and redo gesture commands are quite similar; however, their angles are opposite. is the azimuth angle of start point and is used to judge the start direction of stroke, and is the azimuth angle of end point and is used to judge the end direction of stroke. For example, the select command “” in Figure 3 is determined by these three angles. should be around , should be around , and should also be around , which will limit effectively the shape of select command to a normal one.
3.5. Performance Features
Some features standing for the inner characteristics of a stroke are defined as the performance features (Table 6).
The areas, and (Figure 6, left), represent the size of geometries [22]. The number of times of turning left or right stands for a kind of sketching pattern. The number of times of self-intersection represents the complexity of stroke trace; for instance, (Figure 10, middle); the cross point may be different from the real one [23]. The slope of segment , similar to the azimuth angle of segment , gives the direction of segment. The aspect ratio of container represents the shape of the box. The values with , , or stand for the notion that the shape of the container is a flat rectangle, square, or tall rectangle, respectively (Figure 10, left). The ratios, and , reflect the level of parameter changed; for example, the stroke of sign (Figure 10, middle) is completely turning right; however, the stroke of sign may have (right). The speed , average speed , size of speed , and acceleration stand for the performance of sketching habits. In addition, the speed can be used to find the turning points, where the speed is lower than that in other sections of stroke (Figure 10, right). The ratios of deviation, , , , and , reflect the vibration of the parameters.
3.6. Auxiliary Features
Some features are used for the calculation of other features (Table 7). The change direction of segment in -axis , for example, is defined as 1, 0, −1} for the stroke going up or down, respectively (Figure 11, left). If two containers are overlapped, the value is 1, otherwise 0 (Figure 11, middle). If two segments are intersected in a general case, the value of is 2; for the special case of collineation, it is 1, otherwise 0 (Figure 11, right). These features represent the relationship of two parameters, such as the relationship of direction, overlapping, or position.
4. Added Features
Added features depend on the real applications to some extent. Here, only three kinds of features are discussed. Those are control features of stroke expression, calculated features, and curve features.
4.1. Control Features of Stroke Expression
Unlike the data of image, the data of stroke does not contain features such as color, grayscale, pen type, and width. The stroke can be drawn as natural as possible in AutoSketch with Chinese calligraphy art effect. Therefore, the pressure information is added to control the width and color of stroke. An example of gesture command stroke is shown in Figure 12, and the related parameter values are listed in Table 8. The first nine features (Table 8) are switch parameters with the value of 1 or 0, which stands for yes or no attributed to the control of the display status of stroke, respectively. There are two color modes for filling, color or grayscale. The color is in with the values from 0 to 255. The transparency value is from 0 to 100. The rest of the values are in pixel.
4.2. Calculated Features
In order to express a vivid stroke and use the pressure feature in further applications, some calculated features have been proposed (Table 9).
The pressure of stroke can be converted into the radius or width of pen within the start radius and the end radius ; it can also be transformed into color or grayscale, using the value of or . If distance between two points is quite large, the stroke may be intermittent (Figure 13(a)), and it should use much more circles to fill the gaps. is the number of circles used for filling the gap between two points. The step is controlled by the number and density of circles. Because the pressures of two consecutive points are different, the converted radiuses must be different, and the stroke between two points may be shrunk or expanded sharply and is not smooth at the edge of the stroke (Figure 13(b)). Therefore, the radiuses between two points should be changed gradually. The transit radius is controlled by . The color of stroke should be changed gradually in the same way; otherwise, the stroke will look like the one shown in Figure 13(c).
Even after doing this, the stroke still looks like a zigzag at the edge, and it cannot be smoothed like image antialiasing. Therefore, a pseudoantialiasing method is applied to smooth the edge of the stroke. The colors can be divided into four parts: the background color , the stroke color , the edge color , and the middle color . The last two can be calculated out once the stroke and background colors are known. Sometimes, the opacity value is applied to call some functions so that the value of transparency should be converted into the opacity value, and vice versa. Because the stroke is drawn repeatedly with circles, the transparency should also be changed gradually; the transit transparency is determined by the width of edge .
The pressure of stroke can also be used to control the influence zone of pressure (Figure 14), which is an area of stroke or surface changed in the manipulation of stroke or surface modification. The coordinates and can be obtained according to the radius , which is converted from the pressure.
4.3. Curve Features
There are quite many features coming from the relationship of parameters, such as , , and , which are discussed above. However, from a stroke, there are still some other relationships such as or (Table 10). A stroke contains a lot of points; some of them are actually the noise points which make the stroke unsmooth. Therefore, the stroke should be preprocessed and compressed with a few good points to become a smooth curve and is drawn out using a spline , which is compressed further by the application of some curve fitting algorithms. In addition, the knee point curve is needed for some special applications, such as the identification of gesture commands and geometries. The original pressure curve corresponding to the stroke cannot be applied directly. Similarly, it should also be compressed and smoothed to the curve , which is then translated to the -axis and becomes another curve , where the start point is set to the origin . It is then rotated along the origin to make both ends of the curve become locked on the -axis and become the curve . The curve may be fit into a parabola curve for controlling the stroke changing in one direction ( direction in parametric space), which can control easily the shape of spatial curve. The pressure curve - can be mixed with the stroke in to form a spatial curve consequently while the pressure is converted into to some extent (Figure 15).
Currently, the curves and have not been used yet in AutoSketch. However, they give the new dimensions in the parametric space like the curve and will be applied in further researches.
5. Validation of Features
All the features have been applied in AutoSketch for the identification of gesture commands listed in Figure 3, the expression of the art strokes, and the generation of spatial curve by the pressure of stroke directly.
5.1. Identification of Gesture Commands
The features in Tables 1~7 have been used to determine the commands listed in Figure 3 according to the combinations of the special conditions, as shown in Table 11. The trials demonstrate that the identification of gesture commands is quite satisfied as shown in Table 12.
5.2. Expression of Pressure Stroke
The features in Tables 8~9 have been used to express a pressure vector stroke. For real-time application, the method using the above features is called the pseudoantialiasing method in order to differentiate it from the antialiasing method of raster image in image processing. The style of stroke can be set in the attributes settings window, as shown in Figure 16. There are five different styles shown in the same time to check which one is better. The types of attributes and the values are listed in Table 13.
5.3. Generation of Spatial Curve
The features in Table 10 have been used to generate a spatial curve directly by the pressure of stroke. For instance, a pressure stroke is drawn in plane, and a spatial curve is created immediately by choosing the direction in , as shown in Figure 17. The mirror stroke is shown to demonstrate that the curve is a spatial one, not a planar one.
6. Conclusions
The features coming from a single vector pressure stroke have been investigated and discussed in detail. Those features are classified into three parts: the input features, the calculable features, and the added features. The input features are derived from the output of a digital pen, which may be different for the different input systems. The calculable features, to be honest, are the part of features that a stroke contains. They are just the simple and easy-to-calculate ones for the real-time application in 3D freehand sketching system. For the added features, they may be different depending on the real application, and they are added here for the expression of a vivid stroke, which is a novel pseudoantialiasing method, and the generation of a spatial stroke according to the pressure of the digital pen, which is a novel method to create a spatial curve directly on the planar digital screen. For the further researches, the azimuth and elevation angles of the digital pen may be applied in order to investigate the sketching habit and the gestures of holding the pen.
Competing Interests
The authors declare no competing interests regarding the publication of this paper.
Acknowledgments
This work is supported by the Science and Technology Plan Projects of Zhejiang Province under Grant no. 2013C31085.