Recent Advances on Modeling, Control, and Optimization for Complex Engineering Systems
View this Special IssueResearch Article  Open Access
Measuring Technological Progress of Smart Grid Based on Production Function Approach
Abstract
Production function theory combined with data envelopment analysis (DEA) and ridge regression analysis (RRA) is applied to evaluate the technological progress of the smart grid. The feasible conditions of production function models are determined by the DEA algorithm. RRA is applied to estimate the relevant parameters of the evaluation models under study. One of the significant steps in the design of the assessment algorithm is the structure of production function models. Therefore, the CobbDouglas, constant elasticity of substitution, and translog production functions are employed to evaluate the technological progress of the smart grid, respectively. The results of analysis and calculation mainly include the DEA relative efficiency, slacks in inputs and outputs of inefficient units, estimated parameters, and quantitative indices of technological progress.
1. Introduction
The smart grid is a new modern power grid, and it owns advanced metering technologies, information communication technologies, analysis and decision technologies, automatic control technologies, and highly integrated physical infrastructures [1]. Different from traditional power grid, the intelligence is the most significant attribute, and it is also the core value of the smart grid, to improve the socioeconomic benefits for the public. Generally, the intelligent technologies of the smart grid mainly include advanced technologies and equipment in the generation, transmission, substation, distribution, and dispatching fields, and they will enhance the selfhealing ability, the integration of information and communication, the highly efficiency of management, and the interaction with consumers to play a part in optimizing the operation of the system [2].
The evaluation of technological progress not only will be able to reflect the technological level of a smart grid, but also can measure the economic benefits brought from the applied advanced technologies. However, the smart grid as a comprehensive engineering is with a long construction period, intensive investments, and highly technical difficulties. It is very hard to quantitatively identify the development level of the smart grid. Hence, how to evaluate the construction effect of a smart grid and the intelligent technology availability has become one of the challenges for the current assessment research of the smart grid. It is necessary to present an evaluation methodology to measure technological progress of a smart grid.
Although the construction state of smart grid is still in the initial stage, the evaluation research and practice of smart grid have been reported preliminarily. The US Electric Power Research Institute (EPRI) designed the assessment system for smart grid programs over the planning and construction periods, for the purpose of the identification of the technological levels and the metrics of smart grids. Furthermore, this work would be helpful to perform the costbenefit analysis of smart grids in US [3, 4]. Different from the EPRI, the US Department of Energy only outlined the overall development ideas and some major metrics for the smart grid in the report, not describing the specific interpretations in detail [5]. The European Network of Transmission System Operators (ENTSO) also conducted an evaluation index system of investment grant project for the European smart grid, but not analyzing the technological benefits [6].
As known to all, the smart grid has been a hot topic in electrical engineering sector. An effective and scientific evaluation method is beneficial to the identify, the problems of smart grid construction, and advanced technology application. Hence, in this paper we attempt to present a methodology to evaluate the technical level of the smart grid based on the product function. The production function specifies the maximum output that can be produced with a given quantity of inputs. It is defined for a given state of engineering and technical knowledge. From the economic point of view, technology innovation occurs when new engineering knowledge improves production techniques for existing products. Such technological change is equivalent to a shift in the production function. Consequently, the production function models are developed to assess the technological progress and socioeconomic benefits of the smart grid. Moreover, we consider the application of the DEA methodology to determine the efficient production state of the smart grid, because efficient production is the necessary condition of application to production function theory [7–9]. Through the analysis of DEA, the production function models are built based on the output, input, and technology items. The advantage of the proposed methodology is to quantitatively evaluate the impact of the smart grid technologies on economic benefits, which will show the intellectualization effects of a power grid. In addition, ridge regression is employed to estimate the parameters of production function models. Finally, case studies demonstrate the effectiveness of the proposed approaches [10].
This paper is organized as follows. Section 2 constructs some mathematical models with DEA and production function theory. In Section 3, the application of the proposed methodology is presented. Some discussion about the property of production function is done in Section 4. Finally, Section 5 summarizes the main conclusions and contributions of this paper.
2. Methodology
2.1. Data Envelopment Analysis
The DEA is an efficiency modeling approach that can be widely used to measure the relative efficiency of different decisionmaking units (DMUs). The DEA can not only analyze the simple inputoutput ratio, but also handle multiple inputoutput variables. The purpose of applying the DEA is to provide a judging standard that shows that the production state for the smart grid is efficient, and then the production function can be used to analyze the technological change. Otherwise, supposing the production based on the given inputs and outputs is inefficient, the slack analysis of DEA will offer the improved measure enabling the efficient perfect inputtooutput state. Mathematically, the DEA algorithm is in essence a linear programming procedure. The formulation for the DEA methodology can be described as follows: where and are, respectively, the weight coefficients of input and output variables, is the amount of input utilized by the DMU , is the amount of output produced by DMU , the notation 0 is the designed unit for an optimization run, is a small positive number, is the dimension unit vector, and is the dimension unit vector.
The above model is CharnesCooperRhodes (CCR) model that is suitable for DEAbased study of electric utilities [11]. The CCR model is first presented and named by the operation researchers of A. Charnes, W. W. Cooper, and E. Rhodes. It is remarkable that the CCR model is linear programming problem in mathematics. Instead of solving the CCR model as stated above, an equivalent model is presented since it requires lesser computations and is easier to implement. The dual program form of primary CCR model is represented as follows: where is the scalar quantity that is technical efficiency score, is the decision variable of the DMU , and and are, respectively, the slack and surplus variables.
Once the optimal solution , , it illustrates that the DMU is called DEA efficient [12, 13]. The slack vectors including the input excess and the output shortfalls are defined as where is the gap vector in inputs, is the gap vector in outputs, is the input vector, and is the output vector.
2.2. Production Function Theory
The production function focuses on the relationship between the amount of input required and the amount of output that can be obtained. Suppose is the output vector and is the combination of input variables; the production function can be generally described as
It is noticeable that technological progress is an implicit variable and it is difficult to be calculated by the universal methods. Based on the idea of “residual value,” the technology as the independent variable can be separated from the production function. Hence, technological progress regarded as a residual value is calculated indirectly by this way. Some typical production functions are introduced as follows.
2.2.1. CobbDouglas (CD) Production Function
The CD specification is described as a function with the input and maximum amount of output that can be produced using a combination of applied production technology [14]. The inputs consist of the capital investment and the labor resource. The mathematical expression of a CD production function is presented in [15] as follows: where is the technological progress variable, is the capital investment variable, is the labor resource variable, is the output variable, and and are, respectively, the output elasticities of capital and labor. It is remarkable that some assumptions play a key role in the derivation of CD production function: constant returns to scale and perfect competition. Under the law of constant returns to scale, the sum of and is equal to one. Moreover, it is also assumed that the technological progress for the production is neutral.
2.2.2. Constant Elasticity of Substitution (CES) Production Function
The classical CES production function derived by Arrow, Chenery, Minhas, and Solow in 1961 is one of the most widely used production functions. The CES production function is developed based on the assumption that the relationship between (output per unit of labor) and (the wage rates) is independent of the stock of capital. However, the CES production function is also subject to the limitation that the value of the elasticity of substitution is constant although not necessarily equal to one. The explicit formula of a CES production function is described in [16] where is the proportional distribution parameter, is the scale parameter, , , or , respectively, corresponds to increasing returns to scale, constant returns to scale, or decreasing returns to scale, and is the substitution parameter. In particular, while tends to zero, the CES production function will be transformed into the CD production function, so CD production function is the special form of CES production function.
2.2.3. Translog Production Function
The translog production function imposes no more restrictions on returns to scale and the elasticity of substitution than the production functions above. The translog production function is recommended in [17], and the mathematical representation is defined as follows: where , , , , , , , , , and are the undetermined parameters and is the time variable.
A major advantage of the translog production function is that the elasticity of substitution for each input component is variable. Besides, the translog production function enables a richer specification of the relationships for the inputs compared to other production functions in the previous description. Nevertheless, the translog production function owns more parameters than the CD and CES production functions, which means that the complexity of parameter estimation for translog production function will make a significant challenge.
2.3. Parameter Estimation
Solving the parameter estimation problem is one of the most significant steps in the evaluation procedure using the production functions [18]. Considering the characteristics of the estimated parameters in the proposed production functions, such as the collinearity and correlation properties, the ridge regression analysis (RRA) is adopted to perform the parameter estimation in this paper. The RRA is a dedicated to the analysis of the data of linear biased estimation regression method, and it is in essence an improved leastsquare estimation method [19]. Therefore, it is suitable to employ the RRA to minimize the correlation effects of the variables. The fundamental principle of parameter estimation by the RRA is shown in brief [19].
Give the linear model where is the variable matrix, is the observed vector, is the estimated parameter, and is the error term. The ridge estimator of is where is the transposed matrix of , is the identity matrix, and is the scalar parameter.
2.4. Quantity Property of Technological Progress
Using the production functions, some quantitative indices representing technological progress in the production process need to be calculated. The evaluation indices are described in detail as follows.
2.4.1. Rate of Technical Progress
The rate of technical progress denotes the effect of saving inputs per unit output in the assessment period. The derivation of the index is generally introduced as follows. Consider the following form of the general production function: The differential form of (10) is obtained as follows: where that is the growth rate of outputs and is the input growth rate of the element . Then the definition of rate of technical progress is
2.4.2. Technical Contribution to Output Growth
The technical contribution to output growth is the proportion of the rate of technical progress in the output growth speed. The definition of this index is represented as follows: where is the actual output growth speed.
2.4.3. Technical Merit
The technical merit index indicates the technological level of the smart grid and it can be measured by the following form: where is the growth rate of technical progress at the time point .
3. Application
In this section, the application of production function theory combined with DEA and RRA will be implemented. Figure 1 shows the overall evaluation process, in which the technological level of smart grid technologies and the technological progress of the smart grid can be displayed by means of the selected evaluation indices.
The smart grid associated with a group of various technologies, attributes, and objectives covers comprehensive construction, where major breakthroughs in key technology and equipment should be achieved. It is a challenge to evaluate the technical level of a smart grid considering all concerns. Thus, it is suitable to select a specific attribute or goal of smart grid to study in detail. One of the representative objectives is integrating more clean energy, including solar and wind energy, into electric power grids, which is also taken as a classic example to implement the evaluation of technology level and technological progress for the integration of clean energy in this paper. The integration of largescale clean energy is an important part of smart grid technologies. Generally, the clean energy turbine technology, integration technology into power grids, bulk storage devices, and power forecast technology have a significant impact on the clean energy development. Specifically, a highly efficient clean energy turbine can reduce cost and improve reliability. The optimal operation and sustainable construction are regarded as an effective suggestion for the improvement of clean energy integrated into power systems. The flexible bulk storage devices and the power forecast technology may overcome the volatility of renewable energy, so as to promote the utility of the clean energy widely. Therefore, to adapt the development of clean energy in the smart grid, the inputoutput relationship of such objective and the development level of intelligent technologies will be analyzed deeply.
The data about the clean energy development plan in a regional power grid has been obtained in Table 1 which includes the forecast values of output and input variables in the next decade. The inputs contain labor and three kinds of investments which are the clean energy investments in capacities (CEIC), the bulk energy storage devices (BESD), and the construction investments of power grids (CIPG). The outputs include the reduced paying carbon taxes (RPCT), the benefits from the reduced fossil energy (BRFE), and the electricity sales of the clean energy (ESCE).
 
The units of measurements of investment and income 1, 2, and 3 are hundred million dollars. The unit of measurement of labor force is ten thousand people. 
According to the data about the clean energy development plan in Table 1, the feasibility of the production functions should be analyzed by the DEA technique. The DEA optimization model is solved by MATLAB optimization toolbox. The optimal solution of relative technical efficiency is gained and the performance of intelligent technologies can also be understood. Figure 2 shows the results of relative technical efficiency at the time points. It turns out that the relations between the inputs and outputs are DEA efficient in years 1, 2, 3, 5, 8, 9, and 10, respectively. Furthermore, it also illustrates that the technical efficiencies are available and the returns to scale are constant in these years. However, the ratio efficiency between the inputs and outputs is inefficient in years 4, 6, and 7. In order to identify the reason why the relative technical efficiency is not available, it is necessary to analyze the gaps of the outputs and inputs. The analysis will provide the suggestions of how to adjust the original outputs and inputs, so that the application of the production functions can be achieved. Figure 3 shows the state of slack variables of the inputs for power grids in the evaluation cycle. In Figure 3, it shows that the slack variables of investment 1, investment 2, and investment 3 are unequal to zero, which means that the investments are obviously in the idle states and the values of slack variables are equal to the idle quantity of the investments of power grids.
The parameter estimation is also a significant step in overall procedure of the technological progress evaluation. Due to the advantage of RRA that is capable of coping with the problem of collinearity between each variable in the production functions, the results of the estimated parameters will be more accurate using the RRA technique. Tables 2 and 3 show the estimated parameter values using the CD, CES, and translog production functions, respectively, with the scalar parameter of ridge regression , 0.13, and 0.2. All the numerical results in Tables 2 and 3 are calculated by the SPSS software based on the observed measures of the input and output historical data [20]. The results in Table 2 indicate that the economic scale is the constant returns to scale, owing to the equation of and . The results in Table 3 demonstrate that all the coefficients of the translog production function are positive, which means that the technical progress is neutral and the trend will be accelerating in the evaluation cycle. In addition, the results from SPSS show , 0.951, and 0.865 for the CD, CES, and translog production models, which indicates that the complex determined coefficients are highly significant. Consequently, these estimated parameters are reasonable and nearly conform to the actual condition of the smart grid as well.


Through the DEA examination analysis of multiple inputoutput variables and the parameter estimation of the production functions, the assessment of technological progress of the smart grid can be performed. The rate of technical progress, technical contribution to output growth, and technical merit, representing the indices of the technological progress level of the smart grid, can be calculated by the proposed production functions in the evaluation cycle. Figures 4 and 5 show the index results about the rate of technical progress and technical contribution to output growth, respectively. The results in Figure 4 indicate that the values of the rate of technical progress obtained from the translog production function are significantly smaller than other production functions. Except the results in the second year and the last year, the index values calculated by the CES and CD production functions are approximately the same. The results in Figure 5 demonstrate that the numerical values of the technical contribution to output growth calculated by the translog production function are distinctly less than the results obtained from others. Moreover, the difference between the translog production function and other production functions in the initial stage is slightly bigger than its later stage. With respect to the property of the rate of technical progress, the values calculated by the CES and CD production functions are also approximately uniform. Synthesizing the data analysis for Figures 4 and 5, we can generally summarize that the calculation results obtained from the translog production function tend to be rather conservative, while the calculation results of the CD and CES production functions display the optimistic property. Another point of view from the data analysis is that the calculation results obtained from the CES production function are close to the results of the CD production function. The main reason is that on one hand they have the similar function expression and on the other hand they own the same application condition that the environment is the constant returns to scale.
The index values of the technical merit are shown in Figure 6, and the results indicate that the technical merit of the smart grid is improved annually in the evaluation periods. The growth pattern of the technical merit is similar to an exponential function form. Due to the significant difference of the technical merit between the translog production function and other production functions, the numerical results of the translog production function are also less than the results of others. The technical merit results reveal that the technological level of smart grid technologies is enhanced yearly. It also means that intelligent technologies are generally used widely in the smart grid.
The indices of the rate of technical progress, technical contribution to output growth, and technical merit represent the intelligent properties of the smart grid. Especially, the data results in Figure 5 show that the technological proportion is from about 20% to 30% for the CD production function, which illustrates that the revenues from the intelligent technology are less than the investment and labor. It is necessary for managers to take effective measures to improve the intelligent level of the smart grid, furthermore promoting the extensive application of intelligent technologies.
As for the proposed assessment models based on the production function approaches in this paper, a more important question is how to choose from these production functions to reflect the actual smart grid. In authors’ opinion, it seems that the CD production function could be used to evaluate the technological progress more properly in most cases. The reason is that the CES and translog production functions require more complex computation to estimate their parameters, which may result in the deterioration of calculation precision. Although a critical application condition of the CD production function is limited to constant returns to scale, most actual situations in not only power systems but also other industries are thought to submit to it in general. However, if the decisionmaker tends to implement more complex and detailed analysis to evaluate the technological progress, the CES and translog production functions will be recommended. Under the same conditions, CD production function can be applied more widely and conveniently. Under some special circumstances, it should be noted that different production models have their corresponding and unique applied scopes, which will be discussed in the next section.
4. Discussion
As to the mathematical formulations of the production functions, the structure of the CD production function is similar to the CES production function and they both have the unique inputs such as technology, capital investment, and labor. Moreover, the CD and CES production functions can be widely used because of the simple expressions. However, for the translog production function, though it has more complex mathematical representation than other production functions, the law of restricting the fixed inputs and the constant elasticity of substitution may not be complied. Therefore, the translog production function can be applied on more academic areas considering more and more nonroutine factors. For example, the environmental factor is an important objective for smart grid development. Supposing the additional load demand as an environmental factor is contained in the new translog production function, the specific formulation is introduced as follows: where is the additional load demand.
The additional load demand can be approximately forecasted according to the smart grid plans, technological innovation, and policy orientations. In [21], the forecast results of the additional load demand are given. Based on the data of the forecast additional load demand, the index values of technical contribution to output growth obtained from the different production functions can be shown in Figure 7. Translog production function I includes the additional load demand factor, while translog production function II is the original form of the translog production function previously mentioned. The difference from the results indicates that the influence of additional load demand cannot be ignored. In other words, the additional factors may conduct the different index results to represent technological level; consequently, it is necessary to focus on the impact of the various factors on the technological progress assessment of the smart grid.
For the proposed methodology to evaluate the technological progress of smart grids, it describes an empirical relationship between specific output and inputs for a power grid. In the modeling process, the production functions are used to represent the output production generated from investment and labor inputs, as well as technology. For application to production function approaches, we assume that the input variables include , , and and the output variables are and . Hence, the technological progress of smart grids can be measured by this measure that is a parametric method in operations research and economics for the estimation of power system production. In addition, DEA technique as a nonparametric method is used to select the optimal inputs and output of production functions. DEA is a preprocessing using, underlying the application to production function approaches. The evaluation of technological progress proposes such mathematical problem that is a timeseries estimation of the production state of smart grids based on multiple inputs/outputs in power system planning and operation models. In the solving procedure, the following properties of the evaluation framework can be obtained. (i) The models approached by production functions are built on the assumption of DEA availability. (ii) Besides the input variables , , and determined by DEA, the parameters and also have the impact on the results of technology assessment. (iii) Data for a portion of the technological progress evaluation can provide the primary basis for exploration of the production function model, while the data used to implement the assessment can play a significant role in the accurate estimation.
The evaluation framework can perform the technological progress assessment for smart grids from a macroview. Moreover, we assume that technical contribution to output growth represents technological progress in evaluation models. Different from the commonly evaluation methods, such as comprehensive assessment approach and costbenefit analysis, this paper presents a novel evaluation model based on the parametric and nonparametric estimation methods to implement technologybased assessment for smart grids. The proposed methodologies can be used to evaluate the effects of the adopted intelligent technologies in smart grid construction, which is helpful to direct the future power system planning and operation.
5. Conclusion
This paper presents the evaluation methodology to measure the technological progress of the smart grid based on production function theory. The proposed method is mathematically formulated to analyze the relationship between multiple input and output variables of the smart grid. In the evaluation process, the DEA test is regarded as an important step to ensure the application condition of the production functions in economic law. The indices representing the technological progress characteristic of the smart gird are obtained from the adopted CD, CES, and translog production. Moreover, the simulation results in case studies indicate that the tendency of technological levels is generally improved. The comparison analysis about the different production functions is performed in discussion, from which the application scope, modeling mechanism and engineering value of the production function can be understood. Finally, this study is a first strategic approach for the evaluation of technological progress of the smart grid from the macro view.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
The authors would like to thank the State Energy Smart Grid R&D Center (Shanghai) for the technical support. In addition, this work was supported by the National Natural Science Foundation of China (Grant no. 51377103).
References
 J. D. McDonald, “The nextgeneration grid : EEEnergy infrastructure of the future,” IEEE Power and Energy Magazine, vol. 7, no. 2, pp. 26–94, 2009. View at: Publisher Site  Google Scholar
 M. Amin, “Toward selfhealing energy infrastructure systems,” IEEE Computer Applications in Power, vol. 14, no. 1, pp. 20–28, 2001. View at: Publisher Site  Google Scholar
 W. Wakefield, “Methodological approach for estimating the benefits and costs of smart grid demonstration projects,” U.S.ERPI, Palo Alto, Calif, 2010. View at: Google Scholar
 Smart grid characteristics, values, and metrics, http://energy.gov/.
 U. S. Department of Energy, Smart Grid System Report, U. S. DOE, Washington, DC, USA, 2009.
 European Commission, “European technology platform smart grids: vision and strategy for Europe’s electricity networks of the future,” European Commission, 2010. View at: Google Scholar
 F. Y. Lo, C. F. Chien, and J. T. Lin, “A DEA study to evaluate the relative efficiency and investigate the district reorganization of the Taiwan power company,” IEEE Transactions on Power Systems, vol. 2, no. 16, pp. 170–178, 2001. View at: Google Scholar
 L. D. Otero, G. Centeno, C. E. Otero, and K. Reeves, “A DEAtobit analysis to understand the role of experience and task factors in the efficiency of software engineers,” IEEE Transactions on Engineering Management, vol. 59, no. 3, pp. 391–400, 2012. View at: Publisher Site  Google Scholar
 T. Schoenherr and S. Talluri, “Environmental sustainability initiatives: a comparative analysis of plant efficiencies in Europe and the U.S.,” IEEE Transactions on Engineering Management, vol. 60, no. 2, pp. 353–365, 2013. View at: Publisher Site  Google Scholar
 B. Jayasekara and U. D. Annakkage, “Derivation of an accurate polynomial representation of the transient stability boundary,” IEEE Transactions on Power Systems, vol. 21, no. 4, pp. 1856–1863, 2006. View at: Publisher Site  Google Scholar
 B. Golany, Y. Roll, and D. Rybak, “Measuring efficiency of power plantsin Israel by data envelopment analysis,” IEEE Transactions on Energy Management, vol. 8, no. 41, pp. 291–301, 1994. View at: Google Scholar
 A. Pahwa, X. Feng, and D. Lubkema, “Performance evaluation of electric distribution utilities based on data envelopment analysis,” IEEE Transactions on Power Systems, vol. 17, no. 3, pp. 400–405, 2002. View at: Google Scholar
 M. Niu and Z. Xu, “Efficiency rankingbased evolutionary algorithm for power system planning and operation,” IEEE Transactions on Power System, vol. 29, no. 3, pp. 1437–1438, 2014. View at: Publisher Site  Google Scholar
 S. M. Hatefi, F. Jolai, and H. Kor, “A new model for classifying inputs and outputs and evaluating the DMUs efficiency in DEA based on CobbDouglas production function,” in Proceedings of the International Conference on Computer Engineering and Technology (ICCET '09), pp. 390–394, Singapore, January 2009. View at: Publisher Site  Google Scholar
 C. Xu and R. Pan, “Network infrastructure and economic growth,” in Proceedings of the International Conference on networking and Digital Society, Wenzhou, China, 2010. View at: Google Scholar
 J. Xu and N. Sun, “The impact of meteorological factors on manufacturing economy: a quantitative analysis based on translog production fuction,” in Proceedings of the International Conference on Remote Sensing, Environment and Transportation Engineering (RSETE '11), pp. 874–877, Nanjing, China, June 2011. View at: Publisher Site  Google Scholar
 R. Ayres and V. Voudouris, “The economic growth enigma: capital, labour and useful energy?” Energy Policy, vol. 64, pp. 16–28, 2014. View at: Publisher Site  Google Scholar
 M. Filippini, L. C. Hunt, and J. Zoric, “Impact of energy policy instruments on the estimated level of underlying energy efficiency in the EU residential sector,” Energy Policy, vol. 69, pp. 73–81, 2014. View at: Google Scholar
 Y. Zhou, S.C. Huang, and M. Bergsneider, “Linear ridge regression with spatial constraint for generation of parametric images in dynamic positron emission tomography studies,” IEEE Transactions on Nuclear Science, vol. 48, no. 1, pp. 125–130, 2001. View at: Publisher Site  Google Scholar
 Shanghai Municipal Electric Power Company, The Report for Science and Technology of Shanghai Power Grids in the Twelfth FiveYear Plan, SMEP, Shanghai, China, 2010.
 Z. Hu, B. Shan, and X. Han, The Prospects of Electric Power Demand in China, CEPP, Beijing, China, 2010.
Copyright
Copyright © 2014 Dong Han and Zheng Yan. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.