The Methodology of Doppler-Derived Central Blood Flow Measurements in Newborn Infants

de Waal, Koert A.

doi:https://doi.org/10.1155/2012/680162

International Journal of Pediatrics

On this page

Abstract Introduction References Copyright Related Articles

Review Article | Open Access

Volume 2012 | Article ID 680162 | https://doi.org/10.1155/2012/680162

The Methodology of Doppler-Derived Central Blood Flow Measurements in Newborn Infants

Koert A. de Waal^1,2

Academic Editor: Doff B. McElhinney

Received06 Jul 2011

Accepted04 Oct 2011

Published16 Jan 2012

Abstract

Central blood flow (CBF) measurements are measurements in and around the heart. It incorporates cardiac output, but also measurements of cardiac input and assessment of intra- and extracardiac shunts. CBF can be measured in the central circulation as right or left ventricular output (RVO or LVO) and/or as cardiac input measured at the superior vena cava (SVC flow). Assessment of shunts incorporates evaluation of the ductus arteriosus and the foramen ovale. This paper describes the methodology of CBF measurements in newborn infants. It provides a brief overview of the evolution of Doppler ultrasound blood flow measurements, basic principles of Doppler ultrasound, and an overview of all used methodology in the literature. A general guide for interpretation and normal values with suggested cutoffs of CBFs are provided for clinical use.

1. Introduction

Central blood flow (CBF) measurements are measurements in and around the heart. It incorporates cardiac output, but also measurements of cardiac input and assessment of intra- and extracardiac shunts. Schematically presented in Figure 1, CBF can be measured in the central circulation as right or left ventricular output (RVO or LVO) and/or as cardiac input measured at the superior vena cava (SVC flow). Intra- and extracardiac shunts like the foramen ovale, atrium or ventricular septal defects, or ductus arteriosus are measured in the central circulation to assist in interpreting RVO and LVO. Organ blood flow can be measured in the arterial blood vessels of the main organs. Peripheral blood flow, as part of the organ circulation, refers to blood flow in the skin, mucus membranes, or underlying muscle tissue.

As CBF mainly measures cardiac output and cardiac input, some basic principles of blood flow regulation need to be addressed. Although details of the circulation are complex, there are three basic principles that underlie all functions of the system [1]. (1)The blood flow to each tissue of the body is controlled in relation to the tissue needs. When tissues are active, regional flow can be increased up to 30 times the resting level. Cardiac output can only be increased several times, so blood flow must also be controlled at the local level, which will result in redistribution of total blood flow. Only the increase in total blood flow can be measured with CBF measurements, not the level of distribution in the organ circulation.(2)Cardiac output is controlled mainly by the sum of all the local tissue flows. The heart acts as an automaton; cardiac input determines cardiac output. It facilitates the use of cardiac input measurements as representative for cardiac output. The input = output response is not always sufficient, with the autonomic nervous system playing an important role in maintaining adequate output in disease states by altering heart rate and peripheral vascular resistance.(3)Arterial blood pressure is controlled independently of either local blood flow control or cardiac output control. This will affect the physiological relationship between pressure and flow. Flow () through a blood vessel is determined by the pressure difference () and the vascular resistance (), Ohms’ law, Resistance is proportional to vessel radius (), vessel length (), and blood viscosity (), where viscosity is mainly determined by blood haemoglobin content. The expression for resistance can be combined with the equation describing the relationship between flow, pressure, and resistance in the Poiseuille’s equation: In this equation, the resistance of a vessel is in proportion to the fourth power of its radius, making the vessel diameter the most contributing factor to blood flow.

2. Doppler Ultrasound

CBF can be measured using several techniques, for example, using oxygen consumption as a determinant of flow (using the Fick principle), dilution methods using dye or hot or cold fluid boluses, pulse pressure methods, Doppler ultrasound, or velocity-encoded phase contrast MRI. An extensive review of available methods and their respective advantages and limitations is published elsewhere [2]. The Doppler principle as applied to ultrasound describes the shift in frequency of the returning sound wave in proportion to the velocity of the object imaged. Doppler ultrasound is used to detect and measure blood flow with the major reflector being the red blood cell. Velocity of moving blood () is then calculated using the Doppler shift (), insonating frequency (), speed of sound (), and the angle of insonation (, the angle between the sound waves and direction of moving blood) in the Doppler equation: Appropriate angle of insonation (also called Doppler angle) is essential for accurate determination of Doppler shift and blood flow velocity, with an increasing angle causing progressive underestimation of flow velocity.

Blood flow throughout most of the central circulation is laminar, characterised by a flow profile that is parabolic. This occurs in long, straight blood vessels under steady flow conditions. The practical implication of parabolic laminar flow is that when flow velocity is measured using pulsed Doppler, the velocity represents the average velocity of the cross-section of the vessel. Plotting the average velocity against time allows for calculation of the velocity time integral (Vti), the area under the velocity envelope.

Blood flow () is calculated using the following parameters: Diameter () to calculate the cross-sectional vessel area () assuming that the vessel area is round, flow velocity (Vti), and heart rate. The following formula is used to calculate flow in mL/kg/min:

The diameter can be measured using two-dimensional (2D) images from one or two ultrasound imaging planes, or one can use the M mode technique. M mode records motion of tissue toward and away from the transducer and has the advantage of producing a clear delineation of vessel walls. The disadvantage of M mode is the potential of tangential cuts through the vessel, producing an overestimation of vessel diameter.

Doppler can also be used to estimate pressure. The pressure difference () over a cross-sectional area can be calculated by using the maximum velocity () in the modified Bernoulli equation: With laminar flow, there is a linear relationship between perfusion pressure and flow. This relationship weakens if flow becomes turbulent, with more perfusion pressure needed for a given flow.

In general, Doppler measures of blood flow in the central circulation have been shown to have good inter- and intraobserver reproducibility and to be well correlated with invasive measures of blood flow using microspheres, dye dilution, or thermodilution techniques. Doppler was regarded as most useful when used to track changes [3].

Potential errors of measurements come from inaccurate measurement of vessel diameter due to poor views or tangentially performed measurements, or inaccurate measures of Vti due to a high angle of insonation or placement of the Doppler gate out of the laminar stream.

Measuring CBF in newborns using Doppler ultrasound is performed in 2 steps. First, image acquisition (usually at the bedside) followed by image analysis using the incorporated software of the ultrasound equipment. Image acquisition in newborn infants requires a high-resolution ultrasound (US) machine and high-frequency Doppler probe with colour mode incorporated. Image analysis is the manual tracing (Vti’s) and determination of distances using callipers (vessel diameter and heart rate) for the calculation of CBF.

Since the introduction of US in cardiac medicine in the late seventies, commercial US machines are now in its fourth generation. First-generation US equipment included only M mode measurements of ventricular dimensions and unguided Doppler measurements using a pencil probe. A survey by Sahn in 1978 suggested that major problems existed with M mode interobserver variability [4]. It was difficult to compare echocardiographic data from one laboratory compared with the results from another due to differences in timing of M mode measurements (e.g., at onset or at peak of QRS) and in actual distances taken (e.g., leading to leading edge or trailing to leading edge dimensions). The paper presented new recommendations for uniform M mode measurements. When 2D imaging became widely available in the late seventies (second-generation US equipment), this led to an increase in publications describing the use of cardiac ultrasound and central blood flow measurements. A report of the American Society of Echocardiography Committee on Nomenclature and Standards in 2D Echocardiography brought uniformity in transducer location, imaging planes, and image orientation standards [5].

Further refinement of US equipment included higher spatial resolution with increased Doppler frequency and a greater variety in size and shape of US probes (third-generation US equipment) and development of software for advanced measurements like tissue Doppler, strain rate, 3D visualisation with miniaturisation of US probes, and US equipment (fourth-generation US equipment). It is clear that early pioneer work is invaluable; however, we must realise that the normal values produced with first- and second-generation US equipment are difficult to compare to values produced with current-generation US equipment.

3. Methodology of Doppler Derived Central Blood Flow Measurements

In newborn infants, CBF is measured at 3 sites: right ventricular output (RVO) at the pulmonary valve, left ventricular output (LVO) at the aortic valve, and flow returning to the heart in the superior vena cava (SVC) measured at the point where the SVC starts to open up into the right atrium [6–8]. As mentioned before, additional sites of measurements help in correct interpretation of the CBF values. Systemic blood flow is defined as the proportion of central blood flow directed to the organs of the body, and pulmonary blood flow as the proportion of CBF directed to the lungs. RVO has traditionally been associated with pulmonary blood flow and LVO with systemic blood flow. However, the presumed associations of either parameter are not true if large shunts are present. Referring back the basic principles of cardiac output regulation, LVO reflects all pulmonary venous blood returning to the left side of the heart (systemic blood flow and ductal left-to-right shunt) and RVO is the resultant of cardiac input and the often left-to-right shunt over the foramen ovale. At a time when shunting is prevalent, usually in the first few days of life (early transition), RVO would better reflect systemic blood flow and LVO would better reflect pulmonary blood [9]. Once significant shunting no longer takes place, LVO would represent systemic blood flow and RVO pulmonary blood flow. SVC flow, as a measure of cardiac input, could partly reflect systemic blood flow, as it reflects blood returning from the upper body and brain, but not blood returning from the lower part of the body. Kindly check the enumeration of all sections and subsections. Please check.

This paper will discuss the methodology of Doppler-derived LVO, RVO, SVC flow, ductal assessment, foramen ovale assessment, and flow in the descending aorta (DAo). Unfortunately, blood flow in the inferior vena cava (IVC) is difficult to measure via a transthoracic approach due to the small common confluence and its anatomical position for flow velocity determinations. The same can be said for pulmonary venous blood flow.

3.1. Left Ventricular Output (LVO)

LVO diameter is obtained in the parasternal long axis view, and flow velocity from the subcostal to apical view or the high suprasternal view. LVO measurements using Doppler ultrasound in newborns were first published by Alverson et al. [10]. They used M mode to determine the diameter of the ascending aorta and a high suprasternal continuous wave Doppler position to obtain LVO flow velocity in 8 preterm and 14 term newborns in the first week of life. They reported a mean (SD) LVO of 221 (56) and 236 (47) mL/kg/min, respectively. Walther et al. investigated a larger group of term and preterm infants, using slightly different methodology [6]. The leading edge technique was used instead of the trailing edge technique (see Figure 2 for details). Comparable LVO values were reported for term infants, and higher mean (SD) values of 260 (35) mL/kg/min were found for preterm infants. For clinical use, 325 and 200 mL/kg/min were reported to be used as upper and lower limits of normal. Mandelbaum-Isken introduced the use of the apical window for flow velocity determinations in newborn infants [11]. A much lower LVO of 150 (40) mL/kg/min was found using the apical window and the (smaller) aorta annulus in 18 healthy term newborns. Most current research on LVO in preterm infants uses the methodology and proposed lower limit of 150 mL/kg/min for clinical use as described by Evans and Kluckow [7]. They used the internal diameter from 2D images of the ascending aorta and the apical window for flow velocity to calculate LVO. Mellander et al. compared the different methods to measure the LVO diameter, including 2D measurements at the aortic annulus [12]. They found that aortic root diameters often overestimated and 2D aortic annulus underestimated LVO as compared to thermodilution. The aortic root is an area stretching from the aortic annulus to the proximal ascending aorta, including the sinuses of Valsalva and the supra-aortic ridge. M mode measurements do not prespecify what area is actually measured. If the recommendations by Sahn et al. [4] are followed, then is it likely that the widest area at the sinuses of Valsalva is measured. There are considerable differences in diameter of the aortic annulus and sinuses of Valsalva in children and adults but limited data is available for the newborn population [13, 14]. In preterm infants, the aorta annulus is approximately 0.9 mm smaller than the ascending aorta, decreasing the LVO by 100 mL/kg/min [15].

The methodology of Doppler determination of LVO and the reported values in newborn infants is presented in Table 1. Variation exists in diameter location (ascending aorta versus aortic root or aorta annulus), in methodology of diameter determination (M mode versus 2D, leading edge versus trailing edge technique), in Doppler method (continuous Doppler, CW versus pulse wave Doppler, PW), and in location of flow velocity determinations (suprasternal versus subcostal or apical). Most studies did not use angle correction for flow velocity. However, the anatomic position of the left ventricular outflow tract in newborn infants is seldom truly aligned (in 3D geometry) with any apical or subcostal view, creating an underestimation of the true LVO.

The accuracy of LVO measurements as compared to the Fick method, thermodilution or dye dilution varies between 1 and 36% [3]. Hudson et al. evaluated intra- and interobserver agreement in 20 healthy term infants, using 3 different methods for aortic diameter measurements and 3 different sites of measuring blood flow velocity with both continuous wave and pulsed Doppler [26]. In this study, the most reproducible determination of cardiac output was found when the suprasternal site with continuous wave Doppler was used for measurement of blood flow velocity and M mode trailing edge technique was used for diameter.

More recently, Tsai-Goodman et al. reported on repeatability of measures of LVO in term and preterm infants with no significant differences within or between observers for any of the parameters required to measure LVO [22]. It is possible that improved image quality of current US equipment also improved intra- and interobserver agreement.

3.2. Right Ventricular Output (RVO)

RVO diameter and flow velocity are obtained by a true parasternal view, looking slightly upwards to align with the right ventricular outflow tract. Doppler determinations of RVO in newborns were first published in 1987 by Takenaka and Sholler, but did not receive much clinical attention until later reports by Evans et al. [7, 27, 28]. The right ventricular outflow tract and pulmonary valve lie very close to the anterior chest wall, making Doppler measurements easy. Most studies use the parasternal view to visualise the pulmonary annulus for diameter determinations in end systole, using the hinges of the pulmonary valve as reference point. Flow velocity can be measured using the same view. Some investigators use the short-axis view to obtain the same parameters (Table 2). Diastolic flow and/or turbulent flow from ductal shunting can make flow velocity measurements difficult, as precise tracing of the pulmonary waves is not always possible. The accuracy of Doppler RVO is not known, as there are no publications comparing Doppler RVO versus other methods of right-sided cardiac output determinations. The intra- and interobserver agreement has been determined by Tsai-Goodman et al. with the major difference found being measurements of the pulmonary outflow tract diameter [22]. Intraobserver repeatability was 4%, 7.5%, and 9%, respectively, for measurements of the hinges of the pulmonary valve, pulmonary trunk, and right ventricular outflow tract. There were significant differences between observers for measurement of the pulmonary trunk and right ventricular outflow tract, but not for the hinges of the pulmonary valve. The mean RVO was 255 mL/kg/min with a mean difference between observers of only 0.3 mL/kg/min (95% CI: −24.1 to 23.4 mL/kg/min).

3.3. Superior Vena Cava Flow (SVC Flow)

SVC flow is a relative new method of measuring central blood flow. It measures blood flowing back to the heart from the upper body and brain and is not influenced by atrial or ductal shunting. Its use in newborns was first described by Tamura et al. [31]. They sequentially measured 17 healthy term infants in the first day of life, exploring the maximum venous flow velocity during ventricular systole (S wave) and diastole (D wave). Five to 7 cycles in expiration were used to average the flow velocity. They did not measure SVC diameters. The most used methodology for measuring SVC flow is the methodology presented by Kluckow and Evans [8]. They measured SVC flow in 25 preterm and 14 term infants, using the high parasternal view rotated towards the true sagittal plane for diameter measurements. SVC image acquisition can be difficult, especially in spontaneous breathing infants. It is especially important to obtain the full diameter, as the SVC can “hide” behind the ascending aorta. The minimum and maximum diameters were taken at the point where the SVC starts to open up into the right atrium and averaged from 3 to 5 cardiac cycles. Flow velocity was measured from the low subcostal view with the probe directed towards the SVC. Since SVC flow is venous flow, the beat to beat variability is of importance. Spontaneous respiration will influence flow velocity, therefore it is advised to take at least 10 to 15 cycles to average flow velocity Table 3.

Kluckow and Evans reported that in infants with a closed duct, where LVO and RVO equate to systemic blood flow (assuming there is no significant FO shunt), SVC flow was an average 37% of LVO [8]. The median intraobserver variability for SVC flow measurement was 8.1%, and the median interobserver variability between the measurements was 14%. Measurement of velocity time integral (median variability 7.4%) and diameter (median variability 8.7%) contributed more to the variability than heart rate (median variability 1.8%). Groves et al. showed comparable findings on intra- and interobserver variability [32]. Lee et al. investigated the image quality and intra- and interobserver agreement of SVC flow. Reliable diameter images were obtained in 85% and velocity recordings in 81% of the patients, reflecting difficulties in image acquisition [33]. The mean variability of SVC flow in this study was 17% in the intraobserver analysis and 29% in the interobserver analysis.

3.4. Assessment of the Ductus Arteriosus (DA)

The ductus arteriosus is a pulmonary to systemic shunt in fetal life, where it carries most of the RVO. Shortly after birth, the shunt reverses due to an increase in systemic vascular resistance (release from the low-resistance placental circulation) and a decrease in the pulmonary vasculature resistance (lung inflation). The shunt becomes systemic to pulmonary (left to right, LR) as long as systemic pressure is higher than the pulmonary pressure throughout the cardiac cycle. Normally the DA closes soon after birth and the shunt disappears, but this process is often delayed in very preterm and sick newborns.

Ductal shunting will influence central blood flow, mainly LVO, as most of the volume of shunt will be directed left to right. With significant right-to-left (RL) shunt, often due to high pulmonary vascular resistance and hence decreased pulmonary blood flow, ductal shunting can be associated with reduced LVO but with a normal venous return from the lower body.

It is important to include ductal assessments in central blood flow measurements to be able to interpret the findings. Ductal assessment should include at least ductal diameter, maximum LR flow velocity () and flow pattern (continuous, pulsatile, bidirectional including % RL shunt, that is, the amount of time of the cardiac cycle blood flows right to left). Several other ultrasound measurements are suggested to help determine the degree of shunting. They include the ratio between the dimensions of the left atrium and the aorta (LA/Ao ratio) [34], left pulmonary artery diastolic velocity (LPAd) [35], and measuring the flow pattern of the descending Aorta (DAo) [24], the cerebral arteries [36, 37], or the abdominal organ arteries [38–40].

Ductal diameter is probably the most important parameter to determine the degree of ductal shunting. As with any flow, the diameter will have the largest impact on the amount of flow. Commonly, the duct is wide on the aortic side with constriction starting at the pulmonary site of the duct. To capture this aspect, one should visualise the whole trajectory of the duct. Most investigators measure the DA diameter from the high left parasternal view, with optimised colour flow Doppler mapping scale and gain settings. Care should be given to prevent colour spill from excess gain. The minimum diameter (site of maximal constriction) of the colour flow jet closest to the entry to the main pulmonary artery is then analysed through frame by frame analysis, and the diameter is taken at the clearest appearance in end systolic frames [41, 42]. The coefficient of variation using this methodology was 12% [41]. As the colour jet of a ductus arteriosus will widen out into the pulmonary trunk, it is essential to locate the site of maximal constriction.

Table 4 shows studies investigating ductal diameter in a wide variety of preterm infants and the ranges found. Median diameter is dependent on postnatal age with earlier measurements (within 12 hours of life) usually showing larger diameters.

With current generation ultrasound equipment, it has become increasingly easy to measure the internal diameter of the duct using 2D images. The short axis view to measure ductal diameter is not preferred as it will not always visualise the whole trajectory of the duct and commonly visualise the (wider) ductal jet without showing the site of maximal constriction. Constricting ducts can change shape and show as tortuous or kinked, making it difficult to find the site of maximal constriction.

For interpretation of central blood flow measurements, a ductal diameter greater than 1.5-1.6 mm can decrease SVC flow during transition, increase LVO, and decrease flow in the descending aorta [24, 42, 44, 46]. Evans and Kluckow evaluated the effect of various cardiorespiratory factors on RVO and LVO in 120 ventilated preterm infants [7]. A significant ductal shunt resulted in an increased LVO to RVO ratio. Up to 37% of ventilated preterm infants had suboptimal systemic blood flow which would not be detected if only LVO was measured.

Doppler evaluation at the point of DA diameter measurement will provide the ductal flow pattern. It is important to measure velocity and pattern in the duct itself at its narrowest point and not near the aortic side of the duct or in the ductal jet in the main pulmonary artery. A ductal flow pattern can be left to right (LR), bidirectional (BD), or right to left (RL) depending on the pressure difference between the two ends. Pure RL shunt is always pathological in newborn infants, but a small degree of BD shunting is normal shortly after birth [56]. When the pattern is bidirectional, the proportion of the cardiac cycle with right-to-left shunting could be measured as the time of right-to-left shunting divided by the total length of the cardiac cycle. Assuming that the systemic pressure is normal, a RL ductal shunt percentage of more than 30% is often considered significant pulmonary hypertension.

Su et al. classified ductal Doppler flow patterns on visual appearance into 5 categories; pulmonary hypertension pattern, growing pattern, pulsatile pattern, closing pattern, and closed pattern [57]. Interobserver agreement to classify the patterns was not tested. The pulsatile pattern showed the highest specificity (100%) and sensitivity (93.5%) to predict a persisting DA with clinical signs. The closing pattern, commonly with a flow velocity > 200 cm/s and a continuous appearance, was associated with constriction.

True ductal flow incorporating ductal diameter and Vti’s has been performed in a few studies [58, 59]. Although ductal flow is initially laminar, constriction will often change the flow to turbulent making representative tracings of flow velocity difficult.

3.5. Assessment of Shunt over the Foramen Ovale (FO)

The FO is an area in the midportion of the atrial septum concerning the fossa ovalis. In fetal life is has a function as divider for blood into the right or left side of the heart. The edge of the atrial septum (crista dividends) divides the incoming flow in two arms. Flow from the ductus venosus is diverted predominantly in the left atrium, and blood flow from the inferior vena cava enters the right atrium [60]. After birth, when atrial and ventricular pressures change, the valvular structure has the potential to close the defect. In the first few days of life, a shunt over the FO is common. It shows a dominant left-to-right direction and a bidirectional flow pattern [53].

The atrial septum can be imaged from a subcostal four-chamber view, adding colour flow Doppler mapping with colour scale setting for low velocities to assess shunts across the septum. The diameter can be measured using the color flow jet across the septum [54] or by using 2D images [54, 55]. The pulsed wave Doppler gate is placed in the interatrial shunt at the level of the atrial septum to determine flow direction and flow velocity. The pattern of flow should then be classified as left to right, bidirectional, or right to left. When the pattern is bidirectional, the proportion of the cardiac cycle with right-to-left shunting could be measured as the time of right-to-left shunting divided by the total length of the cardiac cycle as described for ductal flow patterns.

An FO shunt can influence central blood flow measurements, with its main effect increasing RVO. Evans and Iyer reported studies in 51 ventilated preterm infants with an increasing RVO to LVO ratio if the FO diameter exceeded 3 mm [53, 61]. Only few of these large FO shunts persisted during admission, making the FO shunt less likely to influence central blood flows compared to ductal shunting.

The natural course and closure of the FO is influenced by the presence of a DA and gestational age. In a study by Riggs et al. evaluating the natural course of an FO in 80 term and preterm infants, younger gestational age was associated with delayed closure and the presence of a DA at the time of initial diagnosis of an FO was strongly associated with earlier closure of the FO [55]. Table 5 shows studies investigating the FO in newborn infants.

3.6. Descending Aorta Flow (DAo Flow)

To facilitate measurements of blood flow in lower body, Groves et al. designed a method to measure blood flow in the descending aorta, just proximal to the diaphragm [32]. Flow velocity was measured from a subcostal sagittal view and from the high parasternal view with PW Doppler, with the use of angle correction. Reverse flow was deducted from forward flow to create total DAo flow. The diameter of the descending aorta was measured using M mode trailing edge technique in the parasternal short axis view. Intra- and interobserver variability were 14% and 11%, respectively, with the subcostal approach showing better repeatability. Normal values were determined in 14 healthy preterm infants and 13 term infants. Median (range) DAo flow at 24 hours of age was 180 (93–233) for term infants and 133 (108–305) mL/kg/min for preterm infants. In a larger preterm population of less than 31-week gestation infant, the median DAo flow was 145 (29–255) mL/kg/min. DAo flow reversal showed significant correlations with an increased ductal shunt [24].

3.7. LVO to RVO Ratio

The LVO to RVO ratio can describe if significant shunting takes place between systemic and pulmonary system. The ratio does not discriminate where the shunt takes place (atrial or DA) and measurement errors can be multiplied when using ratios. Normal Doppler-derived LVO to RVO ratio in preterm infants is not always one, even if shunts are not present. The anatomical positions of both outflow tracts are not always measurable with less than 20° angle leading to a variable amount of underestimation of the true cardiac output. The angle is usually greater for LVO then it is for RVO.

4. Central Blood Flow Measurements: Clinical Use and Interpretation

Central blood flow measurements reflect global cardiac function (preload, contractility, and afterload) by measuring total blood flow through the pulmonary and/or systemic circulation. Blood flow reflects the transport of oxygen to the tissues and is determined by the demand. Often, all central blood flow parameters are high or all parameters are low. It is preferable to measure central blood flow at more than one site as a cross-check to exclude measurement errors. If a single parameter is low or high, then further exploration should take place to find the cause. This should include a full sequential segmental chamber analysis to rule out structural abnormalities and evaluation of shunts and ventricular failure with severe dilatation and subsequent obstruction of an outflow tract as found in severe pulmonary hypertension or prolonged mechanical ventilation using high distending pressures. Central blood flow measurements are probably most informative if followed over time.

Research in neonatal hemodynamics has extensively studied the transitional circulation using CBF measurements. For a review of the current concepts of transitional circulation, I refer to excellent articles elsewhere [62–65]. In summary, blood flow decreases and blood pressure rises in the first day after birth in healthy term and late-preterm infants. In contrast, very preterm infants show a rise in blood flow (and blood pressure) in the first week of life. In sick and immature infants, blood pressure and/or blood flow often show a decrease in the first hours after birth with its nadir at 5 to 12 hours after birth. It has been shown that this period of low systemic blood flow at the most vulnerable period is associated with mortality and poor neurodevelopmental outcome [66, 67]. Risk factors for developing low systemic blood flow are a very young gestation, steal from blood out of the systemic circulation via a large ductus arteriosus, the use of mechanical ventilation, and severe respiratory disease. Most studies use an SVC flow < 45 mL/kg/min or LVO or RVO < 150 mL/kg/min as the definition of low flow, with SVC flow the best studied parameter in relation to outcome. Table 6 presents mean and standard deviation of CBF values in term and preterm infants using the methodology as described with adjusted diameters for LVO for preterm infants at day 7 to 14 [7, 8, 15, 25, 68]. Table 7 provides suggested cutoffs for low and high central blood flow in preterm infants based on the available evidence.

5. Future Perspectives and Directions for Research

Central blood flow measurements are increasingly used in clinical neonatology [69]. Its importance in clinical decision making in specific situations has been described [70, 71], but some argue that its importance in clinical decision making is not sufficiently studied. However, many currently used imaging techniques and diagnostic tests in neonatal medicine (including blood pressure monitoring) have not been subjected to randomized trials to determine its efficacy on outcome [69]. CBF measurements, as all diagnostic methods, are used to increase insight in the physiology and pathology.

Hemodynamics is an important part of neonatal intensive care. Commonly used parameters to guide treatment (e.g., blood pressure, capillary refill, lactate) are poorly associated with blood flow, and trials have shown that circulatory support treatment does not always have the expected effect on central blood flow [65]. These findings indicate the need to continue to explore neonatal hemodynamics by measuring central blood flow in the NICU [70].

Future studies are needed to evaluate the methodological issues still surrounding CBF measurements. The accuracy of all CBF measurements is currently being evaluated by phase-contrast MRI [72]. For clinical purposes, the interrater variability in longitudinal measurements needs to be minimised. Most variability of Doppler CBF is caused by diameter measurements. Variability could be decreased if population based diameter percentiles are used in the formula to calculate flow, instead of actual diameter measurements. This approach will also decrease accuracy, but longitudinal changes have proven to be the best predictors of morbidity and mortality, not absolute values [45].

More research is needed to study the effect of treatment on blood flow. To date, only very few randomized trials have explored the effect of cardiovascular treatment (volume, inotropes, and inodilators) on blood flow [49, 73, 74]. Randomizing at-risk babies to 2 different treatment regimes and preventing cross-over or contamination might reveal further insight into how to treat cardiovascular compromise. One could also suggest randomization at different thresholds or a combination of pressure and flow thresholds. All trials should use standardised methodology of measuring CBF to be able to compare hemodynamic outcomes.

The key to any successful trial is measuring the right outcome (short and long term) and obtaining the right diagnosis of the cardiovascular problem at hand. For example, the definition of a hemodynamic significant ductus arteriosus varies enormously amongst randomized trials investigating treatment of the duct [75]. There is a wealth of information that can be obtained with functional echocardiography and central blood flow measurements. To achieve its potential, there is a need to move it to the point of care where it is performed by the attending neonatologist and not by consulting specialists or researchers. As with most diagnostic methods, findings to date have not demonstrated that functional echocardiography affects outcomes. However, its value as a tool for assessing the rapidly changing hemodynamic status is essential to ensure future research can be translated into clinical practice.

References

A. C. Guyton, C. E. Jones, and T. G. Coleman, Circulatory Physiology; Cardiac Output and Its Regulation, W. B. Saunders, Philadelphia, Pa, USA, 1973.
W. P. de Boode, “Cardiac output monitoring in newborns,” Early Human Development, vol. 86, no. 3, pp. 143–148, 2010.
View at: Publisher Site | Google Scholar
M. S. Chew and J. Poelaert, “Accuracy and repeatability of pediatric cardiac output measurement using Doppler: 20-year review of the literature,” Intensive Care Medicine, vol. 29, no. 11, pp. 1889–1894, 2003.
View at: Publisher Site | Google Scholar
D. J. Sahn, A. DeMaria, J. Kisslo, and A. Weyman, “Recommendations regarding quantitation in M-mode echocardiography: results of a survey of echocardiographic measurements,” Circulation, vol. 58, no. 6, pp. 1072–1083, 1978.
View at: Google Scholar
W. L. Henry, A. DeMaria, R. Gramiak et al., “Report of the American society of echocardiography committee on nomenclature and standards in two-dimensional echocardiography,” Circulation, vol. 62, no. 2, pp. 212–217, 1980.
View at: Google Scholar
F. J. Walther, B. Siassi, N. A. Ramadan, A. K. Ananda, and P. Y. Wu, “Pulsed Doppler determinations of cardiac output in neonates: normal standards for clinical use,” Pediatrics, vol. 76, no. 5, pp. 829–833, 1985.
View at: Google Scholar
N. Evans and M. Kluckow, “Early determinants of right and left ventricular output in ventilated preterm infants,” Archives of Disease in Childhood: Fetal and Neonatal Edition, vol. 74, no. 2, pp. F88–F94, 1996.
View at: Google Scholar
M. Kluckow and N. Evans, “Superior vena cava flow in newborn infants: a novel marker of systemic blood flow,” Archives of Disease in Childhood: Fetal and Neonatal Edition, vol. 82, no. 3, pp. F182–F187, 2000.
View at: Google Scholar
M. Kluckow and N. Evans, “Low systemic blood flow in the preterm infant,” Seminars in Neonatology, vol. 6, no. 1, pp. 75–84, 2001.
View at: Publisher Site | Google Scholar
D. C. Alverson, M. W. Eldridge, J. D. Johnson, M. Aldrich, P. Angelus, and W. Berman Jr., “Noninvasive measurement of cardiac output in healthy preterm and term newborn infants,” American Journal of Perinatology, vol. 1, no. 2, pp. 148–151, 1983.
View at: Google Scholar
V. H. Mandelbaum-Isken and O. Linderkamp, “Cardiac output by pulsed Doppler in neonates using the apical window,” Pediatric Cardiology, vol. 12, no. 1, pp. 13–16, 1991.
View at: Google Scholar
M. Mellander, K. Sabel, K. Caidahl, L. Solymar, and B. Eriksson, “Doppler determination of cardiac output in infants and children: comparison with simultaneous thermodilution,” Pediatric Cardiology, vol. 8, no. 4, pp. 241–246, 1987.
View at: Google Scholar
M. J. Roman, R. B. Devereux, R. Kramer-Fox, and J. O'Loughlin, “Two-dimensional echocardiographic aortic root dimensions in normal children and adults,” American Journal of Cardiology, vol. 64, no. 8, pp. 507–512, 1989.
View at: Google Scholar
M. L. K. Sheil, O. Jenkins, and G. F. Sholler, “Echocardiographic assessment of aortic root dimensions in normal children based on measurement of a new ratio of aortic size independent of growth,” American Journal of Cardiology, vol. 75, no. 10, pp. 711–715, 1995.
View at: Publisher Site | Google Scholar
A. Khan, A. Lakkundi, and K. A. de Waal, “Variability in measurement of the left ventricular outflow diameter and flow velocity,” in Proceedings of the 15th Congress of the Perinatal Society of Australia and New Zealand (PSANZ '11), Tasmania, Australia, April 2011.
View at: Google Scholar
H. Hirsimäki, P. Kero, O. Wanne, R. Erkkola, and Z. Makoi, “Doppler-derived cardiac output in healthy newborn infants in relation to physiological patency of the ductus arteriosus,” Pediatric Cardiology, vol. 9, no. 2, pp. 79–83, 1988.
View at: Google Scholar
P. Winberg, M. Jansson, L. marions, and B. P. W. Lundell, “Left ventricular output during postnatal circulatory adaptation in healthy infants born at full term,” Archives of Disease in Childhood, vol. 64, no. 10, pp. 1374–1378, 1989.
View at: Google Scholar
F. J. Walther, F. van Bel, and M. Ebrahimi, “Duplex versus unguided pulsed Doppler measurements of right ventricular output in newborn infants,” Acta Paediatrica Scandinavica, vol. 79, no. 1, pp. 41–46, 1990.
View at: Google Scholar
Y. Agata, S. Hiraishi, K. Oguchi et al., “Changes in left ventricular output from fetal to early neonatal life,” Journal of Pediatrics, vol. 119, no. 3, pp. 441–445, 1991.
View at: Google Scholar
V. H. A. Mandelbaum, D. C. Alverson, A. Kirchgessner, and O. Linderkamp, “Postnatal changes in cardiac output and haemorrheology in normal neonates born at full term,” Archives of Disease in Childhood, vol. 66, no. 4, pp. 391–394, 1991.
View at: Google Scholar
P. Pladys, E. Wodey, A. Beuchée, B. Branger, and P. Bétrémieux, “Left ventricle output and mean arterial blood pressure in preterm infants during the 1st day of life,” European Journal of Pediatrics, vol. 158, no. 10, pp. 817–824, 1999.
View at: Publisher Site | Google Scholar
B. Tsai-Goodman, R. P. Martin, N. Marlow, and J. R. Skinner, “The repeatability of echocardiographic determination of right ventricular output in the newborn,” Cardiology in the Young, vol. 11, no. 2, pp. 188–194, 2001.
View at: Google Scholar
M. Murase, A. Ishida, and T. Momota, “Serial pulsed Doppler assessment of early left ventricular output in critically ill very low-birth-weight infants,” Pediatric Cardiology, vol. 23, no. 4, pp. 442–448, 2002.
View at: Publisher Site | Google Scholar
A. M. Groves, C. A. Kuschel, D. B. Knight, and J. R. Skinner, “Does retrograde diastolic flow in the descending aorta signify impaired systemic perfusion in preterm infants?” Pediatric Research, vol. 63, no. 1, pp. 89–94, 2008.
View at: Publisher Site | Google Scholar
S. C. Sloot, K. A. de Waal, J. H. van der Lee, and A. H. van Kaam, “Central blood flow measurements in stable preterm infants after the transitional period,” Archives of Disease in Childhood, vol. 95, no. 5, pp. F369–F372, 2010.
View at: Publisher Site | Google Scholar
I. Hudson, A. Houston, T. Aitchison, B. Holland, and T. Turner, “Reproducibility of measurements of cardiac ouput in newborn infants by Doppler ultrasound,” Archives of Disease in Childhood, vol. 65, no. 1, pp. 15–19, 1990.
View at: Google Scholar
K. Takenaka, F. Waffarn, A. Dabestani, J. M. Gardin, and W. L. Henry, “A pulsed Doppler echocardiographic study of the postnatal changes in pulmonary artery and ascending aortic flow in normal term newborn infants,” American Heart Journal, vol. 113, no. 3, pp. 759–766, 1987.
View at: Google Scholar
G. F. Sholler, J. M. Celermajer, C. M. Whight, and A. E. Bauman, “Echo Doppler assessment of cardiac output and its relation to growth in normal infants,” American Journal of Cardiology, vol. 60, no. 13, pp. 1112–1116, 1987.
View at: Google Scholar
H. Shiraishi, H. Endoh, K. Ichihashi, T. Kuramatsu, S. Yano, and M. Yanagisawa, “Left and right ventricular cardiac outputs in early neonatal periods examined using Doppler echocardiography,” Journal of Cardiology, vol. 18, no. 4, pp. 1127–1137, 1988.
View at: Google Scholar
T. D. Yanowitz, A. C. Yao, K. D. Pettigrew, J. C. Werner, W. Oh, and B. S. Stonestreet, “Postnatal hemodynamic changes in very-low-birthweight infants,” Journal of Applied Physiology, vol. 87, no. 1, pp. 370–380, 1999.
View at: Google Scholar
M. Tamura, K. Harada, and G. Takada, “Changes in superior vena cava velocity patterns in normal neonates,” American Journal of Cardiology, vol. 81, no. 3, pp. 362–365, 1998.
View at: Publisher Site | Google Scholar
A. Groves, C. A. Kuschel, D. B. Knight, and J. R. Skinner, “Echocardiographic assessment of blood flow volume in the superior vena cava and descending aorta in the newborn infant,” Archives of Disease in Childhood: Fetal and Neonatal Edition, vol. 93, no. 1, pp. F24–F28, 2008.
View at: Publisher Site | Google Scholar
A. Lee, K. Liestø, E. Nestaas, L. Brunvand, R. Lindemann, and D. Fugelseth, “Superior vena cava flow: feasibility and reliability of the off-line analyses,” Archives of Disease in Childhood: Fetal and Neonatal Edition, vol. 95, no. 2, pp. F121–F125, 2010.
View at: Publisher Site | Google Scholar
P. Iyer and N. Evans, “Re-evaluation of the left atrial to aortic root ratio as a marker of patent ductus arteriosus,” Archives of Disease in Childhood: Fetal and Neonatal Edition, vol. 70, no. 2, pp. F112–F117, 1994.
View at: Google Scholar
H. Suzumura, A. Nitta, G. Tanaka, and O. Arisaka, “Diastolic flow velocity of the left pulmonary artery of patent ductus arteriosus in preterm infants,” Pediatrics International, vol. 43, no. 2, pp. 146–151, 2001.
View at: Publisher Site | Google Scholar
J. M. Perlman, A. Hill, and J. J. Volpe, “The effect of patent ductus arteriosus on flow velocity in the anterior cerebral arteries: ductal steal in the premature newborn infant,” Journal of Pediatrics, vol. 99, no. 5, pp. 767–771, 1981.
View at: Google Scholar
F. J. Weir, A. Ohlsson, T. L. Myhr, K. Fong, and M. L. Ryan, “A patent ductus arteriosus is associated with reduced middle cerebral artery blood flow velocity,” European Journal of Pediatrics, vol. 158, no. 6, pp. 484–487, 1999.
View at: Publisher Site | Google Scholar
M. Martinussen, A. M. Brubakk, D. T. Linker, T. Vik, and A. C. Yao, “Mesenteric blood flow velocity and its relation to circulatory adaptation during the first week of life in healthy term infants,” Pediatric Research, vol. 36, no. 3, pp. 334–339, 1994.
View at: Google Scholar
K. Maruyama, T. Koizumi, T. Tomomasa, and A. Morikawa, “Intestinal blood-flow velocity in uncomplicated preterm infants during the early neonatal period,” Pediatric Radiology, vol. 29, no. 6, pp. 472–477, 1999.
View at: Publisher Site | Google Scholar
M. Pezzati, V. Vangi, R. Biagiotti, G. Bertini, D. Cianciulli, and F. F. Rubaltelli, “Effects of indomethacin and ibuprofen on mesenteric and renal blood flow in preterm infants with patent ductus arteriosus,” Journal of Pediatrics, vol. 135, no. 6, pp. 733–738, 1999.
View at: Google Scholar
N. Evans and P. Iyer, “Longitudinal changes in the diameter of the ductus arteriosus in ventilated preterm infants: correlation with respiratory outcomes,” Archives of Disease in Childhood: Fetal and Neonatal Edition, vol. 72, no. 3, pp. F156–F161, 1995.
View at: Google Scholar
M. Kluckow and N. Evans, “Early echocardiographic prediction of symptomatic patent ductus arteriosus in preterm infants undergoing mechanical ventilation,” Journal of Pediatrics, vol. 127, no. 5, pp. 774–779, 1995.
View at: Publisher Site | Google Scholar
D. A. Roberson and N. H. Silverman, “Color Doppler flow mapping of the patent ductus arteriosus in very low birthweight neonates: echocardiographic and clinical findings,” Pediatric Cardiology, vol. 15, no. 5, pp. 219–224, 1994.
View at: Google Scholar
N. Evans and M. Kluckow, “Early ductal shunting and intraventricular haemorrhage in ventilated preterm infants,” Archives of Disease in Childhood: Fetal and Neonatal Edition, vol. 75, no. 3, pp. F183–F186, 1996.
View at: Google Scholar
M. Kluckow and N. Evans, “Low superior vena cava flow and intraventricular haemorrhage in preterm infants,” Archives of Disease in Childhood: Fetal and Neonatal Edition, vol. 82, no. 3, pp. F188–F194, 2000.
View at: Google Scholar
D. A. Osborn, N. Evans, and M. Kluckow, “Hemodynamic and antecedent risk factors of early and late periventricular/intraventricular hemorrhage in premature infants,” Pediatrics, vol. 112, no. 1, part 1, pp. 33–39, 2003.
View at: Publisher Site | Google Scholar
M. El Hajjar, G. Vaksmann, T. Rakza, G. Kongolo, and L. Storme, “Severity of the ductal shunt: a comparison of different markers,” Archives of Disease in Childhood: Fetal and Neonatal Edition, vol. 90, no. 5, pp. F419–F422, 2005.
View at: Publisher Site | Google Scholar
A. El-Khuffash, M. Higgins, K. Walsh, and E. J. Molloy, “Quantitative assessment of the degree of ductal steal using celiac artery blood flow to left ventricular output ratio in preterm infants,” Neonatology, vol. 93, no. 3, pp. 206–212, 2008.
View at: Publisher Site | Google Scholar
M. Paradisis, N. Evans, M. Kluckow, and D. Osborn, “Randomized trial of milrinone versus placebo for prevention of low systemic blood flow in very preterm infants,” Journal of Pediatrics, vol. 154, no. 2, pp. 189–195, 2009.
View at: Publisher Site | Google Scholar
M. Fukazawa, J. Fukushige, and K. Ueda, “Atrial septal defects in neonates with reference to spontaneous closure,” American Heart Journal, vol. 116, no. 1, part 1, pp. 123–127, 1988.
View at: Google Scholar
H. Hannu, K. Pentti, E. Henrik, S. Markku, and V. Ilkka, “Patency of foramen ovale—does it influence haemodynamics in newborn infants?” Early Human Development, vol. 20, no. 3-4, pp. 281–287, 1989.
View at: Google Scholar
S. Hiraishi, Y. Agata, K. Saito et al., “Interatrial shunt flow profiles in newborn infants: a colour flow and pulsed Doppler echocardiographic study,” British Heart Journal, vol. 65, no. 1, pp. 41–45, 1991.
View at: Google Scholar
N. Evans and P. Iyer, “Incompetence of the foramen ovale in preterm infants supported by mechanical ventilation,” Journal of Pediatrics, vol. 125, no. 5, part 1, pp. 786–792, 1994.
View at: Publisher Site | Google Scholar
D. G. Markhorst, E. Rothuis, M. Sobotka-Plojhar, and R. J. Moene, “Transient foramen ovale incompetence in the normal newborn: an echocardiographic study,” European Journal of Pediatrics, vol. 154, no. 8, pp. 667–671, 1995.
View at: Publisher Site | Google Scholar
T. Riggs, S. E. Sharp, D. Batton, M. E. Hussey, and E. Weinhouse, “Spontaneous closure of atrial septal defects in premature vs full-term neonates,” Pediatric Cardiology, vol. 21, no. 2, pp. 129–134, 2000.
View at: Publisher Site | Google Scholar
J. Skinner, “Diagnosis of patent ductus arteriosus,” Seminars in Neonatology, vol. 6, no. 1, pp. 49–61, 2001.
View at: Publisher Site | Google Scholar
B. H. Su, T. Watanabe, M. Shimizu, and M. Yanagisawa, “Echocardiographic assessment of patent ductus arteriosus shunt flow pattern in premature infants,” Archives of Disease in Childhood: Fetal and Neonatal Edition, vol. 77, no. 1, pp. F36–F40, 1997.
View at: Google Scholar
F. J. Walther, M. J. Benders, and J. O. Leighton, “Early changes in the neonatal circulatory transition,” Journal of Pediatrics, vol. 123, no. 4, pp. 625–632, 1993.
View at: Google Scholar
E. Z. Phillipos, M. A. Robertson, and P. J. Byrne, “Serial assessment of ductus arteriosus hemodynamics in hyaline membrane disease,” Pediatrics, vol. 98, no. 6, pp. 1149–1153, 1996.
View at: Google Scholar
T. Kiserud and G. Acharya, “The fetal circulation,” Prenatal Diagnosis, vol. 24, no. 13, pp. 1049–1059, 2004.
View at: Publisher Site | Google Scholar
N. Evans and P. Iyer, “Assessment of ductus arteriosus shunt in preterm infants supported by mechanical ventilation: effect of interatrial shunting,” Journal of Pediatrics, vol. 125, no. 5, part 1, pp. 778–785, 1994.
View at: Publisher Site | Google Scholar
D. A. Osborn, “Diagnosis and treatment of preterm transitional circulatory compromise,” Early Human Development, vol. 81, no. 5, pp. 413–422, 2005.
View at: Publisher Site | Google Scholar
M. Kluckow, “Low systemic blood flow and pathophysiology of the preterm transitional circulation,” Early Human Development, vol. 81, no. 5, pp. 429–437, 2005.
View at: Publisher Site | Google Scholar
N. Evans, “Assessment and support of the preterm circulation,” Early Human Development, vol. 82, no. 12, pp. 803–810, 2006.
View at: Publisher Site | Google Scholar
N. Evans, “Support of the preterm circulation: keynote address to the fifth evidence vs experience conference, Chicago, June 2008,” Journal of Perinatology, vol. 29, no. 2, pp. S50–S57, 2009.
View at: Publisher Site | Google Scholar
R. W. Hunt, N. Evans, I. Rieger, and M. Kluckow, “Low superior vena cava flow and neurodevelopment at 3 years in very preterm infants,” Journal of Pediatrics, vol. 145, no. 5, pp. 588–592, 2004.
View at: Publisher Site | Google Scholar
J. Miletin and E. M. Dempsey, “Low superior vena cava flow on day 1 and adverse outcome in the very low birthweight infant,” Archives of Disease in Childhood: Fetal and Neonatal Edition, vol. 93, no. 5, pp. F368–F371, 2008.
View at: Publisher Site | Google Scholar
K. de Waal and N. Evans, “Hemodynamics in preterm infants with late-onset sepsis,” Journal of Pediatrics, vol. 156, no. 6, pp. 918.e1–922.e1, 2010.
View at: Publisher Site | Google Scholar
N. Evans, V. Gournay, F. Cabanas et al., “Point-of-care ultrasound in the neonatal intensive care unit: international perspectives,” Seminars in Fetal and Neonatal Medicine, vol. 16, no. 1, pp. 61–68, 2011.
View at: Publisher Site | Google Scholar
K. de Waal and M. Kluckow, “Functional echocardiography; from physiology to treatment,” Early Human Development, vol. 86, no. 3, pp. 149–154, 2010.
View at: Publisher Site | Google Scholar
A. Sehgal and P. J. McNamara, “Does point-of-care functional echocardiography enhance cardiovascular care in the NICU?” Journal of Perinatology, vol. 28, no. 11, pp. 729–735, 2008.
View at: Publisher Site | Google Scholar
A. Groves, G. Chiesa, G. Durighel et al., “Repeatability and internal validation of magnetic resonance assessment of cardiac function in newborn infants,” Acta Paediatrica, vol. 98, pp. 24–25, 2009.
View at: Google Scholar
K. Lundstrøm, O. Pryds, and G. Greisen, “The haemodynamic effects of dopamine and volume expansion in sick preterm infants,” Early Human Development, vol. 57, no. 2, pp. 157–163, 2000.
View at: Publisher Site | Google Scholar
D. Osborn, N. Evans, and M. Kluckow, “Randomized trial of dobutamine versus dopamine in preterm infants with low systemic blood flow,” Journal of Pediatrics, vol. 140, no. 2, pp. 183–191, 2002.
View at: Publisher Site | Google Scholar
I. Zonnenberg and K. A. de Waal, “The definition of a haemodynamic significant duct in randomized controlled trials: a systematic literature review,” Acta Paediatr, 2011, [Epub ahead of print].
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2012 Koert A. de Waal. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

4324

Downloads

2038

Citations