Getting to the Point: Accuracy of Point Count in Monitoring Ecosystem Change
Ecological monitoring programs depend on the robust estimation of descriptive parameters. Percent cover, gleaned from transects sampled with video imagery, is a popular benthic ecology descriptor often estimated using point counting, an image-based method for identifying substrate types beneath random points. We tested the hypothesis that the number of points needed to robustly estimate benthic cover in video imagery transects depends on cover itself, predicting that lower cover will require more points/frame to be accurately estimated. While this point may seem obvious to the statistically inclined, the justification of point density has been largely ignored in the literature. We examined the statistical behavior of point count estimates using computer-simulated 20 m-long transects patterned after data from a Bahamian reef. The minimum number of points necessary to insure accurate percent cover estimation, the Optimal Point Count (OPC), is a function of mean percent cover and spatial heterogeneity of the benthic community. More points are required to characterize reefs with lower cover and more homogeneously distributed coral colonies. These results show that careful consideration must be given to sampling design and data analysis prior to attempting to estimate benthic cover, especially in the context of long-term monitoring of degrading coral reef ecosystems.
A common problem while working in ecological characterization and monitoring programs is how to effectively test and optimize methods and experimental designs. Live percent cover is a widely used ecological descriptor in marine conservation biology and large-scale monitoring projects (e.g., [1–4]) and has been a key parameter in the quantification of coral reef degradation over large spatial scales (e.g., [5–7]). Using point count to analyze video transects and photoquadrats is an efficient way to estimate percent cover and monitor large areas, because it allows copious amounts of data to be collected while minimizing underwater time, and provides a permanent record of the benthic community [8, 9]. Substrate types underlying randomly assigned points are identified. Percent cover is then estimated as a ratio of the number of points overlaying a substrate type to the total number of points.
The number of points to be used per unit area (point density and the unit area being an image frame from a video transect) is crucial to obtaining a robust estimate of percent cover, and initial statistical tests should be performed to establish the point density that will provide adequate precision and accuracy while maximizing efficiency (the time spent identifying substrate types underlying points). Such tests include power analysis [1, 4, 10], performance curves (graphical examination of standard deviation inflation relative to point density [4, 11–13]), bootstrap estimate of bias , Pearson’s product moment correlations , and examination of the interaction term in a two-way ANOVA for transects sampled twice using different point densities .
While the need for determining adequate point density is mentioned in coral reef monitoring manuals (e.g., ) and peer-reviewed publications (e.g., ), this essential parameter is rarely appropriately reported in the literature. “Coral Point Count with Excel extensions” (CPCe) was released in 2006 , and between 2006 and October 2009, its use for percent cover estimation was reported 32 times in the peer-reviewed literature. We were able to examine 30 of these papers, 23 of which used CPCe to estimate percent cover. Sixteen out of 23 studies reported point density/frame without any form of justification. Four studies did not even report the number of points that was used, and only 3 studies assessed the statistical robustness of the number of points per unit area to be used.
In addition to bias introduced by poorly calibrated sampling efforts, the accuracy of cover estimates could significantly decrease as reef structure and cover levels change over time. In this case, even if resource managers assess the statistical rigor at the beginning of a long-term monitoring program, nothing guaranties that cover estimates will be bias-free over multiyear surveys. To our knowledge, the behavior of point count estimates relative to varying percent cover has not been addressed, although it is a central question given the ongoing global decline of coral reefs. If the number of random points used for accurately estimating cover is a function of cover itself, the optimal number of points (Optimal Point Count, OPC) sufficient for robust cover estimates for high-cover reefs is likely to increase as cover decreases. In this communication, we test this idea using computer-generated video transects that allowed us to directly compare true (simulated) cover and its estimation by point count. We are not trying to review coral monitoring sampling design; rather, we recognize a severe problem with one specific aspect of video transect sampling using point counting and address it with simulations.
2. Material and Methods
2.1. Settings of Computer Simulations
To assess the statistical robustness of the point count method, we used the R environment  to simulate the sampling of a benthic community with video transects . Computer simulations have the advantage of mimicking given levels of live benthic cover, which can be directly compared to their estimation using the point count method. Parameters mimicked field conditions encountered at Rainbow Gardens Reef, a Bahamian patch reef near Lee Stocking Island in the Exumas [12, 18, 19], where different monitoring techniques have been tested and optimized . Based on these empirical studies, simulated transects contained 50 nonoverlapping frames of pixels (approximate area coverage: 0.2 m2/frame; 10 m2/transect).
Our simulation engine was initialized by setting a target mean percent cover () and its standard deviation (); cover values are therefore controlled at the transect level, using frames as replicates. Benthic heterogeneity was set by generating random numbers from a normal distribution to artificially create internal structure within each transect.
For each simulated transect, cover was estimated using the point count method, by incrementing a “cover” counter if a random point coincided with an area of simulated cover. Random points were generated with the “splancs” package . For all tests, the statistical significance limit was set at .
2.2. Realism of the Computer Simulations
Estimations of computer-simulated cover were qualitatively compared to estimations from human generated data (ground-truth data) to assess realism of the computer model. We judged our simulation procedure to be realistic, as estimations of ground-truth cover and simulated cover were similar (Figure 1). However, variation within field data was higher than within simulated data. The absence of frame-level spatial heterogeneity in the model (only transect-level heterogeneity was simulated) and absence of skew in the normal distribution used to generate transect-level heterogeneity might explain this difference. We chose to ignore these parameters to keep the simulation engine as simple as possible.
2.3. Using Computer Simulations to Assess the Robustness of the Point Count Method
was set to 30, 20, 10, and 5% to simulate realistic decline in live coral cover (e.g., ), and was set to 0.2, 0.4, and 0.6 times . These values of allowed for realistic variation along transects (values tailored using empirical data). For example, when and , cover varies between 15% and 40%. Five hundred simulations were run for each combination of and values. For each simulation, percent cover was estimated using one to 50 points per frame. Distributions of true and estimated percent cover were compared using the Wilcoxon test, a nonparametric alternative to the -test ; although we simulated internal transect structure using a normal distribution, cover estimations based on few points per frame yielded nonnormal distributions. values for each test were plotted as box-and-whisker plots to visualize the lower limit of each set of simulations. The optimal point count per frame (OPC) was determined as the number of points for which all nonoutlier values (outliers defined as values >1.5 times the interquartile range) are above the statistical significance limit of 0.05. In other terms, the OPC corresponds to the minimum number of points required to see a statistical difference between true and estimated cover in only rare instances (outlier values).
The efficacy of the point count method was also tested by quantifying bias (the absolute difference between a parameter and its estimate) for the mean and the standard deviation of percent cover over simulated transects. Bias was calculated using the values of and described above, for 10, 50, and 100 points per frame, and 100 replicates were ran for each parameter combination (reported values are means for these 100 replicates).
Estimated and simulated parameter values converged as the number of points per frame increased (Figure 2). More points were required to accurately estimate percent cover from low-cover transects (Figure 3), and this relationship was consistent across heterogeneity levels (Table 1). Bias was systematically higher for lower cover values (Figure 4). Increasing the number of points per frame significantly reduced bias in all cases. While has little to no effect on the bias of the mean, it significantly affected the bias of the standard deviation; the standard deviation estimated from homogeneous transects was significantly more biased than for heterogeneous ones.
Reef ecologists are faced with the dilemma of having to capture the realities of complex and dynamic environments with statistical precision. The design of any ecosystem monitoring program must be based on an understanding of population dynamics and obey the statistical premise that the sampling regime reflects the true abundance of organisms. Coral reef ecologists adopted some of the general survey techniques from plant ecology owing to the structural similarities of forests and reefs (i.e., quadrats, line transects, and nearest neighbor analysis). Modifications occurred due to the need to optimize underwater working time and other aspects of working underwater. Line intercept transects were replaced by point intercept, still photography, and currently video.
We have chosen to address the issue of percent cover determination because it lies at the heart of the survey methodology. Obviously, the requirements for a long-term, health-status monitoring program demand more information than simply percent cover and therefore require more data than offered here for planning complex, community-oriented ecological surveys. As the number of benthic categories (e.g., coral species, substrate types, and health status, etc.) increases, obviously more attention must be applied to determining the OPC (particularly for detecting rare species). The project goal, spatiotemporal scope, and ecosystemic resolution should dictate the experimental design. Each project should undertake its own statistical design based on ecological sampling theory beginning with an overall assessment of how many stations, transects, quadrats (etc.) all the way down to how many categories and points/frame to be used. While all of these parameters influence the robustness and reliability of a sampling design, this communication recognizes a severe problem with one specific aspect of video transect sampling using point counting and has addressed it with simulations.
As ecosystems change, the optimal number of points (OPC) should be adjusted to maintain a consistent degree of accuracy. For example, when and μ, true cover and its estimation were never significantly different with a sampling effort of 13 points per frame. When was dropped to 5%, and OPC was kept at 13 points per frame, 60% of our replicate simulations detected a statistically significant difference between the distributions of estimated and true cover values. In other terms, point count estimation of cover was satisfactory only 40% of the time. Coral mortality and loss of live coral cover are widely acknowledged symptoms of reef decline, with 5–20% decrease in live cover being considered biologically meaningful [22–25]. In the example of simulation in Figure 2, was set to 20%, and 6 points per frame yielded an estimate of 15.2% cover. Comparing this estimate to the true mean percent cover would lead to a “false alarm” (cover dropped by 4.8%) or type I error. If, on the other hand, this estimate was used as a baseline, and the same transect was to lose 5% cover over the years, a comparison would lead to a “false negative” or type II error (e.g., ). Investigators must therefore insure that measurement error remains below levels of biologically meaningful variation.
Transect-level heterogeneity in cover strongly affected the OPC. At Rainbow Gardens Reef, scleractinian coral cover varied twice as much within transects as octocoral cover. Benthic types might therefore strongly influence point count calibration. Also, benthic heterogeneity might depend on reef geomorphology itself (e.g., [27, 28]). Finally, coral reef degradation is associated with loss of architectural complexity or “reef flattening” , a phenomenon caused by a perturbed balance between accretion and erosion. If reef heterogeneity changes overtime (which was the case at Rainbow Gardens Reef; ), the OPC should be reevaluated accordingly.
The results of our study make it clear that OPC must be determined for a specific set of environmental circumstances. However, providing a simple rule of thumb to calculate OPC is difficult, because OPC depends on more factors than just reef cover and heterogeneity. OPC determination must be tailored to a specific nested design, to reach accuracy at the required spatial scale. In addition to the sampling variables listed above (e.g., quadrat size, transect length, and number of transects, etc.), the OPC will depend on the biologically meaningful difference one wishes to detect between treatment groups (i.e., the “effect size”). Our simulations could have therefore incorporated a plethora of variables. Instead, we chose to limit ourselves to a set of parameters that reflect a real situation (the study of Rainbow Gardens Reef, [12, 18, 19]) and demonstrate the ailing effects of poorly chosen point density. We hope that this communication will motivate the community to more critically evaluate their sampling design when using point counting.
The authors thank Amélia Viricel, Paul Leberg, John Fauth, Judith Lang, and two anonymous reviewers for commenting on earlier drafts; Amélia Viricel and Catherine Booker for field help, and the personnel of the Caribbean Marine Research Center (Lee Stocking Island) for technical support. Work at Rainbow Gardens Reef was funded by a Grant from the Perry Institute for Marine Science and the National Oceanic and Atmospheric Administration (NOAA) Undersea Research Program (Project number CMRC-04-PRPD-04-04A) to PD and EP and by a grant from the Slocum Lunz Foundation and the College of Charleston to EP. Views expressed herein are those of the authors and do not necessarily reflect the views of PIMS, NOAA, or any of their subagencies. The authors thank the Department of Biology at the College of Charleston for covering page charges. This is Grice Marine Biological Laboratory contribution number 381.
H. Sweatman, “Long-term monitoring of the Great Barrier Reef,” Status Report 2, Australian Institute of Marine Science, Townsville, Australia, 1997.View at: Google Scholar
J. Porter, V. Kosmynin, K. Patterson et al., “Detection of coral reef change by the Florida keys coral reef monitoring project,” in The Everglades, Florida Bay, and Coral Reefs of the Florida Keys: An Ecosystem Sourcebook, J. Porter and K. Porter, Eds., pp. 749–769, CRC Press, Boca Raton, Fla, USA, 2002.View at: Google Scholar
J. Lang, “Status of Coral Reefs in the western Atlantic: results of initial surveys, Atlantic and Gulf Rapid Assessment (AGGRA) Program,” Tech. Rep. 496, Atoll Research Bulletin, 2003.View at: Google Scholar
E. Brown, E. Cox, P. Jokiel et al., “Development of benthic sampling methods for the Coral Reef Assessment and Monitoring Program (CRAMP) in Hawai'i,” Pacific Science, vol. 58, no. 2, pp. 145–158, 2004.View at: Google Scholar
C. R. Wilkinson, Status of Coral Reef of the World: 2004, Global Coral Reef Monitoring Network and Australian Institute of Marine Science, Townsville, Australia, 2004.
R. B. Aronson, P. J. Edmunds, W. F. Precht, D. W. Swanson, and D. R. Levitan, “Large-scale, long-term monitoring of Caribbean coral reefs: simple, quick, inexpensive techniques,” Atoll Research Bulletin, vol. 415–425, no. 421, pp. 1–19, 1994.View at: Google Scholar
L. B. Preskitt, P. S. Vroom, and C. M. Smith, “A rapid ecological assessment (REA) quantitative survey method for benthic algae using photoquadrats with scuba,” Pacific Science, vol. 58, no. 2, pp. 201–209, 2004.View at: Google Scholar
B. Riegl, R. P. Moyer, B. K. Walker, K. Kohler, D. Gilliam, and R. E. Dodge, “A tale of germs, storms, and bombs: geomorphology and coral assemblage structure at Vieques (Puerto Rico) compared to St. Croix (U.S. Virgin Islands),” Journal of Coastal Research, vol. 24, no. 4, pp. 1008–1021, 2008.View at: Publisher Site | Google Scholar
P. A. Kramer, “Synthesis of coral reef health indicators for the western Atlantic: results of the AGRRA program (1997–2000),” Atoll Research Bulletin, no. 496, pp. 1–58, 2003.View at: Google Scholar
E. Pante, Temporal variation in a Bahamian patch reef community: the decline of Rainbow Gardens Reef, M.S. thesis, College of Charleston, Charleston, SC, USA, 2005.
J. Hill and C. R. Wilkinson, Methods for Ecological Monitoring of Coral Reefs : A Resource for Managers, Australian Institute of Marine Science, Townsville, Queensland, Australia, 2004.
A. King, Digital analysis of a patch reef community: Rainbow Gardens Reef, Lee Stocking Island, Exuma Cays, Bahamas, M.S. thesis, College of Charleston, Charleston, SC, USA, 1995.
B. S. Rowlingson and P. J. Diggle, “Splancs: spatial point pattern analysis code in S-plus,” Computers and Geosciences, vol. 19, no. 5, pp. 627–655, 1993.View at: Google Scholar
R. Sokal and F. Rohlf, Biometry, W. H. Freeman and Compagny, New York, NY, USA, 3rd edition, 1995.
T. P. Hughes, “Catastrophes, phase shifts, and large-scale degradation of a Caribbean coral reef,” Science, vol. 265, no. 5178, pp. 1547–1551, 1994.View at: Google Scholar
R. B. Aronson, W. F. Precht, T. J. T. Murdoch, and M. L. Robbart, “Long-term persistence of coral assemblages on the Flower Garden Banks, Northwestern Gulf of Mexico: implications for science and management,” Gulf of Mexico Science, vol. 23, no. 1, pp. 84–94, 2005.View at: Google Scholar
M. Risk and A. Risk, “Reef surveys as an aid in management,” in Proceedings of the 8th International Coral Reef Symposium, H. A. Lessios and I. G. Macintyre, Eds., vol. 2, pp. 1471–1474, Smithsonian Tropical Research Institute, Panama, 1997.View at: Google Scholar
E. Pante, M. Adjeroud, P. Dustan, L. Penin, and M. Schrimm, “Spatial patterns of benthic invertebrate assemblages within atoll lagoons: importance of habitat heterogeneity and considerations for marine protected area design in French Polynesia,” Aquatic Living Resources, vol. 19, no. 3, pp. 207–217, 2006.View at: Publisher Site | Google Scholar