Abstract
Errors from past rounds of population projections can provide both diagnostic information to improve future projections as well as information for users on the likely uncertainty of current projections. This paper assesses the forecast accuracy of official Australian Bureau of Statistics (ABS) population projections for the states and territories of Australia and is the first major study to do so. For the states and territories, it is found that, after 10-year projection durations, absolute percentage errors lie between about 1% and 3% for the states and around 6% for the territories. Age-specific population projections are also assessed. It is shown that net interstate migration and net overseas migration are the demographic components of change which contributed most to forecast error. The paper also compares ABS projections of total population against simple linear extrapolation, finding that, overall, ABS projections just outperformed extrapolation. No identifiable trend in accuracy over time is detected. Under the assumption of temporal stability in the magnitude of error, empirical prediction intervals are created from past errors and applied to the current set of ABS projections. The paper concludes with a few ideas for future projection rounds.
1. Introduction
Population projections for states, provinces, statistical regions, and other major subnational areas are produced periodically by government agencies for a variety of planning purposes. Ex-post evaluations of the forecast accuracy of such projections remain comparatively rare, even though there are good reasons for undertaking such work. For example, they may reveal hitherto concealed deficiencies in methods, assumption setting, or data which can be resolved in future projection rounds. In addition, past errors can provide users with guidance on the approximate magnitude of error which current forecasts might be subject to. Ideally, past errors would be used to create prediction intervals for current forecasts, either in the form of empirical prediction intervals based directly on the observed error distributions [1, 2] or in the form of fully probabilistic forecasts in which error distributions for demographic rates are represented by time series models [3, 4].
Population projections for Australia’s states and territories have been published on a regular basis by the Australian Bureau of Statistics (ABS) since the late 1970s. They constitute the official population projections for Australia, are generally highly regarded, and are widely used and cited by government, researchers, business, and the media. Yet to date, no comprehensive evaluation of ABS state and territory projections has been published, although brief assessments have been made by Adam [5] and the ABS [6], and an evaluation of ABS projections at the national scale was completed by Wilson [7]. Bell and Skinner [8] assessed projections prepared by state and territory governments and academics, but not those of the ABS. Published evaluations of official state/regional projections for other countries are rare, though examples can be found from Statistics New Zealand [9], ONS [10] for the UK, Rees et al. [11] for European Union regions, Smith and Rayer [12] for Florida, and Wang [13] for US states. In common with national level evaluations, these studies have found that forecast errors tend to be positively associated with projection duration and negatively associated with population size.
The purpose of this paper is to determine how well ABS state and territory projections performed as forecasts of future population and to make use of past forecast errors to create predictive distributions for the current round of ABS projections. The paper aims to provide the first detailed analysis of ABS state and territory forecast error and uncertainty as well as to contribute a new subnational-scale case study to the international literature. Particular features of this analysis which are rare in the literature include an examination of errors by age group and a focus on error distributions as well as averages. Specifically, the paper seeks to address the following questions.
(1) How well did ABS projections by state and territory perform as forecasts? How accurate were projections of total population and age-specific populations?
It is important to understand the proximate causes of these errors so the paper also asks:
(2) What were the relative contributions of errors in births, deaths, net interstate migration, net overseas migration, and jump-off error to total forecast error?
Third, to examine the amount of “value added” by the ABS as a result of their methods and careful assumption setting, ABS projections of population totals are compared with naive projections created by a very simple model. Thus, the third question is:
(3) Did the ABS projections of total population yield more accurate forecasts than linear extrapolation?
Fourth, one would hope that improvements in data, theory, and methods in demography would be reflected in more accurate projections over time. The paper therefore asks the following.
(4) Have ABS projections total populations become more accurate over time?
Finally, past errors are used to inform the future:
(5) Assuming the magnitude of past errors continues into the future, how much uncertainty surrounds the current ABS medium series projections?
On the matter of terminology, many producers of projections stress that they only produce projections and not forecasts, and the ABS is no exception. They assert that their projections “are not intended as predictions or forecasts but are illustrations of growth and change in the population that would occur if assumptions made about future demographic trends were to prevail over the projection period” [25, page 2]. However, most users will nonetheless interpret medium series projections as forecasts. Following Smith [26], the data under examination are termed projections but they are assessed as if they were forecasts, and references are therefore made to “forecast accuracy” and “forecast error.”
The paper continues as follows. Section 2 includes a description of the projections and estimates data used to undertake this forecast error analysis and definitions of the various error measures employed. Findings which provide answers to the first four research questions are the focus of Section 3, whilst Section 4 attempts to apply distributions of past errors to the current set of ABS state and territory projections in the form of empirical prediction intervals. The conclusions consist of a summary of key findings, lessons learnt from this evaluation, and some suggestions for further research.
2. Data and Methods
2.1. ABS Population Projections Data
Population projections data were obtained from the ABS Population Projections, Australia publications (and earlier volumes with a variety of titles) in both electronic and hard copy formats. Data on total and age disaggregated populations from twelve projection rounds were examined, starting with the 1978-based round and extending to the most recent 2006-based set. Projections for broad age groups only were published in the 1978- and 1981-based projections (0–14, 15–44, 45–64, and 65+), whilst at least five-year-age detail is available for all later rounds.
All projections from the 1981-based round onwards refer to the usually resident population at 30th June each year. The 1978-based projections, however, were produced at a time when ABS used a “persons present” definition of population, shortly before they switched to a usually resident definition. Error measures for the 1978-based round were adjusted to take account of this difference (as explained below in the section on error measurement).
The ABS has often produced many alternative projection series and has deliberately avoided labelling any of them as forecasts. Users, however, have a tendency to interpret one of the series as a forecast. From the 2002-based projections onwards three main series have been produced, A, B, and C, with series B providing total projected populations lying clearly between series A and C. Understandably, users have generally interpreted series B as the forecast. Prior to the 2002-based projections, the question is more difficult to answer because some rounds contain four main series, whilst other rounds have three main series but with no single series clearly occupying a middle position. The approach taken here was to select series on the basis of total projected population, choosing the middle trajectory where there were three to choose from and the middle two in cases where four main series were published. In some circumstances, especially where two series possess almost identical total populations, judgement overrode these rules (e.g., the 1999-based projections for Victoria and South Australia have two series which hardly differ in total population, so all three series were chosen). The selected series are listed in Table 1 below.
2.2. Naïve Populations Projections Data
ABS projections of total population from the series listed in Table 1 were compared to naïve projections of total population with the same jump-off years. These naïve projections were generated by extrapolating from linear regressions fitted over a ten-year base period. Linear extrapolation was chosen because it is easy to calculate and is one of the most accurate of simple population projection methods [27, 28].
2.3. Population Estimates Data
Projections were assessed against subsequently published 30th June estimated resident populations (ERPs), the official usually resident population count for Australia produced by the ABS [29]. An important assumption in any assessment of population projections such as this is that ERPs represent accurate counts of the usually resident population. Being based on census counts, vital statistics, and migration estimates, all of which contain some degree of inaccuracy, ERPs will never provide a perfect count of the population, but they are nonetheless the best estimates available.
2.4. ABS Projection Method
The projections assessed in this paper were all produced by a standard cohort-component model with population disaggregated by sex and single years of age. Details, including projection equations, are given in the ABS Demographic Estimates and Projections reference volume [30]. Fertility assumptions are specified in terms of the total fertility rate and mortality assumptions by life expectancy at birth, and both overseas migration and interstate migration assumptions are summarised as annual average net totals. The ABS projection model does handle directional migration flows, however. Preliminary projections of directional migration are iteratively adjusted to agree with the specified net totals.
2.5. Error Measures
Error is defined as the forecast population minus the ERP. To standardise for varying population sizes between states and territories and over time, Percentage Errors (PE) were calculated as follows where Forecast denotes the population forecast, ERP is the subsequently published estimated resident population, and is a chosen year in the projection horizon. Positive values indicate overprojections, whilst negative values reflect an underprojection. When just the extent (and not direction) of error was of concern, absolute percentage errors were used. These are percentage errors with all negative signs ignored, that is, For some purposes, a modified percentage error measure was used which removes any jump-off year error. This corrected percentage error (CPE) [31] is defined as
where 0 denotes the jump-off year. This measure was used in place of PE to calculate errors of the 1978-based projections because they were produced on a “persons present” population base which is inconsistent with ERPs. Its absolute version, absolute corrected percentage error (ACPE), was employed to measure the error of all the projections listed in Table 1 for the creation of empirical prediction intervals. Section 4 explains how ACPE was used.
To assess the degree of error across many observations, several average error values were used. The first of these, mean percentage error (MPE), is calculated as where is the number of observations. MPE is a measure of bias; it indicates whether a set of projections was, overall, too high (positive values) or too low (negative values). Mean absolute percentage error (MAPE) is similar to MPE except that it measures the absolute value of error only. It is calculated as MAPE gives equal weighting to each APE of which it is the mean. Thus, any MAPE for 8 states and territories includes a weighting of 1/8 for each APE. Weighted mean absolute percentage error (WMAPE), which weights each APE by population size, provides an alternative perspective. It is defined as WMAPE is particularly appropriate when population sizes vary considerably, as they do across Australian states and territories. Effectively, this measure regards the absolute size of errors across observations as more important than percentage errors. WMAPE is equivalent to the MAD/mean ratio (mean absolute deviation/mean population) [32] and mean percentage absolute deviation [33].
Where there is a small sample or a skewed distribution, it is often sensible to use median errors. Median percentage Error (MedPE) is the middle value of a set ranked PEs, whilst median absolute percentage error (MedAPE) is the middle value of a set of ranked APEs.
Percentage Better describes the percentage of projections forecast more accurately by method A than method B [34].
To assess the improvement in accuracy of ABS projections over those obtained from linear extrapolation, the proportionate reduction in error (PRE) [35] was calculated, in this case using WMAPE. It was calculated as
3. Forecast Error
3.1. How Accurate Were Projections of Total Population?
Table 2 presents selected error statistics for state and territory total populations at particular projection durations. The top panel shows median absolute percentage errors at projection durations of 0, 5, 10, and 15 years, where a duration of 0 years refers to jump-off year error. Such errors arise from projections starting in noncensus years before ERPs are finalised following the next census. With the exception of the Northern Territory, these errors are not substantial.
As the table shows, New South Wales (the most populous state) has experienced the smallest average forecast errors, whilst projections for the Northern Territory and Australian Capital Territory, which have much smaller populations, have proved much harder to forecast. However, the relationship between average error and population size is only very approximate, as Figure 1 demonstrates. This deviates somewhat from the results of a number of other forecast accuracy studies which have found a clear negative relationship between these two variables (e.g., [10, 26, 36]). In contrast, the positive relationship between average absolute error and projection duration does align more closely with findings from other studies. Forecasts for most states and territories experience increases in MedAPE with increasing projection duration.
To what extent were the projections, on average, too high or too low over their first 15 years? The median percentage errors in the second panel of Table 2 demonstrate little bias for New South Wales and South Australia. The population of fast-growing Queensland was underprojected at durations of 10 and 15 years, whilst the populations of the two territories (NT and ACT) were on average overprojected after 10 and 15 years.
The bottom three panels of the table focus on the distribution of error, showing how error distributions widen with increasing projection duration. Following Lutz et al. [37], 80% intervals were calculated. These intervals refer to the middle 80% of ranked absolute percentage errors, so that 10% of these errors occur above the upper bound, and similarly, 10% fall below the lower bound. An 80% interval is useful in this case because with a small number of observations it reduces the likelihood of including outliers (which would not always be the case if a 95% interval had been chosen) whilst still covering the majority of errors. The bottom panel reports the width of the 80% interval (the upper bound minus the lower bound). Clearly there is a strong positive association between projection duration and interval width, or to express it another way, between projection duration and forecast uncertainty.
3.2. What Were the Relative Contributions of Each of the Demographic Components of Change to Total Forecast Error?
To what extent were forecast errors the result of errors in fertility, mortality, migration, and the jump-off population? Previous studies have answered this question in a variety of ways, including separate analyses of errors in fertility and mortality rates and migration volumes (e.g., [5]), regression (e.g., [38]), and decomposition (e.g., [39]). A very simple method is used here. Absolute percentage errors for total population were calculated for all projections listed in Table 1, assuming that only one of the following demographic components had been forecast incorrectly:(i)jump-off population,(ii)births,(iii)deaths,(iv)net interstate migration,(v)net overseas migration.
So, for example, an “incorrect births” projection for a particular state or territory uses projected births but actual values for all other components. Then the absolute percentage error of the total population is calculated. This approach was applied to all components, projection rounds, states, and territories. An obvious limitation is that it ignores the dependence between the components. So, for example, it ignores the fact that overprojections of net overseas migration will give higher numbers of births for any assumed fertility rate. Nonetheless, this approach gives a broad brush and an easily interpretable overview of the relative contributions of the demographic components of change to overall forecast error.
Median absolute percentage errors, where the median refers to the average across all projection rounds, are shown in Figure 2. Overall, migration errors dominate. In New South Wales, Victoria, South Australia, and Western Australia, net overseas migration made the greatest contribution to error; for the other states and territories, net interstate migration errors were more substantial. For the Northern Territory, unfortunately, errors were quite large for all components. If there is a lesson for improving future rounds of state and territory projections, it is that most effort in understanding demographic trends and setting assumptions should be reserved for migration.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
These findings hint at the temporal variability of net migration trends as a good predictor of forecast uncertainty. This is because, at their most basic level, projections tend to be largely an exercise in extrapolation. To investigate this, crude net migration rates (combining net interstate and net overseas migration) for each of the states and territories were calculated for the period 1971 to 2011. Differences from long-run linear trends were determined, including the standard deviations of these differences. The standard deviations were then plotted against 80% absolute percentage error intervals. Figure 3 shows a reasonably close association between these variables at 5- and 10-year projection durations. After 15 years (not shown to avoid cluttering), a similar relationship exists, although it is a little weaker.
3.3. How Accurate Were Projections of Age-Specific Populations?
For many projection users, it is not total population which is of primary concern but projections of specific age groups, such as school-age children and the very elderly, because these are relevant for government policy and planning. This subsection reports on the accuracy of projections by five-year age group from the 1984-based projections onwards (five-year age detail was not published for earlier projection rounds). Figure 4 presents median absolute percentage errors by state and territory and age group after 0-, 5-, 10-, and 15-year projection durations.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
Overall, errors for individual age groups tend to be larger than those for the population as a whole. For most states and territories, projections for 0–4-year olds were, on average, less accurate than those for older children. This concurs with evidence from other subnational and national studies (e.g., [40, 41]). The finding is not surprising given that projections for these infants suffer from both error in projections of births and error in projections of demographic processes once they enter the population.
Errors were often larger than average at ages 25–34, reflecting relatively high migration rates at the young adult stage of the life course. This is particularly the case for the Australian Capital Territory and is almost certainly due to rates of interstate migration being very high and fluctuating [42] and hence difficult to forecast. Although smaller in magnitude, errors at these ages are relatively high for South Australia and Tasmania at projection durations of 10 and 15 years. These findings contrast somewhat with forecast error patterns for national populations where the absolute error pattern by age tends to be u-shaped due to the dominance of fertility and mortality errors (e.g., [43, 44]).
At the older working ages and younger elderly ages, errors generally proved to be smaller. Projections should be more accurate at these ages because these stages of the life course are not usually subject to high rates of mortality or long-distance migration. Projections should therefore be closely related to the numbers living in each state and territory 5, 10, or 15 years earlier. At the highest ages, average errors tend to increase a little, especially for the Northern Territory and Australian Capital Territory. The Northern Territory is particularly affected by jump-off errors at these ages, whilst both territories have probably also suffered from errors in projected mortality and also migration given the migratory nature of these populations.
In Figure 5, the focus switches from average errors to their distribution. It shows the distribution of percentage errors by age group for 10-year projection durations, illustrating both the median and upper and lower bounds of the middle 80% of errors. Although the number of projection rounds is quite small (between 10 and 12), resulting in a fair degree of random fluctuation, a number of patterns can be seen. The largest median PEs and widest ranges can be seen in the Northern Territory elderly populations and the ACT young adult populations. In most states and territories, the elderly population is underprojected, reflecting repeated underprojections of life expectancy [7]. The exception to this is Queensland where underprojections of life expectancy were probably offset by overprojections of net migration gains. Unlike many other studies, there is no universal pattern of overprojections at the youngest ages.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
3.4. Did ABS Projections of Total Population Yield More Accurate Forecasts Than Linear Extrapolation?
Absolute percentage errors of naïve projections created from linear extrapolation were compared with those of the ABS projection series listed in Table 1. How well did the ABS projections perform? Table 3 answers this question by presenting percentage better statistics: the percentage of ABS projections which were better in terms of absolute percentage error than linear extrapolation. Numbers in bold indicate where linear extrapolation provided more accurate projections of total population for the majority of projection rounds. Of course, the value of ABS projections extends beyond the forecast accuracy of total populations. Their projections provide considerable age and temporal detail as well as projections of the demographic components of change and in addition, all states and territories are projected using a consistent method and set of assumptions. Nonetheless, it is instructive to examine the forecast accuracy of total population as this is the “headline” figure which many users focus on.
On the basis of Table 3, projections of Tasmania’s population have proved the most successful of all ABS state and territory projections. Projections for this state have been more accurate than those of linear extrapolation for the majority of projection rounds and for every projection duration shown in the table. Interestingly, although the two territories have been forecast the least accurately in terms of total populations and population growth, the majority of ABS projections have prove more accurate than linear extrapolations. In other words, the ABS projections have added value beyond simple extrapolations. Conversely, whilst ABS projections of South Australia’s total population have prove reasonably accurate (Table 2), for the majority of projection rounds, better forecasts of total population would have been obtained from linear extrapolation. This reflects the long-run stability of the trend in total population numbers in South Australia.
3.5. Have ABS Projections of Total Population Become More Accurate over Time?
Have ABS projections of state and territory total populations become more accurate over time? Table 4 presents average errors of the projections at 5- and 10-year durations by jump-off year. The values are means of state and territory projections for each jump-off year. MAPE is shown in the table because of its common usage, but it has the disadvantage of giving equal weighting to each percentage error. Thus, the higher percentage errors for the Northern Territory and Australian Capital Territory are given as much weight as the smaller errors for New South Wales, which has twelve times as many people as these two territories combined. Because of the considerable variation in population sizes across the states and territories, WMAPE, the sum of population-weighted absolute percentage errors, is also shown.
From the MAPE values for 5-year projection durations, it is difficult to discern any clear trend in forecast accuracy. MAPEs lie between 1.2% and 2.5% for all years except for the 2004-based projection. The larger error for this projection round is mostly due to a huge, but temporary, increase in net overseas migration to Australia between 2006 and 2010 [42]. According to WMAPE, there appears to be a slightly clearer trend of improving accuracy up to the 1999-based round before the recent net overseas migration rise resulted in large errors for the 2004-based round. At 10-year projection durations, there is no obvious trend in accuracy over time.
However, examining an error measure such as MAPE or WMAPE is only one way of assessing the performance of projections over time. A fuller picture would take into account variations in forecasting difficulty. Some projections may have prove to be highly accurate only because demographic trends fortuitously turned out to be stable for a number of years, not because it was based on any great foresight about future trends. One approach to evaluating accuracy whilst accounting for forecasting difficulty is to measure how much more accurate ABS projections proved to be against the linear extrapolations reported in the previous subsection. To provide an answer, the proportionate reduction in error (PRE) of WMAPE was calculated for each jump-off year at 5- and 10-year projection durations (Table 4). The PRE describes the percentage reduction in WMAPE achieved by the ABS projections over linear extrapolation. As the table shows, it is difficult to see any clear improvement, or indeed deterioration, in accuracy over time. Similar conclusions have been made in other studies. For example, in his evaluation of national population forecasts of 14 European countries, Keilman [45] discovered no improvements in forecast accuracy for these countries over the last 25 years.
4. Empirical Prediction Intervals for Current Projections of Total Population
New sets of population projections are usefully accompanied by information on the likely error (or uncertainty) of those projections. An increasing number of demographers have represented demographic uncertainty in the form of probabilistic population projections (e.g., [3, 4, 37]). However, the application of probabilistic methods at subnational scales faces a number of challenges: the bulk of probabilistic methodology has been developed for national scale projections, the data requirements for subnational probabilistic models are considerable, and the preparation of such projections is complex and time-consuming.
An alternative is to create prediction intervals based on the distribution of past errors (e.g., [1, 2, 46, 47]). An evaluation of as many past projections as possible is conducted, the distribution of forecast errors is determined, and then those distributions are applied in some way to current forecasts. Such an approach is dependent on the assumption that future errors will, at least approximately, resemble those of the past. Of course, no guarantee of this outcome can be given, though several studies have found a fair degree of stability in errors over time (e.g., [47, 48]). Key advantages of empirically derived prediction intervals over probabilistic methods include their simplicity and the speed with which they can be produced.
For this paper, empirically based prediction intervals were calculated and applied to the most recent set of ABS state and territory series B population projections, the 2006-based set [25]. Series B is generally interpreted as a forecast. Given the lack of a clear trend in changes to forecast accuracy over time, the errors of all projection series listed in Table 1 were used in the creation of prediction intervals. However, because the 2006 ERPs are final census-based population estimates and not subject to any further revision, there is, by definition, no jump-off error. Thus, instead of basing the prediction intervals on the past distributions of absolute percentage errors, absolute corrected percentage errors (ACPEs) were used. Following the practice of Lutz et al. [37], 80% prediction intervals were calculated. In addition, because of the common research finding “there is no way to know in advance whether a particular forecast ⋯ will be too high or too low” [49, page 782], predictive distributions were assumed to be symmetrical. Furthermore, rather than combining error observations across all states and territories and disaggregating by population size and/or base period growth rate, each state and territory was handled separately. Finally, it is acknowledged that some errors used to calculate the predictive distributions are based on forecasts and ERPs for the period 2006 to 2011. If this method was being applied during the preparation of 2006-based projections, then, of course, errors for these recent years would not be available; for the purposes of illustration in this paper, their inclusion is unimportant.
Predictive distributions for the 2006-based ABS Series B projections were calculated as follows.(1)Values describing the width of the inner 80% of the distribution of absolute corrected percentage errors were obtained for each state and territory for projection durations of 0 to 10 years. Values for longer durations were not used because of the small number of observations.(2)The observed 80% ACPE intervals by projection duration were smoothed by fitting straight lines to smooth out random variation due to small sample sizes. Intercepts were forced through the origin to signify no jump-off error. As an example, Figure 6 shows an actual and smoothed 80% ACPE interval for Tasmania.(3)Prediction intervals were created for the 2006-based ABS projections. Populations at the upper bound of the 80% prediction interval were calculated as where denotes the ABS series B projection and the projection duration in years. Populations describing the lower bound were computed as:
Figure 7 illustrates the ABS series B 2006-based projections along with 80% prediction intervals shown as grey bands. Given that these intervals were created from projection durations of up to 10 years only, they are best applied only for the first ten years of the 2006-based projections. However, error distributions for total population often expand in a linear manner with time so it seems reasonable to extend the 80% intervals a little further into the future.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
How much forecast uncertainty do these 80% intervals suggest? Taking Queensland as an example, by 2026 the empirical prediction intervals suggest an 80% chance of the state’s population lying within of the ABS Series B projection of 6.04 million, which equates to the range 5.39–6.69 million. New South Wales enjoys the greatest forecast certainty, with an 80% chance of its 2026 population forecast lying within 8.03 and 8.76 million . Relative uncertainty is greatest for the Northern Territory. By 2026, the 80% interval spans 215,000 to 355,000, of the ABS Series B projection of 285,000.
Because these projections were produced a few years ago, it is possible to undertake a short-run test of the prediction intervals by plotting estimated resident populations from 2006 to 2011 (shown in Figure 7 as the dashed lines). As the graphs show, for all states and territories, these ERPs lie within the 80% prediction intervals.
5. Summary and Conclusions
This paper has evaluated the forecast accuracy of twelve sets of ABS state and territory population projections, covering the 1978- to the 2006-based rounds. The key findings can be summarised as follows.(i)Errors in forecasting total population were generally modest for the states, with absolute percentage errors at projection durations of 10 years averaging around 1–3%, but quite large for the two territories, at roughly 6% after 10 years.(ii)New South Wales and South Australia experienced the least bias in their projections of total population; Queensland was generally underprojected; the two territories were overprojected at projection durations of 10 years or more.(iii)There was an approximately linear relationship between average error and projection duration and between the 80% distribution of errors and projection duration.(iv)Errors in net interstate migration and net overseas migration projections were the major contributors to total population forecast error.(v)A reasonably close association between the variability of historical net migration trends and the widths of the 80% absolute percentage error intervals was established.(vi)Projections of age-specific populations were generally less accurate for 0–4 year olds than older children, often quite inaccurate at the peak migration ages, and in some cases quite inaccurate at the very oldest ages.(vii)ABS projections of total population were more accurate than linear extrapolations in the majority of projections for Tasmania, the Northern Territory, and the Australian Capital Territory; for South Australia, linear extrapolation tended to be more accurate; for the other states, the findings were mixed.(viii)There was no clear evidence of ABS projections of total population becoming more accurate or less accurate over time.(ix)Past distributions of absolute percentage errors were used as the basis for creating 80% empirical prediction intervals applied to the 2006-based ABS projections. For all states and territories, 2006–2011 ERPs fell within the 80% intervals.
Ideally, a comparison of the accuracy of ABS state and territory projections would have been made with similar subnational projections from other countries. Regrettably, there are very few similar studies available, and the different error measures, projection durations, and limited data reported in these studies prevent any reliable comparison from being undertaken. As suggested by a delegate at a conference in which an earlier version of this paper was presented, a robust comparison can only be achieved with a detailed multinational database consisting of the “raw” data of population projections and subsequently published population estimates.
Does the paper’s analysis reveal anything which could improve the accuracy of ABS state and territory population projections in the future? Minor improvements may result from using only census-year final ERPs as the jump-off points for projections. Because these ERPs are not subject to any further revision, there cannot be any jump-off year error–a type of error which particularly affected projections for the Northern Territory.
Aside from that, this author’s view is that any significant improvements in accuracy are unlikely to be realised unless robust methods of forecasting turning points and cyclical patterns in migration and fertility are discovered. Undoubtedly this is a major challenge. Nonetheless, the new type of total fertility rate and fertility projection method devised by McDonald and Kippen [50] offers some promise in this regard. The approach of these authors involves calculating a total fertility rate standardised for parity and duration since last birth. McDonald and Kippen demonstrated how short-term projections of age-parity-duration-specific fertility rates from 2000 to 2005 successfully predicted the upturn in the Australian TFR. The ABS is currently updating the McDonald-Kippen projections using 2006 and 2011 census data and will be reporting their findings later in the year. Research to ascertain whether the method is equally successful at the state and territory scale would be extremely valuable.
Regular national-level overseas migration forecasts have recently been initiated by the Department of Immigration and Citizenship [51]. These are short-term forecasts looking a few years ahead and are based on recent offshore visa grants, probabilities of individuals with particular visas becoming international migrants, forecasts of visas being issued, as well as forecasts of nonvisa migration streams, such as those involving Australian and New Zealand citizens. It is too early to determine whether these forecasts will prove more accurate than past ABS overseas migration projections, but the disaggregation by visa or resident status appears promising. Again, extension to the state and territory scale might prove valuable and should certainly be investigated.
As for the successful prediction interstate migration, the challenges seem greater. Although interstate migration is known to be affected by spatial variations in employment and housing costs as well as factors relating to lifestyle, climate, environment, and family networks [52, 53], our understanding remains partial. The forecasting of predictor variables in any model of interstate migration would not be easy. Clearly, this is an area requiring much more research effort.
Whatever improvements are made to forecast methods, and hopefully accuracy, in the future, errors will still unfortunately occur. Projection users deserve to be supplied with some form of guidance on the uncertainty surrounding population forecasts. Ideally, one would create fully probabilistic population forecasts to communicate this uncertainty. But where this is not possible, then a simpler alternative is to create empirical prediction intervals for total population from past errors, as demonstrated in Section 4 of the paper.
Abbreviations
| AST: | Australian States and Territories |
| NSW: | New South Wales |
| Vic.: | Victoria |
| Qld: | Queensland |
| SA: | South Australia |
| WA: | Western Australia |
| Tas.: | Tasmania |
| NT: | Northern Territory |
| ACT: | Australian Capital Territory |
| ABS: | Australian Bureau of Statistics |
| APE: | Absolute percentage error |
| ACPE: | Absolute corrected percentage error |
| CPE: | Corrected percentage error |
| ERP: | Estimated resident population |
| MAPE: | Mean absolute percentage error |
| MedAPE: | Median absolute percentage error |
| MedPE: | Median percentage error |
| MPE: | Mean percentage error |
| PE: | Percentage error |
| PRE: | Proportionate reduction in error |
| WMAPE: | Weighted mean absolute percentage error. |
Acknowledgments
Assistance with data entry and analysis by Philipp Ueffing and Bala Jones is gratefully acknowledged. Comments from the anonymous reviewers helped to improve the paper. Any remaining errors are the author’s.