Table of Contents Author Guidelines Submit a Manuscript
Journal of Probability and Statistics
Volumeย 2011ย (2011), Article IDย 182049, 15 pages
Research Article

Determinant Efficiencies in Ill-Conditioned Models

Department of Statistics, Virginia Polytechnic Institute, Blacksburg, VA 24061, USA

Received 18 May 2011; Accepted 1 August 2011

Academic Editor: Michaelย Lavine

Copyright ยฉ 2011 D. R. Jensen. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


The canonical correlations between subsets of OLS estimators are identified with design linkage parameters between their regressors. Known collinearity indices are extended to encompass angles between each regressor vector and remaining vectors. One such angle quantifies the collinearity of regressors with the intercept, of concern in the corruption of all estimates due to ill-conditioning. Matrix identities factorize a determinant in terms of principal subdeterminants and the canonical Vector Alienation Coefficients between subset estimatorsโ€”by duality, the Alienation Coefficients between subsets of regressors. These identities figure in the study of D and ๐ท๐‘  as determinant efficiencies for estimators and their subsets, specifically, ๐ท๐‘ -efficiencies for the constant, linear, pure quadratic, and interactive coefficients in eight known small second-order designs. Studies on D- and ๐ท๐‘ -efficiencies confirm that designs are seldom efficient for both. Determinant identities demonstrate the propensity for ๐ท๐‘ -inefficient subsets to be masked through near collinearities in overall D-efficient designs.

1. Introduction

Given {๐˜=๐—๐œท+๐} of full rank with homogeneous, uncorrelated errors, the OLS estimators ๎๐œท are unbiased with second-moment matrix ๎๐‘‰(๐œท)=๐œŽ2(๐—๎…ž๐—)โˆ’1. Such moment matrices pervade experimental design, to include determinants as gauges of ๐ท- and ๐ท๐‘ -efficiencies for estimators and their subsets. Early references trace to [1โ€“4], and more recently to [5โ€“10] and others. Finding ๐ท๐‘ -efficient designs for polynomial models is considered in [11โ€“20], for example. Studies examining the ๐ท๐‘ -efficiencies of ๐ท-efficient designs confirm that designs are seldom efficient for both; see [13, 21โ€“23]. From those beginnings, the study of ๐ท- and ๐ท๐‘ -efficiencies continues apace. To wit, a recent key-word search in the Current Index to Statistics shows in excess of 60 listings from 2006 to 2010, and more than 100 from 2001 to 2010. Moreover, these ideas bear fruit in a widening diversity of applications as evidenced in the following.

To fix ideas, let ๐ท correspond to a polynomial ๐‘ƒ๐‘ of degree ๐‘, namely, โˆ‘๐‘”(๐œ‡)=๐‘๐‘–=0๐›ฝ๐‘–๐‘ก๐‘–. In toxicology studies, a two-stage experiment is proffered in [24], seeking ๐ท-efficiency in estimating ๐‘˜=๐‘+1 overall parameters at the first stage, then ๐ท1-efficiency at the second stage in estimating a critical โ€œthreshold parameter,โ€ using quasilikelihood in nonlinear models. Coupled with this is the ๐ท๐‘˜โˆ’1-efficiency for the remaining ๐‘˜โˆ’1 parameters at the second stage. In related work [25], experiments with ๐‘ chemicals in combination are to be examined along fixed-ratio rays. When restricted to a specified ray, the fundamental hypothesis of noninteracting factors can be rejected when higher-order polynomial terms are required in the total dose-response model ๐‘”(๐œ‡)=๐›ฝ0+๐›ฝ1โˆ‘๐‘ก+๐‘๐‘–=2๐›ฝ๐‘–๐‘ก๐‘– in the linear predictor ๐‘ก. Here ๐ท2 refers to [๐›ฝ0,๐›ฝ1] and ๐ท๐‘โˆ’1 to ๐ท๐‘ -efficiency in the critical estimation of [๐›ฝ2,โ€ฆ,๐›ฝ๐‘], which vanish under the conjectured additivity. Moreover, in [11, 12, 21] ๐ท refers to ๐‘ƒ๐‘˜+1, ๐ท๐‘˜ to ๐‘ƒ๐‘˜, and ๐ท1 to ๐›ฝ๐‘˜+1, the highest-order term in ๐‘ƒ๐‘˜+1, for example. In short, users often are properly concerned with both ๐ท- and ๐ท๐‘ -efficiencies, and connections between these basic criteria deserve further study, to be undertaken here.

Ill-conditioning, as near-collinearity among the columns of ๐—, โ€œcauses crucial elements of ๐—๎…ž๐— to be large and unstable,โ€ โ€œcreating inflated variances,โ€ and estimates that are โ€œvery sensitive to small changes in ๐—,โ€ having โ€œdegraded numerical accuracy;โ€ see [26โ€“28], for example. Diagnostics include the condition number ๐‘1(๐—๎…ž๐—), the ratio of largest to smallest eigenvalues; and the Variance Inflation Factors ฬ‚๐›ฝ{VIF(๐‘—)=๐‘ฃ๐‘—๐‘—๐‘ค๐‘—๐‘—;1โ‰ค๐‘—โ‰ค๐‘} with ๐–=๐—๎…ž๐— and ๐•=(๐—๎…ž๐—)โˆ’1, that is, ratios of actual (๐‘ฃ๐‘—๐‘—) to โ€œidealโ€ (1/๐‘ค๐‘—๐‘—) variances had the columns of ๐— been orthogonal. In models with intercept, โ€œcollinearity with the intercept can quite generally corrupt the estimates of all parameters in the model whether or not the intercept is itself of interest and whether or not the data have been (mean) centered,โ€ as noted in [29].

To the foregoing list of ills from ill-conditioning, we add that not only are designs seldom efficient for both, but ๐ท๐‘ -inefficient estimators may be masked in overall ๐ท-efficient designs, and conversely. This masking may be quantified in terms of structural dependencies, specifically, through determinant identities linking ๐ท- and ๐ท๐‘ -efficiencies to various gages of nonorthogonality of the data. The latter include nonvanishing inner products between columns of regressors, Hotellingโ€™s [30] canonical correlations among OLS solutions, and VIFs. An outline follows.

Section 2 contains supporting material. Details surrounding collinearity diagnostics are topics in Section 3, to include duality of angles between subspaces of the design and parameter spaces, and their connections to VIFs. Section 4 develops basic determinant identities and inequalities of independent interest. Section 5 revisits eight small second-order designs with regard to ๐ท๐‘ -efficiencies in estimating the constant, linear, pure quadratic, and interactive coefficients, to include the masking of inefficient estimators. Though in wide usage, with no apparent accounting for collinearity, these designs are seen to exhibit varying degrees of collinearity of regressors with the constant. Since computations proceed from the design matrix itself, an advantage is that prospective designs can be evaluated beforehand in regard to issues studied here, before committing to an actual experiment. Section 6 concludes with a brief summary.

2. Preliminaries

2.1. Notation

Spaces of note include โ„๐‘˜ as Euclidean ๐‘˜-space; โ„๐‘˜+ as its positive orthant; ๐”ฝ๐‘›ร—๐‘˜ as the real matrices of order (๐‘›ร—๐‘˜); ๐•Š๐‘˜ as the (๐‘˜ร—๐‘˜) real symmetric matrices; and ๐•Š+๐‘˜ as their positive definite varieties. The transpose, inverse, trace, and determinant of ๐€โˆˆ๐•Š+๐‘˜ are ๐€๎…ž,โ€‰๐€โˆ’1, tr(๐€), and |๐€|; and ๐€1/2 is its spectral square root. Special arrays include the unit vector ๐Ÿ๎…ž๐‘›=[1,1,โ€ฆ,1]โˆˆโ„๐‘›, the identity ๐ˆ๐‘› of order (๐‘›ร—๐‘›), the block-diagonal matrix Diag(๐€1,๐€2)โˆˆ๐•Š๐‘˜, the idempotent form ๐๐‘›=(๐ˆ๐‘›โˆ’๐‘›โˆ’1๐Ÿ๐‘›๐Ÿ๎…ž๐‘›), and ๐’ช(๐‘˜) as the real orthogonal group of (๐‘˜ร—๐‘˜) matrices. For ๐—(๐‘›ร—๐‘) of rank ๐‘โ‰ค๐‘›, designate a pseudoinverse as ๐—โ€ , its ordered singular values as ๐œŽ(๐—)={๐œ‰1โ‰ฅ๐œ‰2โ‰ฅโ‹ฏโ‰ฅ๐œ‰๐‘>0}, and by ๐’ฎ๐‘(๐—)โŠ‚โ„๐‘›, the linear span of columns of ๐—. Its condition number is ๐‘2(๐—)=๐œ‰1/๐œ‰๐‘, specifically, ๐‘2(๐—)=[๐‘1(๐—๎…ž๐—)]1/2.

The mean, dispersion matrix, and generalized variance for a random ๐”โˆˆโ„๐‘˜ are designated as ๐ธ(๐”)=๐โˆˆโ„๐‘˜,โ€‰๐‘‰(๐”)=๐šบโˆˆ๐•Š+๐‘˜, and ๐บ๐‘‰(๐”)=|๐šบ|, respectively. To account for dimension, consider ๐บ(๐”)=[๐บ๐‘‰(๐”)]1/๐‘˜=|๐šบ|1/๐‘˜ as a function homogeneous of unit degree. The class M0โˆถ{๐˜=๐›ฝ0๐Ÿ๐‘›+๐—๐œท+๐}, comprising models with intercept and dispersion ๐‘‰(๐)=๐œŽ2๐ˆ๐‘›, is our principal focus. Unless stated otherwise, we take ๐œŽ2=1.0, since variance ratios are scale-invariant. A distinction is drawn between centered and uncentered VIFs, namely, VIF๐‘s and VIF๐‘ขs, the former from columns of ๐— centered to their means. The latter, designated as {VIF๐‘ข(ฬ‚๐›ฝ๐‘—);๐‘—=0,1,โ€ฆ,๐‘˜}, are diagonal elements of (๐—๎…ž0๐—0)โˆ’1 divided by reciprocals of diagonals of ๐—๎…ž0๐—0 itself. These are of subsequent interest. Special distributions on โ„1+ include the Snedecor-Fisher distribution ๐น(โ‹…;๐œˆ1,๐œˆ2,๐œ†) having (๐œˆ1,๐œˆ2) degrees of freedom and noncentrality ๐œ†.

3. Collinearity Diagnostics

Ill-conditioned models {๐˜=๐—๐œท+๐}, burdened with difficulties as cited, trace to nonorthogonality among columns of ๐—. To examine aspects of near collinearity, we first establish duality between design linkage parameters among columns of ๐—, and collinearity among the OLS solutions as quantified by Hotelling's [30] canonical correlations.

3.1. Duality Results

Partition a generic ๐—โˆˆ๐”ฝ๐‘›ร—๐‘ as ๐—=[๐—1,๐—2] with {๐—,๐—1,๐—2} of orders {(๐‘›ร—๐‘),(๐‘›ร—๐‘Ÿ),(๐‘›ร—๐‘ )}, respectively, having ranks {๐‘,๐‘Ÿ,๐‘ } such that ๐‘Ÿโ‰ค๐‘  and ๐‘Ÿ+๐‘ =๐‘<๐‘›. Accordingly, write {๐˜=๐—1๐œท1+๐—2๐œท2+๐}, taking ๐œท๎…ž=[๐œท๎…ž1,๐œท๎…ž2], and denoting by ๐’ฎ๐‘(๐—1) and ๐’ฎ๐‘(๐—2), the subspaces of โ„๐‘› spanned by columns of ๐—1 and ๐—2. We seek a canonical form preserving these subspaces and linkage between (๐—1,๐—2), a geometric concept independent of bases for representing ๐’ฎ๐‘(๐—1) and ๐’ฎ๐‘(๐—2). Accordingly, let ๐†1=(๐—๎…ž1๐—1)โˆ’1/2๐ and ๐†2=(๐—๎…ž2๐—2)โˆ’1/2๐, with ๐โˆˆ๐’ช(๐‘Ÿ) and ๐โˆˆ๐’ช(๐‘ ) to be stipulated. The original model becomes {๐˜=๐™1๐œถ1+๐™2๐œถ2+๐} with ๐™=[๐™1,๐™2] and ๐œถ๎…ž=[๐œถ๎…ž1,๐œถ๎…ž2], such that ๐™1=๐—1๐†1, ๐™2=๐—2๐†2, ๐œถ1=๐†1โˆ’1๐œท1, and ๐œถ2=๐†2โˆ’1๐œท2.

Following [31], cosines of angles between ๐’ฎ๐‘(๐—1) and ๐’ฎ๐‘(๐—2) are found as singular values generated by (๐—1,๐—2), to be designated as design linkage parameters {๐›ฟ1,โ€ฆ,๐›ฟ๐‘Ÿ}. To these ends, observe that ๐—๎…ž๐— in partitioned form transitions into ๐™๎…ž๐™ through ๐—๎…žโŽกโŽขโŽขโŽฃ๐—๐—=๎…ž1๐—1๐—๎…ž1๐—2๐—๎…ž2๐—1๐—๎…ž2๐—2โŽคโŽฅโŽฅโŽฆโŸถโŽกโŽขโŽขโŽฃ๐†๎…ž1๐—๎…ž1๐—1๐†1๐†๎…ž1๐—๎…ž1๐—2๐†2๐†๎…ž2๐—๎…ž2๐—1๐†1๐†๎…ž2๐—๎…ž2๐—2๐†2โŽคโŽฅโŽฅโŽฆ=โŽกโŽขโŽขโŽฃ๐ˆ๐‘Ÿ๐๎…ž๐๐‘๐๎…ž๐‘๎…ž๐๐ˆ๐‘ โŽคโŽฅโŽฅโŽฆ=โŽกโŽขโŽขโŽฃ๐ˆ๐‘Ÿ๐ƒ๐ƒ๎…ž๐ˆ๐‘ โŽคโŽฅโŽฅโŽฆ=๐™๎…ž๐™.(3.1) Here ๐‘=(๐—๎…ž1๐—1)โˆ’1/2๐—๎…ž1๐—2(๐—๎…ž2๐—2)โˆ’1/2; its singular decomposition is ๐‘ = ๐๐ƒ๐๎…ž, where ๐ƒ=[๐ƒ๐›ฟ,๐ŸŽ]; and elements of ๐ƒ๐›ฟ=Diag(๐›ฟ1,โ€ฆ,๐›ฟ๐‘Ÿ) comprise the singular values of ๐‘. In particular, {๐œ™๐‘—=arccos(๐›ฟ๐‘—);1โ‰ค๐‘—โ‰ค๐‘Ÿ} defines the design linkage angles between ๐’ฎ๐‘(๐—1) and ๐’ฎ๐‘(๐—2) as subspaces of โ„๐‘›.

To continue, partition ๎๐‘‰(๐œท)=๐šบ=[๐šบ๐‘–๐‘—] conformably with ๐œท๎…ž=[๐œท๎…ž1,๐œท๎…ž2],โ€‰โ€‰๐œท1โˆˆโ„๐‘Ÿ,๐œท2โˆˆโ„๐‘ ; designate their inner product space as (โ„๐‘ŸโŠ•โ„๐‘ ,(โ‹…,โ‹…)๐šบ), where โ„๐‘ŸโŠ•โ„๐‘  is the direct sum and (โ‹…,โ‹…)๐šบ their inner product, as in Eaton [32, page 409]. Denote by {๐œŒ1,โ€ฆ,๐œŒ๐‘Ÿ} Hotellingโ€™s [30] canonical correlations. Then by Propositionโ€‰โ€‰10.2 of [32], {๐œŒ1,โ€ฆ,๐œŒ๐‘Ÿ} are cosines of angles between (โ„๐‘Ÿ,โ„๐‘ ) as subspaces of (โ„๐‘ŸโŠ•โ„๐‘ ,(โ‹…,โ‹…)๐šบ). In keeping with earlier usage, identify {๐’ฎ๐‘(๐œท1),๐’ฎ๐‘(๐œท2)} with {โ„๐‘Ÿ,โ„๐‘ }. As Hotelling's canonical correlations are invariant under affine transformations {๐œท1โ†’๐€๐œท1+๐œ1,๐œท2โ†’๐๐œท2+๐œ2}, parameters may be redefined linearly, preserving subspaces, thus leaving the canonical correlations invariant. Retracing steps leading to the canonical design model embodied in (3.1), but now to preserve {๐’ฎ๐‘(๐œท1),๐’ฎ๐‘(๐œท2)}, it thus suffices to begin with the canonical model {๐˜=๐™1๐œถ1+๐™2๐œถ2+๐}, where ๎๐‘‰(๐œถ)=๐œŽ2(๐™๎…ž๐™)โˆ’1 with ๐™๎…ž๐™ as the rightmost matrix of (3.1).

We next establish connections between the design linkage parameters ๐ƒ๐›ฟ from (3.1), and the corresponding canonical correlations ๐ƒ๐œŒ=Diag(๐œŒ1,โ€ฆ,๐œŒ๐‘Ÿ), as derived eventually from ๐šบ=(๐—๎…ž๐—)โˆ’1. A critical duality result is encoded in the following.

Theorem 3.1. Consider the design linkage parametersโ€‰โ€‰๐ƒ๐›ฟ between {๐’ฎ๐‘(๐—1),๐’ฎ๐‘(๐—2)} as subspaces of โ„๐‘› and Hotellingโ€™s [30] canonical correlationsโ€‰โ€‰๐ƒ๐œŒ between {๐’ฎ๐‘(๐œท1),๐’ฎ๐‘(๐œท2)} as subspaces of (โ„๐‘ŸโŠ•โ„๐‘ ,(โ‹…,โ‹…)๐šบ). Then ๐ƒ๐›ฟ and ๐ƒ๐œŒ coincide.

Proof. In view of invariance of {๐œŒ1,โ€ฆ,๐œŒ๐‘Ÿ} under nonsingular linear transformations of ๐œท1โˆˆโ„๐‘Ÿ and of ๐œท2โˆˆโ„๐‘ , canonical correlations between (๎๐œท1,๎๐œท2) proceed as in expression (3.1), but beginning instead on the left with ๎๐‘‰(๐œถ) = (๐™๎…ž๐™)โˆ’1โ€‰โ€‰in lieu of ๐—๎…ž๐—. Specifically, with ๐ƒ = [๐ƒ๐›ฟ,๐ŸŽ], and using rules for block-partitioned inverses, we have ๎€ท๐™๎…ž๐™๎€ธโˆ’1=โŽกโŽขโŽขโŽฃ๐ˆ๐‘Ÿ๎€บ๐ƒ๐›ฟ๎€ป๎€บ๐ƒ,๐ŸŽ๐›ฟ๎€ป,๐ŸŽ๎…ž๐ˆ๐‘ โŽคโŽฅโŽฅโŽฆโˆ’1=โŽกโŽขโŽขโŽฃ๎€ท๐ˆ๐‘Ÿโˆ’๐ƒ2๐›ฟ๎€ธโˆ’1๎€ท๐ˆโˆ’๐ƒ๐‘ โˆ’๐ƒ0๎€ธโˆ’1โˆ’๐ƒ๎…ž๎€ท๐ˆ๐‘Ÿโˆ’๐ƒ2๐›ฟ๎€ธโˆ’1๎€ท๐ˆ๐‘ โˆ’๐ƒ0๎€ธโˆ’1โŽคโŽฅโŽฅโŽฆโŸถโŽกโŽขโŽขโŽฃ๐ˆ๐‘Ÿ๎€บ๐ƒ๐›ฟ๎€ป๎€บ๐ƒ,๐ŸŽ๐›ฟ๎€ป,๐ŸŽ๎…ž๐ˆ๐‘ โŽคโŽฅโŽฅโŽฆ,(3.2) where equality at the first step follows using ๐ƒ๐ƒ๎…ž=๐ƒ2๐›ฟ and ๐ƒ๎…ž๐ƒ=Diag(๐ƒ2๐›ฟ,๐ŸŽ)=๐ƒ0. The succeeding step utilizes the factors (๐ˆ๐‘Ÿโˆ’๐ƒ2๐›ฟ)1/2 and (๐ˆ๐‘ โˆ’๐ƒ0)1/2, taking the principal diagonal blocks of (๐™๎…ž๐™)โˆ’1 into (๐ˆ๐‘Ÿ,๐ˆ๐‘ ) as in the rightmost matrix of (3.2), and its off-diagonal block from ๎€ท๐ˆ๐‘Ÿโˆ’๐ƒ2๐›ฟ๎€ธ1/2๎€บ๐ƒ๐›ฟ๐ˆ,๐ŸŽ๎€ป๎€ท๐‘ โˆ’๐ƒ0๎€ธโˆ’1๎€ท๐ˆ๐‘ โˆ’๐ƒ0๎€ธ1/2=๎€บ๐ƒ๐›ฟ๎€ป,๐ŸŽ,(3.3) since diagonal matrices commute. But the off-diagonal block is precisely [๐ƒ๐œŒ,๐ŸŽ], the canonical correlations between (๎๐œท1,๎๐œท2), to complete our proof.

For subsequent reference, designate ๐œน(๐—1โˆถ๐—2)=(๐›ฟ1,โ€ฆ,๐›ฟ๐‘Ÿ) and ๎๐œท๐†(1โˆถ๎๐œท2)=(๐œŒ1,โ€ฆ,๐œŒ๐‘Ÿ). Moreover, the foregoing analysis applies for models ๐—0=[๐Ÿ๐‘›,๐—] in M0, where ๐‘Ÿ=1 and ๐‘ =๐‘˜ as partitioned. In short, we have the following equivalences.

Corollary 3.2. (i) Consider the design linkage parameters {cos(๐œ™๐‘—)=๐›ฟ๐‘—;1โ‰ค๐‘—โ‰ค๐‘Ÿ}, gaging collinearity between {๐’ฎ๐‘(๐—1),๐’ฎ๐‘(๐—2)} as subspaces of โ„๐‘› and the canonical correlations {cos(๐œ™๐‘—)=๐œŒ๐‘—;1โ‰ค๐‘—โ‰ค๐‘Ÿ}, between {๐’ฎ๐‘(๐œท1),๐’ฎ๐‘(๐œท2)} as subspaces of (โ„๐‘ŸโŠ•โ„๐‘ ,(โ‹…,โ‹…)๐šบ). Then angles between these pairs of subspaces correspond one-to-one, that is, {๐œ™๐‘—=arccos(๐›ฟ๐‘—)=arccos(๐œŒ๐‘—);1โ‰ค๐‘—โ‰ค๐‘Ÿ}.
(ii) For models ๐—0=[๐Ÿ๐‘›,๐—] in M0, the element ๐œน(๐Ÿ๐‘›โˆถ๐—)=๐›ฟ1 generates the angle cos(๐œ™1)=๐›ฟ1 between the regressor vectors and the constant vector. Equivalently, this is given by cos(๐œ™1)=๐œŒ1ฬ‚๐›ฝ=๐†(0โˆถ๎๐œท) from duality.

3.2. Collinearity Indices

Stewart [33] reexamined numerical aspects of ill-conditioning, to the following effects for ๐—0=[๐Ÿ๐‘›,๐—]. Taking ๐—โ€ 0 = (๐—๎…ž0๐—0)โˆ’1๐—๎…ž0 as the pseudoinverse of note, and letting ๐ฑโ€ ๐‘— be its ๐‘—th row, each collinearity index in the collection ๎‚†๐œ…๐‘—=โ€–โ€–๐ฑ๐‘—โ€–โ€–โ‹…โ€–โ€–๐ฑโ€ ๐‘—โ€–โ€–๎‚‡;๐‘—=0,1,โ€ฆ,๐‘˜(3.4) is constructed to be scale-invariant. Clearly โ€–๐ฑโ€ ๐‘—โ€–2 is found along the principal diagonal of [(๐—โ€ 0)(๐—โ€ 0)๎…ž] = (๐—๎…ž0๐—0)โˆ’1. In addition, the conventional VIF๐‘ขs are squares of the collinearity indices, that is, {VIF๐‘ข(ฬ‚๐›ฝ๐‘—)=๐œ…2๐‘—;๐‘—=0,1,โ€ฆ,๐‘˜}. In particular, since ๐ฑ0=๐Ÿ๐‘› in ๐—0, we have ๐œ…20 = ๐‘›โ€–๐ฑโ€ 0โ€–2.

Transcending Stewartโ€™s analysis, we connect his collinearity indices to angles between subspaces as follows. Choose a typical ๐ฑ๐‘— in ๐—0; rearrange ๐—0 as [๐ฑ๐‘—,๐—[๐‘—]] and similarly ๐œท๎…ž as [๐›ฝ๐‘—,๐œท๎…ž[๐‘—]]; and seek elements of โŽงโŽชโŽจโŽชโŽฉ๐๎…ž๐‘—๎€ท๐—๎…ž0๐—0๎€ธโˆ’1๐๐‘—=โŽกโŽขโŽขโŽฃ๐ฑ๎…ž๐‘—๐ฑ๐‘—๐ฑ๎…ž๐‘—๐—[๐‘—]๐—๎…ž[๐‘—]๐ฑ๐‘—๐—๎…ž[๐‘—]๐—[๐‘—]โŽคโŽฅโŽฅโŽฆโˆ’1โŽซโŽชโŽฌโŽชโŽญ;๐‘—=0,1,โ€ฆ,๐‘˜(3.5) as reordered by each permutation matrix ๐๐‘—. From the clockwise rule, the (1,1) element of each inverse is ๎‚†๎€บ๐ฑ๎…ž๐‘—๎€ท๐ˆ๐‘›โˆ’๐๐‘—๎€ธ๐ฑ๐‘—๎€ปโˆ’1=๎€บ๐ฑ๎…ž๐‘—๐ฑ๐‘—โˆ’๐ฑ๎…ž๐‘—๐๐‘—๐ฑ๐‘—๎€ปโˆ’1=โ€–โ€–๐ฑโ€ ๐‘—โ€–โ€–2๎‚‡;๐‘—=0,1,โ€ฆ,๐‘˜,(3.6) where ๐๐‘— = ๐—[๐‘—][๐—๎…ž[๐‘—]๐—[๐‘—]]โˆ’1๐—๎…ž[๐‘—] is the projection operator onto the subspace ๐’ฎ๐‘(๐—[๐‘—])โŠ‚โ„๐‘›. These relationships in turn enable us to connect {๐œ…2๐‘—;๐‘—=0,1,โ€ฆ,๐‘˜} to the geometry of ill-conditioning as follows.

Theorem 3.3. For models in M0, let {VIF๐‘ข(ฬ‚๐›ฝ๐‘—)=๐œ…2๐‘—;๐‘—=0,1,โ€ฆ,๐‘˜} be conventional VIF๐‘ขs in terms of Stewartโ€™s collinearity indices. These in turn quantify collinearities between subspaces through angles (in deg) as follows.
(i)Angles between [๐ฑ๐‘—,๐—[๐‘—]] are given by ๐œ™๐‘—=arccos[(1โˆ’1/๐œ…2๐‘—)1/2], in succession for {๐‘—=0,1,โ€ฆ,๐‘˜}.(ii)Equivalently, {๐œ…2๐‘—=1/[1โˆ’๐œน2(๐ฑ๐‘—โˆถ๐—[๐‘—])]=1/[1โˆ’๐†2(ฬ‚๐›ฝ๐‘—โˆถ๎๐œท[๐‘—])];๐‘—=0,1,โ€ฆ,๐‘˜}.(iii)In particular, ๐œ™0=arccos[(1โˆ’1/๐œ…20)1/2] quantifies the degree of collinearity between the regressor vectors and the constant vector.

Proof. From the geometry of the right triangle formed by (๐ฑ๐‘—,๐๐‘—๐ฑ๐‘—), the squared lengths satisfy โ€–๐ฑ๐‘—โ€–2=โ€–๐๐‘—๐ฑ๐‘—โ€–2+๐‘…๐‘†๐‘—, where ๐‘…๐‘†๐‘—=โ€–๐ฑ๐‘—โˆ’๐๐‘—๐ฑ๐‘—โ€–2 is the residual sum of squares from the projection. Accordingly, the principal angle between (๐ฑ๐‘—,๐๐‘—๐ฑ๐‘—) is given by ๎€ท๐œ™cos๐‘—๎€ธ=๐ฑ๎…ž๐‘—๐๐‘—๐ฑ๐‘—โ€–โ€–๐ฑ๐‘—โ€–โ€–โ‹…โ€–โ€–๐๐‘—๐ฑ๐‘—โ€–โ€–=โ€–โ€–๐๐‘—๐ฑ๐‘—โ€–โ€–โ€–โ€–๐ฑ๐‘—โ€–โ€–=๎ƒฉ1โˆ’๐‘…๐‘†๐‘—โ€–โ€–๐ฑ๐‘—โ€–โ€–2๎ƒช1/2=๎ƒฉ11โˆ’๐œ…2๐‘—๎ƒช1/2(3.7) for {๐‘—=0,1,โ€ฆ,๐‘˜}, to give conclusion (i) and conclusion (ii) by duality. Conclusion (iii) follows on specializing (๐ฑ0,๐0๐ฑ0) with ๐ฑ0=๐Ÿ๐‘› and ๐0=๐—(๐—๎…ž๐—)โˆ’1๐—๎…ž, to complete our proof.

Remark 3.4. The foregoing developments specialize from Section 3.1 in that the partition [๐ฑ๐‘—,๐—[๐‘—]] always has ๐‘Ÿ=1 and ๐‘ =๐‘˜, giving a single angle ๐œ™๐‘—. Rules-of-thumb in common use for problematic VIFs include those exceeding 10, as in [34], or even 4 as in [35], for example. In angular measure, these correspond respectively to ๐œ™๐‘—<18.435deg and ๐œ™๐‘—<30.0deg.

3.3. Case Study 1

Consider the model M0โˆถ{๐‘Œ๐‘–=๐›ฝ0+๐›ฝ1๐‘‹1+๐›ฝ2๐‘‹2+๐œ–๐‘–}, the design ๐—0=[๐Ÿ5,๐—1,๐—2] of order (5ร—3), and ๐—๎…ž0๐—0 and its inverse as in ๐—๎…ž0=โŽกโŽขโŽขโŽขโŽขโŽฃโŽคโŽฅโŽฅโŽฅโŽฅโŽฆ1111110.50.510โˆ’11100,๐—๎…ž0๐—0=โŽกโŽขโŽขโŽขโŽขโŽฃโŽคโŽฅโŽฅโŽฅโŽฅโŽฆ,๎€ท๐—53132.50103๎…ž0๐—0๎€ธโˆ’1=โŽกโŽขโŽขโŽขโŽขโŽฃโŽคโŽฅโŽฅโŽฅโŽฅโŽฆ.0.9375โˆ’1.1250โˆ’0.3125โˆ’1.12501.75000.3750โˆ’0.31250.37500.4375(3.8) Note first that VIF๐‘ข(ฬ‚๐›ฝ0)=๐œ…20=0.9375ร—5=4.6875. Next apply first principles to find ๐œน2๎€ท๐Ÿ5โˆถ๎€บ๐—1,๐—2=๎€ป๎€ธ(5)โˆ’1๎‚ƒ๎‚„โŽกโŽขโŽขโŽฃโŽคโŽฅโŽฅโŽฆ312.5003โˆ’1โŽกโŽขโŽขโŽฃ31โŽคโŽฅโŽฅโŽฆ๐†=0.786666,2๎€ทฬ‚๐›ฝ0โˆถ๎€บฬ‚๐›ฝ1,ฬ‚๐›ฝ2=๎€ป๎€ธ(0.9375)โˆ’1๎‚ƒ๎‚„โŽกโŽขโŽขโŽฃโŽคโŽฅโŽฅโŽฆโˆ’1.1250โˆ’0.31251.75000.37500.37500.4375โˆ’1โŽกโŽขโŽขโŽฃโŽคโŽฅโŽฅโŽฆโˆ’1.1250โˆ’0.3125=0.786666,(3.9) both equal to 1โˆ’(1/๐œ…20) as in Theorem 3.3(ii). The remaining VIF๐‘ขs are found directly as VIF๐‘ข(ฬ‚๐›ฝ1)=1.7500ร—2.5=4.3750 and VIF๐‘ข(ฬ‚๐›ฝ2)=0.4375ร—3=1.3125. Using duality and earlier findings, we further compute ๐œน2๎€ท๐—1โˆถ๎€บ๐Ÿ5,๐—2๎€ป๎€ธ=๐†2๎€ทฬ‚๐›ฝ1โˆถ๎€บฬ‚๐›ฝ0,ฬ‚๐›ฝ2๎ƒฉ1๎€ป๎€ธ=1โˆ’๐œ…21๎ƒช1=1โˆ’๐œน4.3750=0.771429,2๎€ท๐—2โˆถ๎€บ๐Ÿ5,๐—1๎€ป๎€ธ=๐†2๎€ทฬ‚๐›ฝ2โˆถ๎€บฬ‚๐›ฝ0,ฬ‚๐›ฝ1๎ƒฉ1๎€ป๎€ธ=1โˆ’๐œ…22๎ƒช1=1โˆ’1.3125=0.238095,(3.10) thereby preempting the need to undertake singular decompositions as required heretofore.

4. Determinant Identities

4.1. Background

The generalized variance, as a design criterion for {๐˜=๐—๐œท+๐}, rests in part on the geometry of ellipsoids of the type ๎‚ป๐‘…(๐œท)=๐œทโˆˆโ„๐‘˜โˆถ๎‚€๎๎‚๐œทโˆ’๐œท๎…ž๐—๎…ž๐—๎‚€๎๎‚๐œทโˆ’๐œทโ‰ค๐‘2๎‚ผ.(4.1) Choices for ๐‘2 in common usage give first (i) a confidence region for ๐œท, whose normal-theory confidence coefficient is 1โˆ’๐›ผ on taking ๐‘2=๐‘†2๐‘2๐›ผ, with ๐‘†2 as the residual mean square and ๐‘2๐›ผ the 100(1โˆ’๐›ผ) percentage point of ๐น(โ‹…;๐‘˜,๐‘›โˆ’๐‘˜); and otherwise admitting a lower Chebychev bound as in [36, pageโ€‰โ€‰92]. The alternative choice ๐‘2=๐‘˜+2 gives (ii) Cramรฉrโ€™s [37] ellipsoid of concentration for ๎๐œท, that is, the measure uniform over ๐‘…(๐œท) having the same mean and dispersion matrix as ๎๐œท. The generalized variance ๎๎๐บ๐‘‰(๐œท)=|๐‘‰(๐œท)| is proportional to the squared volumes of these ellipsoids, smaller volumes reflecting tighter concentrations.

4.2. Factorizations

To continue, let some ๎๐“(๐˜)=๐œฝโˆˆโ„๐‘˜ be random having ๎๐ธ(๐œฝ)=๐œฝ and ๎๐‘‰(๐œฝ)=๐šบโˆˆ๐•Š+๐‘˜; partition ๐œฝ๎…ž=[๐œฝ๎…ž1,๐œฝ๎…ž2] and ๐šบ=[๐šบ๐‘–๐‘—] conformably, with ๐œฝ1โˆˆโ„๐‘Ÿ and ๐œฝ2โˆˆโ„๐‘  such that ๐‘Ÿโ‰ค๐‘  and ๐‘Ÿ+๐‘ =๐‘˜; and let ๎๐บ(๐œฝ)=|๐šบ|1/๐‘˜. The canonical correlations [30], as singular values of ๐šบโˆ’1/211๐šบ12๐šบโˆ’1/222, are now to be designated as ๐†(1โˆถ2) = [๐œŒ1,โ€ฆ,๐œŒ๐‘Ÿ],โ€‰โ€‰in lieu of ๎๐œท๐†(1โˆถ๎๐œท2), and to be ordered as {๐œŒ1โ‰ฅ๐œŒ2โ‰ฅโ‹ฏโ‰ฅ๐œŒ๐‘Ÿโ‰ฅ0}. Moreover, the quantity ๐œธ(1โˆถ2)=ฮ ๐‘Ÿ๐‘–=1(1โˆ’๐œŒ2๐‘–) is the Vector Alienation Coefficient of Hotelling [30]. The factorization |๐šบ|=|๐šบ11||๐šบ22| for ๐šบ=Diag(๐šบ11,๐šบ22) extends directly as an upper bound for any ๐šบ=[๐šบ๐‘–๐‘—], with further ramifications as follows.

Theorem 4.1. Consider ๎๐œฝ๎…ž๎๐œฝ=[๎…ž1,๎๐œฝ๎…ž2]โˆˆโ„๐‘˜ having ๎๐œฝ๐ธ(๎…ž)=[๐œฝ๎…ž1,๐œฝ๎…ž2] and ๎๐‘‰(๐œฝ)=[๐šบ๐‘–๐‘—], such that ๐œฝ1โˆˆโ„๐‘Ÿ and ๐œฝ2โˆˆโ„๐‘  with ๐‘Ÿโ‰ค๐‘  and ๐‘Ÿ+๐‘ =๐‘˜.
(i)The determinant of ๐šบ = [๐šบ๐‘–๐‘—] admits the factorization ||๐šบ||=||๐šบ11||||๐šบ22||๐œธ(1โˆถ2)(4.2) such that |๐šบ|โ‰ค|๐šบ11||๐šบ22| and ๐œธ(1โˆถ2)=ฮ ๐‘Ÿ๐‘–=1(1โˆ’๐œŒ2๐‘–)โ‰ค1.(ii)If ๐šบ=Diag(๐šบ11,๐šบ22), then ๎๐บ(๐œฝ) is the geometric mean ๎๎๐œฝ๐บ(๐œฝ)=[๐บ(1)]๐‘Ÿ/๐‘˜๎๐œฝ[๐บ(2)]๐‘ /๐‘˜ of the quantities ๎๐œฝ๐บ(1) and ๎๐œฝ๐บ(2).(iii)Generally, for any ๐šบ, the quantity ๎๐บ(๐œฝ) becomes ๐บ๎‚€๎๐œฝ๎‚=๎‚ƒ๐บ๎‚€๎๐œฝ1๎‚๎‚„๐‘Ÿ/๐‘˜๎‚ƒ๐บ๎‚€๎๐œฝ2๎‚๎‚„๐‘ /๐‘˜[]๐œธ(1โˆถ2)1/๐‘˜(4.3) in terms of ๎๐œฝ{๐บ(1๎๐œฝ),๐บ(2),๐œธ(1โˆถ2)}.(iv)If ๎๎๐œฝ๐œฝ=[๎…ž1,๎๐œฝ๎…ž2,๎๐œฝ3]๎…ž and ๐šบ=[๐šบ๐‘–๐‘—;1โ‰ค๐‘–,๐‘—โ‰ค3] are partitioned conformably, with ๎๐œฝ1โˆˆโ„๐‘Ÿ,โ€‰โ€‰๎๐œฝ2โˆˆโ„๐‘ , and ๎๐œฝ3โˆˆโ„๐‘ก, such that ๐‘Ÿ+๐‘ +๐‘ก=๐‘˜, then ๎๐บ(๐œฝ) admits the factorization ๐บ๎‚€๎๐œฝ๎‚=๎‚ƒ๐บ๎‚€๎๐œฝ1๎‚๎‚„๐‘Ÿ/๐‘˜๎‚ƒ๐บ๎‚€๎๐œฝ2๎‚๎‚„๐‘ /๐‘˜๎‚ƒ๐บ๎‚€๎๐œฝ3๎‚๎‚„๐‘ก/๐‘˜[]๐œธ(1โˆถ23)๐œธ(2โˆถ3)1/๐‘˜,(4.4) with ๐œธ(1โˆถ23) and ๐œธ(2โˆถ3) as the Vector Alienation Coefficients between {๎๐œฝ1๎๐œฝ,[2,๎๐œฝ3]} and between {๎๐œฝ2,๎๐œฝ3}, respectively.

Proof. As in Section 3.1 with ๐‘=๐šบโˆ’1/211๐šบ12๐šบโˆ’1/222=๐๐ƒ๐๎…ž and ๐ƒ=[๐ƒ๐œŒ,๐ŸŽ], we have โŽกโŽขโŽขโŽฃ๐šบ11๐šบ12๐šบ21๐šบ22โŽคโŽฅโŽฅโŽฆ=โŽกโŽขโŽขโŽฃ๐–1๐ŸŽ๐ŸŽ๐–2โŽคโŽฅโŽฅโŽฆโŽกโŽขโŽขโŽฃ๐ˆ๐‘Ÿ๐ƒ๐ƒ๎…ž๐ˆ๐‘ โŽคโŽฅโŽฅโŽฆโŽกโŽขโŽขโŽฃ๐–๎…ž1๐ŸŽ๐ŸŽ๐–๎…ž2โŽคโŽฅโŽฅโŽฆ(4.5) with ๐–1=๐šบ1/211๐ and ๐–2=๐šบ1/222๐. The middle factor on the right has determinant |๐ˆ๐‘Ÿโˆ’๐ƒ๐ƒ๎…ž|=ฮ ๐‘Ÿ๐‘–=1(1โˆ’๐œŒ2๐‘–) from the clockwise rule, so that |๐šบ|=|๐šบ11||๐šบ22|ฮ ๐‘Ÿ๐‘–=1(1โˆ’๐œŒ2๐‘–) to give conclusion (i). Conclusion (ii) follows directly from ๎๎๐บ(๐œฝ)=[๐บ๐‘‰(๐œฝ)]1/๐‘˜, and conclusion (iii) on combining (i) and (ii). Conclusion (iv) now follows on applying (iii) twice, first on partitioning ๎๐œฝ into {๎๐œฝ1๎๐œฝ,[2,๎๐œฝ3]}, whose canonical correlations are ๐†(1โˆถ23), then [๎๐œฝ2,๎๐œฝ3] into {๎๐œฝ2,๎๐œฝ3} having canonical correlations ๐†(2โˆถ3), to complete our proof.

Remark 4.2. In short, Theorem 4.1 links determinants and principal subdeterminants precisely through angles between subspaces. Moreover, arguments leading to conclusion (iv) may be iterated recursively to achieve a hierarchical decomposition for four or more factors, as in the following with ๐‘˜=๐‘Ÿ+๐‘ +๐‘ก+๐‘ฃ, namely, ๐บ๎‚€๎๐œฝ๎‚=๎‚ƒ๐บ๎‚€๎๐œฝ1๎‚๎‚„๐‘Ÿ/๐‘˜๎‚ƒ๐บ๎‚€๎๐œฝ2๎‚๎‚„๐‘ /๐‘˜๎‚ƒ๐บ๎‚€๎๐œฝ3๎‚๎‚„๐‘ก/๐‘˜๎‚ƒ๐บ๎‚€๎๐œฝ4๎‚๎‚„๐‘ฃ/๐‘˜[]๐œธ(1โˆถ234)๐œธ(2โˆถ34)๐œธ(3โˆถ4)1/๐‘˜.(4.6)

Remark 4.3. Hotellingโ€™s [30] Vector Alienation Coefficient ๐œธ(1โˆถ2)=ฮ ๐‘Ÿ๐‘–=1(1โˆ’๐œŒ2๐‘–) is a composite index of linkage between {๐’ฎ๐‘(๐œท1),๐’ฎ๐‘(๐œท2)} as subspaces of (โ„๐‘ŸโŠ•โ„๐‘ ,(โ‹…,โ‹…)๐šบ), decreasing in each {๐œŒ2๐‘–;1โ‰ค๐‘–โ‰ค๐‘Ÿ}. Equivalently, duality asserts that ๐œธ(1โˆถ2)=ฮ ๐‘Ÿ๐‘–=1(1โˆ’๐›ฟ2๐‘–) is the identical composite index of linkage between {๐’ฎ๐‘(๐—1),๐’ฎ๐‘(๐—2)} as subspaces of โ„๐‘›.

Theorem 4.1 anticipates that ๐ท๐‘ -inefficient subset estimators may be masked in a design exhibiting good overall ๐ท-efficiency. Conversely, a ๐ท๐‘ -inefficient subset may contraindicate, incorrectly, the overall ๐ท-efficiency of a design. Details are provided in case studies to follow.

5. Case Studies

5.1. The Setting

Our tools are informative in input-output studies. In particular, specify {๐˜=๐—0๐œท+๐} as a second-order model ๐‘Œ(๐‘ฅ1,๐‘ฅ2,๐‘ฅ3) in three regressors and ๐‘=10 parameters, namely, ๎€ฝ๐‘Œ๐‘–=๐›ฝ0+๐›ฝ1๐‘ฅ๐‘–1+๐›ฝ2๐‘ฅ๐‘–2+๐›ฝ3๐‘ฅ๐‘–3+๐›ฝ11๐‘ฅ2๐‘–1+๐œท22๐‘ฅ2๐‘–2+๐›ฝ33๐‘ฅ2๐‘–3+๐›ฝ12๐‘ฅ๐‘–1๐‘ฅ๐‘–2+๐›ฝ13๐‘ฅ๐‘–1๐‘ฅ๐‘–3+๐›ฝ23๐‘ฅ๐‘–2๐‘ฅ๐‘–3+๐œ–๐‘–๎€พ.;๐‘–=1,2,โ€ฆ,๐‘›(5.1) Next partition ๐œท๎…ž=[๐›ฝ0,๐œท๎…ž๐ฟ,๐œท๎…ž๐‘„,๐œท๎…ž๐ผ] with ๐œท๎…ž๐ฟ=[๐›ฝ1,๐›ฝ2,๐›ฝ3] as slopes; ๐œท๎…ž๐‘„=[๐›ฝ11,๐›ฝ22,๐›ฝ33] as pure quadratic terms reflecting diminishing (โˆ’) or increasing (+) returns to inputs; and ๐œท๎…ž๐ผ=[๐›ฝ12,๐›ฝ13,๐›ฝ23] as interactive terms reflecting synergistic (+) or antagonistic (โˆ’) effects for pairs of regressors in combination. Further let ๐œท๎…ž๐‘€=[๐œท๎…ž๐ฟ,๐œท๎…ž๐‘„,๐œท๎…ž๐ผ] exclusive of ๐›ฝ0, the latter a base line for ๐‘Œ(0,0,0). We proceed under conventional homogeneous and uncorrelated errors, the minimizing solution ๎๐œท=(๐—๎…ž0๐—0)โˆ’1๐—๎…ž0๐˜ being unbiased with ๎๐‘‰(๐œท)=๐œŽ2(๐—๎…ž0๐—0)โˆ’1. We take ๐œŽ2, although unknown, to reflect natural variability in experimental materials and protocol, and thus applicable in a given setting independently of the choice of design. Accordingly, for present purposes we may standardize to ๐œŽ2=1.0 for reasons cited earlier.

5.2. The Designs

Early polynomial response designs made use of factorial experiments, setting levels as needed to meet the required degree. For example, the second-order model (5.1) in three regressors would require 33=27 runs. However, in the early 1950s such designs were seen to be excessive, in carrying redundant interactions beyond the pairs required in the model (5.1). In industrial and other settings where parsimony is desired, several small second-order designs have evolved, often on appending a few additional runs to two-level factorials or fractions thereof.

Eight such small designs of note here are the hybrids (H310,H311B) of [38], the small composite SCD [39], the BBD [40], the central composite rotatable design CCD [41], and designs ND [42], HD [43], and BDD [44]. The designs [H310,H311B,SCD,BBD,CCD,ND,HD,BDD] have numbers of runs as [11,11,11,13,15,11,11,11], respectively. These follow on adding a center run to all but design ND, rendering all as unsaturated having at least one degree of freedom for error. Specifically, the design ND of [42] already has 11 runs and is unsaturated. All designs have been scaled to span the same range for each regressor; and none strictly dominates another under the positive definite dispersion ordering. All determinants as listed derive from the respective ๎๐‘‰(๐œท) = (๐—๎…ž0๐—0)โˆ’1 and its submatrices. Subset efficiencies for {๐œท๐ฟ,๐œท๐‘„,๐œท๐ผ} were examined in [45] for selected designs using criteria other than ๐ท- and ๐ท๐‘ -efficiencies. Our usage here, as elsewhere in the literature, considers ๎๐บ๐‘‰(๐œท) and ๎๐บ(๐œท) to be efficiency indices for ๎๐œท specific to a particular design, to include subsets {๎๐œท๐‘–;๐‘–โˆˆ๐–จ}, and smaller values reflect greater efficiencies through smaller volumes of concentration ellipsoids. On the other hand, the comparative efficiencies of two designs for estimating ๐œท or {๐œท๐‘–;๐‘–โˆˆ๐–จ} are found as ratios of these quantities.

5.3. Numerical Studies

Details for these designs are listed in the accompanying tables. Table 1 gives values ๐บ(โ‹…)=[๐บ๐‘‰(โ‹…)]1/dim for ๎๐œท and selected subsets, with dim as the order of the determinant. Also listed are angles ๐œ™(๐Ÿ๐‘›โˆถ๐—)deg between regressors and the constant, to be noted subsequently. Table 2 displays the squared canonical correlations ๐†2(๎๐œท๐‘–โˆถ๎๐œท๐‘—) between designated subsets, and Table 3 the corresponding Vector Alienation Coefficient ๎๐œท๐œธ(๐‘–โˆถ๎๐œท๐‘—)=ฮ ๐‘Ÿ๐‘–=1(1โˆ’๐œŒ2๐‘–), for specified pairs. Here {0L,QI} refers to the pair ฬ‚๐›ฝ{[0,๎๐œท๐ฟ๎๐œท],[๐‘„,๎๐œท๐ผ]}, for example. Moreover, values of the composite indices ๎๐œท๐œธ(๐‘–โˆถ๎๐œท๐‘—) = ๐œธ(๐—๐‘–โˆถ๐—๐‘—), if much less than unity, serve to alert the user as to potential problems with ill-conditioning.

Table 1: Roots ๎๐บ(๐œท) and ๎๐œท๐บ(๐‘–) of generalized variances for ๎๐œท and subset estimators {๎๐œท๐‘–โˆˆโ„๐‘ก}, and angles ๐œ™(1๐‘›โˆถ๐—)๐‘‘๐‘’๐‘” between regressors and the constant, against reference values ๐œ™โˆ—โˆ—<18.44 and ๐œ™โˆ—<30.00, in eight small second-order designs.
Table 2: Squared canonical correlations between designated subsets (๎๐œท๐‘–,๎๐œท๐‘—) of estimators for eight small second-order designs.
Table 3: Vector Alienation Coefficients ๎๐œท๐›พ(๐‘–โˆถ๎๐œท๐‘—) between subsets (๎๐œท๐‘–,๎๐œท๐‘—) of estimators, and the factor ๐‘Ž๐‘๐‘=[๐›พ(0โˆถ๐ฟ๐‘„๐ผ)๐›พ(๐ฟโˆถ๐‘„๐ผ)๐›พ(๐‘„โˆถ๐ผ)]1/10, for each of eight small second-order designs.
5.3.1. An Overview

To fix ideas, observe for the CCD that ๎๐œท๐บ(๐‘„,๎๐œท๐ผ)=1.13633, ๎๐œท๐บ(๐‘„)=1.03300, and ๎๐œท๐บ(๐ผ)=1.25000 from Table 1. These not only are comparable in magnitude, but are commensurate, in having been adjusted for dimensions and thus homogeneous of unit degree, as are all entries in Table 1. Moreover, since (๎๐œท๐‘„,๎๐œท๐ผ) are uncorrelated and ๎๐œท๐›พ(๐‘„,๎๐œท๐ผ) = 1.0 for the CCD from Table 3, ๎๐œท๐บ(๐‘„,๎๐œท๐ผ) is the geometric mean 1.13633=(1.03300)3/6(1.25000)3/6 from Theorem 4.1(ii). A further rough spot check of Table 1 may be summarized as follows.

Summary Properties
(P1)Compared with ๎๐บ(๐œท), values for ฬ‚๐›ฝ๐บ(0) appear excessive throughout.(P2)Values for ๎๐œท๐บ(๐ฟ) are roughly comparable across designs.(P3)The eight designs sort essentially into two groups.(P4)Designs {H310,H311B,SCD,BBD,CCD} overall are comparatively ๐ท- and ๐ท๐‘ -efficient, with the noted exception being ๎๐œท๐บ(๐ผ)=4.16667 for the SCD.(P5)The designs {ND,HD,BDD} are considerably less ๐ท-efficient, with their generalized variances ๎๐บ๐‘‰(๐œท) being {1192.09,4768.37,2886.03}, respectively, in comparison with {57.342,11.852,74.422,2.722,0.523} for the remaining designs; and each of the former is burdened by unequivocal ๐ท๐‘ -inefficiency for ๐œท๐‘„, to be treated subsequently.

5.3.2. Further Details

We next examine Hartleyโ€™s [39] SCD in some detail, first in terms of generalized variances. Values for ๎๐บ๐‘‰(๐œท),โ€‰โ€‰ฬ‚๐›ฝ๐บ๐‘‰(0), and ๎๐œท๐บ๐‘‰(๐‘€) appear in the first row of Table 4, along with ฬ‚๐›ฝ๐œธ(0โˆถ๎๐œท๐‘€)=(1.0โˆ’0.909090)=0.090909 using ๐†2(0โˆถ๐‘€)=0.909090 from Table 2. Theorem 4.1 (i) now asserts that ๎ฬ‚๐›ฝ๐บ๐‘‰(๐œท)=๐บ๐‘‰(0๎๐œท)๐บ๐‘‰(๐‘€ฬ‚๐›ฝ)๐œธ(0โˆถ๎๐œท๐‘€), as verified numerically through 74.4216=10.00(81.8638)(0.090909). In a similar manner, ๎๐œท๐‘€ partitions into {๎๐œท๐ฟ๎๐œท,[๐‘„,๎๐œท๐ผ]}, where ๎๐œท๐บ๐‘‰(๐ฟ)=4.6296 and ๎๐œท๐บ๐‘‰([๐‘„,๎๐œท๐ผ])=81.8638 from Table 4. The squared canonical correlations between {๎๐œท๐ฟ๎๐œท,[๐‘„,๎๐œท๐ผ]} are ๐†2(๐ฟโˆถ๐‘„๐ผ)=[0.4000,0.4000,0.4000]๎…ž from Table 2, so that ๐œธ(๐ฟโˆถ๐‘„๐ผ)=0.21600 as in Table 3. Theorem 4.1 (i) again recovers ๎๐œท๐บ๐‘‰(๐‘€) as 81.8638=4.6926(81.8638)(0.21600) since ๎๐œท๐บ(๐ฟ) and ๐œธ(๐ฟโˆถ๐‘„๐ผ) are reciprocals in this instance. Moreover, ๎๐œท๐บ๐‘‰(๐‘„,๎๐œท๐ผ๎๐œท)=๐บ๐‘‰(๐‘„๎๐œท)๐บ๐‘‰(๐ผ)๐œธ(๐‘„โˆถ๐ผ) translates into 81.8638=1.1317(72.3379)(1.0), where ๐œธ(๐‘„โˆถ๐ผ)=1.0 since elements of {๎๐œท๐‘„,๎๐œท๐ผ} are mutually uncorrelated from Table 2. In summary, the value ๎๐บ(๐œท) for the SCD admits the factorization of (4.6), on identifying {๎๐œฝ1,๎๐œฝ2,๎๐œฝ3,๎๐œฝ4} with {ฬ‚๐›ฝ0,๎๐œท๐ฟ,๎๐œท๐‘„,๎๐œท๐ผ}, respectively, given numerically from Tables 1 and 3 as 1.53876=(10.00)1/10(1.66667)3/10(1.04210)3/10(4.16667)3/10[](0.090909)(0.21600)(1.0)1/10.(5.2)

Table 4: Generalized variances and Vector Alienation Coefficients between designated subsets for Hartleyโ€™s [39] ๐‘†๐ถ๐ท in ๐‘˜=3 regressors.

Corresponding factorizations proceed similarly for other designs. Details are left to the reader, but values for [๐œธ(0โˆถ๐ฟ๐‘„๐ผ)๐œธ(๐ฟโˆถ๐‘„๐ผ)๐œธ(๐‘„โˆถ๐ผ)]1/10, the rightmost factor of (5.2), are supplied for each design as the final row of Table 3. Although the tables, together with Theorem 4.1, support other factorizations, the one featured here seems most natural in terms of the parameters {๐›ฝ0,๐œท๐ฟ,๐œท๐‘„,๐œท๐ผ}, together with their central roles in identifying noteworthy treatment effects in second-order models.

5.4. Masking

The ๐ท-efficiency index of the SCD, at ๎๐บ๐‘‰(๐œท)=74.4216, is larger but roughly comparable to that of H310 at ๎๐บ๐‘‰(๐œท)=57.3418. What cannot be anticipated from these facts alone, however, is that the (3ร—3) determinant ๎๐œท๐บ๐‘‰(๐ผ)=72.3379 for the SCD is comparable to its (10ร—10) determinant ๎๐บ๐‘‰(๐œท)=74.4216, despite their disparate dimensions. Adjusting for dimensions gives ๎๐บ(๐œท)=(74.4216)1/10=1.53876 and ๎๐œท๐บ(๐ผ)=(72.3379)1/3=4.16667 for the SCD. This illustrates the masking of a remarkably inefficient estimator for ๐œท๐ผ, despite the value ๎๐บ(๐œท)=1.53876 in estimating all parameters. This masking stems from the nonorthogonality of subset estimators as reflected in their canonical correlations and Vector Alienation Coefficients. In contrast are the corresponding commensurate values for the H310 design, namely, ๎๐บ(๐œท)=(57.3418)1/10=1.49916 and ๎๐œท๐บ(๐ผ)=(4.1768)1/3=1.61045. It may be noted that the condition number ๐‘1(๐—๎…ž0๐—0) is 21.59 for H310, with the somewhat larger value 54.01 for the SCD.

We next examine the ๐ท๐‘ -inefficiencies of ND and HD for ๐œท๐‘„ as noted earlier, with ๎๐œท๐บ(๐‘„) taking values 7.80031 and 6.61313, respectively. Our reference for masking is ๎๐œท๐บ(๐ฟ,๎๐œท๐‘„). These values are not listed in Table 1, but may be recovered from Tables 2 and 3 as follows. Specifically, for ND we have ๎‚ƒ๎‚€๎๐œท๐บ๐‘‰๐ฟ๎‚๎‚€๎๐œท๐บ๐‘‰๐‘„๎‚๎‚„๐›พ(๐ฟโˆถ๐)1/6=๎‚ƒ๎‚€๎๐œท๐บ๐‘‰๐ฟ,๎๐œท๐‘„๎‚๎‚„1/6๎‚€๎๐œท=๐บ๐ฟ,๎๐œท๐‘„๎‚,(1.25000)3/6(7.80031)3/6(0.72428)1/6=2.95912,(5.3) where neither ๎๐œท๐บ(๐ฟ)=1.25000 nor ๎๐œท๐บ(๐ฟ,๎๐œท๐‘„)=2.95912 appears excessive. In consequence, that ๎๐œท๐บ(๐‘„)=7.80031 is excessive would be masked on examining ๎๐œท๐บ(๐ฟ) and ๎๐œท๐บ(๐ฟ,๎๐œท๐‘„) only. Parallel steps for HD give the factorization (1.91189)3/6(6.61313)3/6(0.78514)1/6 = 3.41528, with similar conclusions in regard to masking.

5.5. Collinearity with the Constant

Advocates for these and other small designs have focused on ๐ท,๐ท๐‘ , and other efficiency criteria, as well as the parsimony of small designs and their advantage in industrial experiments. To the knowledge of this observer, none has considered prospects for ill-conditioning and its consequences, despite the fact that columns of ๐— are necessarily inter-linked as a consequence of second-order from first-order effects. Nonetheless, from Section 3.1 and Corollary 3.2, we may compute angles between the constant vector and the span of the regressors using duality together with the information at hand. This may prove to be critical in view of the admonition [29] that โ€œcollinearity with the intercept can quite generally corrupt the estimates of all parameters in the model.โ€ As noted in Remark 3.4, rules-of-thumb for problematic VIFs include those exceeding 10 or 4 or, in angular measure, ๐œ™โˆ—โˆ—<18.435deg and ๐œ™โˆ—<30.00deg. From the last row of Table 2, the angles ๐œ™(๐Ÿ๐‘›โˆถ๐—) have been computed for each of the eight designs, as listed in the final column of Table 1. For example, โˆšarccos(0.81990)=25.1116deg for H310. It is seen that all designs are flagged as potentially problematic using rules-of-thumb as cited. This adds yet another layer of concerns, heretofore unrecognized, in seeking further to implement these designs already in wide usage.

6. Conclusions

Duality of (i) Hotellingโ€™s [30] canonical correlations {๐œŒ1,โ€ฆ,๐œŒ๐‘Ÿ} between the ๐‘‚๐ฟ๐‘† estimators {๎๐œท1,๎๐œท2} and (ii) the design linkage parameters {๐›ฟ1,โ€ฆ,๐›ฟ๐‘Ÿ} between {๐—1,๐—2} is established at the outset. Stewartโ€™s [33] collinearity indices are then extended to encompass angles {๐œ™0,๐œ™1,โ€ฆ,๐œ™๐‘˜} between each column of ๐—0=[๐Ÿ๐‘›,๐—1,โ€ฆ,๐—๐‘˜] and remaining columns. In particular, ๐œ™0 quantifies numerically the collinearity of regressors with the intercept, of concern in the prospective corruption of all estimates due to ill-conditioning.

Matrix identities factorize a determinant in terms of principal subdeterminants and the Vector Alienation Coefficients of [30] between {๎๐œท1,๎๐œท2}. By duality, the latter also are Alienation Coefficients between {๐—1,๐—2}. These identities in turn are applied in the study of ๐ท๐‘ -efficiencies for the parameters {๐›ฝ0,๐œท๐ฟ,๐œท๐‘„,๐œท๐ผ} in eight small second-order designs from the literature. Studies on ๐ท๐‘ - and ๐ท-efficiencies, as cited in our opening paragraph, confirm that designs are seldom efficient for both. Our determinant identities support a rational explanation. In particular, these identities unmask the propensity for ๐ท๐‘ -inefficient subset estimators to be masked through near collinearities in overall ๐ท-efficient designs.

Finally, the evidence suggests that all eight designs are vulnerable, to varying degrees, to the corruption of all estimates due to ill-conditioning. In short, we have exposed quantitatively the structural origins of masking through Hotellingโ€™s [30] canonical correlations, and their equivalent design linkage parameters. This analysis in turn proceeds from the design matrix itself rather than empirical estimates, so that any design can be evaluated beforehand with regard to masking and possible subset inefficiencies, rather than retrospectively after having committed to a given design in a particular experiment.


  1. J. C. Kiefer, โ€œOptimum experimental designs,โ€ Journal of the Royal Statistical Society. Series B, vol. 21, pp. 272โ€“319, 1959. View at Google Scholar ยท View at Zentralblatt MATH
  2. J. C. Kiefer, โ€œOptimum designs in regression problems. II,โ€ Annals of Mathematical Statistics, vol. 32, pp. 298โ€“325, 1961. View at Publisher ยท View at Google Scholar ยท View at Zentralblatt MATH
  3. J. C. Kiefer and J. Wolfowitz, โ€œOptimum designs in regression problems,โ€ Annals of Mathematical Statistics, vol. 30, pp. 271โ€“294, 1959. View at Publisher ยท View at Google Scholar ยท View at Zentralblatt MATH
  4. A. Wald, โ€œOn the efficient design of statistical investigations,โ€ Annals of Mathematical Statistics, vol. 14, pp. 134โ€“140, 1943. View at Publisher ยท View at Google Scholar ยท View at Zentralblatt MATH
  5. M. J. Box and N. R. Draper, โ€œFactorial designs, the |XX| criterion, and some related matters,โ€ Technometrics, vol. 13, pp. 731โ€“742, 1971. View at Google Scholar
  6. J. M. Lucas, โ€œWhich response surface design is best,โ€ Technometrics, vol. 18, pp. 411โ€“417, 1976. View at Google Scholar
  7. R. H. Myers, A. I. Khuri, and W. H. Carter, Jr., โ€œResponse surface methodology: 1966–1988,โ€ Technometrics, vol. 31, no. 2, pp. 137โ€“157, 1989. View at Google Scholar ยท View at Zentralblatt MATH
  8. R. Sibson, โ€œDA-optimality and duality,โ€ in Progress in Statistics (European Meeting Statisticians, Budapest, 1972), Vol. II, J. Gani, K. Sarkadi, and I. Vincze, Eds., North-Holland, Amsterdam, The Netherlands, 1974. View at Google Scholar ยท View at Zentralblatt MATH
  9. S. D. Silvey, Optimal Design, Chapman & Hall, London, UK, 1980.
  10. R. C. St. John and N. R. Draper, โ€œD-optimality for regression designs: a review,โ€ Technometrics, vol. 17, pp. 15โ€“23, 1975. View at Google Scholar ยท View at Zentralblatt MATH
  11. H. Dette, โ€œA generalization of D- and D1-optimal designs in polynomial regression,โ€ The Annals of Statistics, vol. 18, no. 4, pp. 1784โ€“1804, 1990. View at Publisher ยท View at Google Scholar
  12. H. Dette and T. Franke, โ€œRobust designs for polynomial regression by maximizing a minimum of D- and D1-efficiencies,โ€ The Annals of Statistics, vol. 29, no. 4, pp. 1024โ€“1049, 2001. View at Publisher ยท View at Google Scholar ยท View at MathSciNet
  13. S. Huda, โ€œOn some Ds-optimal designs in spherical regions,โ€ Communications in Statistics, vol. 20, no. 9, pp. 2965โ€“2985, 1991. View at Publisher ยท View at Google Scholar ยท View at MathSciNet
  14. S. Huda and A. A. Al-Shiha, โ€œOn D-optimal designs for estimating slope,โ€ Sankhyá. Series A, vol. 61, no. 3, pp. 488โ€“495, 1999. View at Google Scholar
  15. C. M.-S. Lee, โ€œD-optimal designs for polynomial regression when lower degree parameters are more important,โ€ Utilitas Mathematica, vol. 34, pp. 53โ€“63, 1988. View at Google Scholar ยท View at Zentralblatt MATH
  16. Y. B. Lim and W. J. Studden, โ€œEfficient Ds-optimal designs for multivariate polynomial regression on the q-cube,โ€ The Annals of Statistics, vol. 16, no. 3, pp. 1225โ€“1240, 1988. View at Publisher ยท View at Google Scholar
  17. W. J. Studden, โ€œDs-optimal designs for polynomial regression using continued fractions,โ€ The Annals of Statistics, vol. 8, no. 5, pp. 1132โ€“1141, 1980. View at Publisher ยท View at Google Scholar ยท View at Zentralblatt MATH
  18. D. M. Wardrop and R. H. Myers, โ€œSome response surface designs for finding optimal conditions,โ€ Journal of Statistical Planning and Inference, vol. 25, no. 1, pp. 7โ€“28, 1990. View at Publisher ยท View at Google Scholar ยท View at Zentralblatt MATH ยท View at MathSciNet
  19. H. P. Wynn, โ€œThe sequential generation of D-optimum experimental designs,โ€ Annals of Mathematical Statistics, vol. 41, pp. 1655โ€“1664, 1970. View at Publisher ยท View at Google Scholar
  20. H. P. Wynn, โ€œResults in the theory and construction of D-optimum experimental designs,โ€ Journal of the Royal Statistical Society. Series B. Methodological, vol. 34, pp. 133โ€“147, 1972. View at Google Scholar
  21. H. Dette, โ€œOn a mixture of the D- and D1-optimality criterion in polynomial regression,โ€ Journal of Statistical Planning and Inference, vol. 35, no. 2, pp. 233โ€“249, 1993. View at Publisher ยท View at Google Scholar ยท View at MathSciNet
  22. S. Huda, โ€œOn Ds-efficiency of some D-optimal designs,โ€ Calcutta Statistical Association Bulletin, vol. 34, no. 133-134, pp. 119โ€“122, 1985. View at Google Scholar
  23. S. Huda and I. H. Khan, โ€œOn efficiency of some D-optimal designs in subset estimation,โ€ Journal of the Indian Society of Agricultural Statistics, vol. 47, no. 2, pp. 151โ€“164, 1995. View at Google Scholar
  24. P. F. Schwartz, C. Gennings, L. K. Teuschler, and M. W. Farris, โ€œOptimizing the precision of toxicity threshold estimation using a two-stage experimental design,โ€ Journal of Agricultural, Biological, and Environmental Statistics, vol. 6, pp. 409โ€“428, 2001. View at Google Scholar
  25. S. L. Meadows-Shropshire, C. Gennings, W. H. Carter, and J. E. Simmons, โ€œAnalysis of mixtures of drugs/chemicals along a fixed-ratio ray without single-chemical data to support an additivity model,โ€ Journal of Agricultural, Biological, and Environmental Statistics, vol. 9, pp. 500โ€“514, 2004. View at Google Scholar
  26. D. A. Belsley, โ€œCentering, the constant, first-differencing, and assessing conditioning,โ€ in Model Reliability, D. A. Belsley and E. Kuh, Eds., pp. 117โ€“153, MIT Press, Cambridge, Mass, USA, 1986. View at Google Scholar
  27. E. R. Mansfield and B. P. Helms, โ€œDetecting multicollinearity,โ€ American Statistician, vol. 36, pp. 158โ€“160, 1982. View at Google Scholar
  28. S. D. Simon and J. P. Lesage, โ€œThe impact of collinearity involving the intercept term on the numerical accuracy of regression,โ€ Computer Science in Economics and Management, vol. 1, no. 2, pp. 137โ€“152, 1988. View at Publisher ยท View at Google Scholar
  29. D. A. Belsley, โ€œComment: well-conditioned collinearity indices,โ€ Statistical Science, vol. 2, no. 1, pp. 86โ€“91, 1987. View at Google Scholar ยท View at Zentralblatt MATH
  30. H. Hotelling, โ€œRelations between two sets of variates,โ€ Biometrika, vol. 28, pp. 321โ€“377, 1936. View at Google Scholar
  31. Å. Björck and G. H. Golub, โ€œNumerical methods for computing angles between linear subspaces,โ€ Mathematics of Computation, vol. 27, pp. 579โ€“594, 1973. View at Google Scholar ยท View at Zentralblatt MATH
  32. M. L. Eaton, Multivariate Statistics: A Vector Space Approach, John Wiley & Sons, New York, NY, USA, 1983.
  33. G. W. Stewart, โ€œCollinearity and least squares regression,โ€ Statistical Science, vol. 2, no. 1, pp. 68โ€“100, 1987. View at Google Scholar ยท View at Zentralblatt MATH
  34. R. H. Myers, Classical and Modern Regression with Applications, PWS–KENT, Boston, Mass, USA, 2nd edition, 1990.
  35. R. M. O'Brien, โ€œA caution regarding rules of thumb for variance inflation factors,โ€ Quality and Quantity, vol. 41, no. 5, pp. 673โ€“690, 2007. View at Publisher ยท View at Google Scholar
  36. S. S. Wilks, Mathematical Statistics, John Wiley & Sons, New York, NY, USA, 1962.
  37. H. Cramér, Mathematical Methods of Statistics, Princeton Mathematical Series, vol. 9, Princeton University Press, Princeton, NJ, USA, 1946.
  38. K. G. Roquemore, โ€œHybrid designs for quadratic response surfaces,โ€ Technometrics, vol. 18, pp. 419โ€“423, 1979. View at Google Scholar
  39. H. O. Hartley, โ€œSmallest composite designs for quadratic response surfaces,โ€ Biometrics. Journal of the Biometric Society, vol. 15, pp. 611โ€“624, 1959. View at Google Scholar ยท View at Zentralblatt MATH
  40. G. E. P. Box and D. W. Behnken, โ€œSome new three level designs for the study of quantitative variables,โ€ Technometrics, vol. 2, pp. 455โ€“475, 1960. View at Google Scholar
  41. G. E. P. Box and K. B. Wilson, โ€œOn the experimental attainment of optimum conditions,โ€ Journal of the Royal Statistical Society. Series B, vol. 13, pp. 1โ€“45, 1951. View at Google Scholar ยท View at Zentralblatt MATH
  42. W. Notz, โ€œMinimal point second order designs,โ€ Journal of Statistical Planning Inference, vol. 6, no. 1, pp. 47โ€“58, 1982. View at Publisher ยท View at Google Scholar ยท View at Zentralblatt MATH
  43. A. T. Hoke, โ€œEconomical second-order designs based on irregular fractions of the 3n factorial,โ€ Technometrics, vol. 16, pp. 375โ€“384, 1974. View at Google Scholar ยท View at Zentralblatt MATH
  44. M. J. Box and N. R. Draper, โ€œOn minimum-point second-order designs,โ€ Technometrics, vol. 16, pp. 613โ€“616, 1974. View at Google Scholar ยท View at Zentralblatt MATH
  45. D. R. Jensen, โ€œEfficiencies of some small second-order designs,โ€ Statistics & Probability Letters, vol. 21, no. 4, pp. 255โ€“261, 1994. View at Publisher ยท View at Google Scholar ยท View at Zentralblatt MATH ยท View at MathSciNet