Determinant Efficiencies in Ill-Conditioned Models

Abstract

The canonical correlations between subsets of OLS estimators are identified with design linkage parameters between their regressors. Known collinearity indices are extended to encompass angles between each regressor vector and remaining vectors. One such angle quantifies the collinearity of regressors with the intercept, of concern in the corruption of all estimates due to ill-conditioning. Matrix identities factorize a determinant in terms of principal subdeterminants and the canonical Vector Alienation Coefficients between subset estimators—by duality, the Alienation Coefficients between subsets of regressors. These identities figure in the study of D and 𝐷_𝑠 as determinant efficiencies for estimators and their subsets, specifically, 𝐷_𝑠-efficiencies for the constant, linear, pure quadratic, and interactive coefficients in eight known small second-order designs. Studies on D- and 𝐷_𝑠-efficiencies confirm that designs are seldom efficient for both. Determinant identities demonstrate the propensity for 𝐷_𝑠-inefficient subsets to be masked through near collinearities in overall D-efficient designs.

1. Introduction

Given {𝐘=𝐗𝜷+𝝐} of full rank with homogeneous, uncorrelated errors, the OLS estimators 𝜷 are unbiased with second-moment matrix 𝑉(𝜷)=𝜎²(𝐗^𝐗)⁻¹. Such moment matrices pervade experimental design, to include determinants as gauges of 𝐷- and 𝐷_𝑠-efficiencies for estimators and their subsets. Early references trace to [1–4], and more recently to [5–10] and others. Finding 𝐷_𝑠-efficient designs for polynomial models is considered in [11–20], for example. Studies examining the 𝐷_𝑠-efficiencies of 𝐷-efficient designs confirm that designs are seldom efficient for both; see [13, 21–23]. From those beginnings, the study of 𝐷- and 𝐷_𝑠-efficiencies continues apace. To wit, a recent key-word search in the Current Index to Statistics shows in excess of 60 listings from 2006 to 2010, and more than 100 from 2001 to 2010. Moreover, these ideas bear fruit in a widening diversity of applications as evidenced in the following.

To fix ideas, let 𝐷 correspond to a polynomial 𝑃^𝑐 of degree 𝑐, namely, 𝑔(𝜇)=∑^𝑐𝑖=0𝛽_𝑖𝑡^𝑖. In toxicology studies, a two-stage experiment is proffered in [24], seeking 𝐷-efficiency in estimating 𝑘=𝑐+1 overall parameters at the first stage, then 𝐷₁-efficiency at the second stage in estimating a critical “threshold parameter,” using quasilikelihood in nonlinear models. Coupled with this is the 𝐷_𝑘−1-efficiency for the remaining 𝑘−1 parameters at the second stage. In related work [25], experiments with 𝑐 chemicals in combination are to be examined along fixed-ratio rays. When restricted to a specified ray, the fundamental hypothesis of noninteracting factors can be rejected when higher-order polynomial terms are required in the total dose-response model 𝑔(𝜇)=𝛽₀+𝛽₁𝑡+∑^𝑐𝑖=2𝛽_𝑖𝑡^𝑖 in the linear predictor 𝑡. Here 𝐷₂ refers to [𝛽₀,𝛽₁] and 𝐷_𝑐−1 to 𝐷_𝑠-efficiency in the critical estimation of [𝛽₂,…,𝛽_𝑐], which vanish under the conjectured additivity. Moreover, in [11, 12, 21] 𝐷 refers to 𝑃^𝑘+1, 𝐷_𝑘 to 𝑃^𝑘, and 𝐷₁ to 𝛽_𝑘+1, the highest-order term in 𝑃^𝑘+1, for example. In short, users often are properly concerned with both 𝐷- and 𝐷_𝑠-efficiencies, and connections between these basic criteria deserve further study, to be undertaken here.

Ill-conditioning, as near-collinearity among the columns of 𝐗, “causes crucial elements of 𝐗^𝐗 to be large and unstable,” “creating inflated variances,” and estimates that are “very sensitive to small changes in 𝐗,” having “degraded numerical accuracy;” see [26–28], for example. Diagnostics include the condition number 𝑐₁(𝐗^𝐗), the ratio of largest to smallest eigenvalues; and the Variance Inflation Factors {VIF(𝛽̂_𝑗)=𝑣_𝑗𝑗𝑤_𝑗𝑗;1≤𝑗≤𝑝} with 𝐖=𝐗^𝐗 and 𝐕=(𝐗^𝐗)⁻¹, that is, ratios of actual (𝑣_𝑗𝑗) to “ideal” (1/𝑤_𝑗𝑗) variances had the columns of 𝐗 been orthogonal. In models with intercept, “collinearity with the intercept can quite generally corrupt the estimates of all parameters in the model whether or not the intercept is itself of interest and whether or not the data have been (mean) centered,” as noted in [29].

To the foregoing list of ills from ill-conditioning, we add that not only are designs seldom efficient for both, but 𝐷_𝑠-inefficient estimators may be masked in overall 𝐷-efficient designs, and conversely. This masking may be quantified in terms of structural dependencies, specifically, through determinant identities linking 𝐷- and 𝐷_𝑠-efficiencies to various gages of nonorthogonality of the data. The latter include nonvanishing inner products between columns of regressors, Hotelling’s [30] canonical correlations among OLS solutions, and VIFs. An outline follows.

Section 2 contains supporting material. Details surrounding collinearity diagnostics are topics in Section 3, to include duality of angles between subspaces of the design and parameter spaces, and their connections to VIFs. Section 4 develops basic determinant identities and inequalities of independent interest. Section 5 revisits eight small second-order designs with regard to 𝐷_𝑠-efficiencies in estimating the constant, linear, pure quadratic, and interactive coefficients, to include the masking of inefficient estimators. Though in wide usage, with no apparent accounting for collinearity, these designs are seen to exhibit varying degrees of collinearity of regressors with the constant. Since computations proceed from the design matrix itself, an advantage is that prospective designs can be evaluated beforehand in regard to issues studied here, before committing to an actual experiment. Section 6 concludes with a brief summary.

2. Preliminaries

2.1. Notation

Spaces of note include ℝ^𝑘 as Euclidean 𝑘-space; ℝ^𝑘+ as its positive orthant; 𝔽_𝑛×𝑘 as the real matrices of order (𝑛×𝑘); 𝕊_𝑘 as the (𝑘×𝑘) real symmetric matrices; and 𝕊^+𝑘 as their positive definite varieties. The transpose, inverse, trace, and determinant of 𝐀∈𝕊^+𝑘 are 𝐀^, 𝐀⁻¹, tr(𝐀), and |𝐀|; and 𝐀^1/2 is its spectral square root. Special arrays include the unit vector 𝟏^𝑛=[1,1,…,1]∈ℝ^𝑛, the identity 𝐈_𝑛 of order (𝑛×𝑛), the block-diagonal matrix Diag(𝐀₁,𝐀₂)∈𝕊_𝑘, the idempotent form 𝐁_𝑛=(𝐈_𝑛−𝑛⁻¹𝟏_𝑛𝟏^𝑛), and 𝒪(𝑘) as the real orthogonal group of (𝑘×𝑘) matrices. For 𝐗(𝑛×𝑝) of rank 𝑝≤𝑛, designate a pseudoinverse as 𝐗^†, its ordered singular values as 𝜎(𝐗)={𝜉₁≥𝜉₂≥⋯≥𝜉_𝑝>0}, and by 𝒮_𝑝(𝐗)⊂ℝ^𝑛, the linear span of columns of 𝐗. Its condition number is 𝑐₂(𝐗)=𝜉₁/𝜉_𝑝, specifically, 𝑐₂(𝐗)=[𝑐₁(𝐗^𝐗)]^1/2.

The mean, dispersion matrix, and generalized variance for a random 𝐔∈ℝ^𝑘 are designated as 𝐸(𝐔)=𝝁∈ℝ^𝑘, 𝑉(𝐔)=𝚺∈𝕊^+𝑘, and 𝐺𝑉(𝐔)=|𝚺|, respectively. To account for dimension, consider 𝐺(𝐔)=[𝐺𝑉(𝐔)]^1/𝑘=|𝚺|^1/𝑘 as a function homogeneous of unit degree. The class M₀∶{𝐘=𝛽₀𝟏_𝑛+𝐗𝜷+𝝐}, comprising models with intercept and dispersion 𝑉(𝝐)=𝜎²𝐈_𝑛, is our principal focus. Unless stated otherwise, we take 𝜎²=1.0, since variance ratios are scale-invariant. A distinction is drawn between centered and uncentered VIFs, namely, VIF_𝑐s and VIF_𝑢s, the former from columns of 𝐗 centered to their means. The latter, designated as {VIF_𝑢(𝛽̂_𝑗);𝑗=0,1,…,𝑘}, are diagonal elements of (𝐗^0𝐗₀)⁻¹ divided by reciprocals of diagonals of 𝐗^0𝐗₀ itself. These are of subsequent interest. Special distributions on ℝ¹⁺ include the Snedecor-Fisher distribution 𝐹(⋅;𝜈₁,𝜈₂,𝜆) having (𝜈₁,𝜈₂) degrees of freedom and noncentrality 𝜆.

3. Collinearity Diagnostics

Ill-conditioned models {𝐘=𝐗𝜷+𝝐}, burdened with difficulties as cited, trace to nonorthogonality among columns of 𝐗. To examine aspects of near collinearity, we first establish duality between design linkage parameters among columns of 𝐗, and collinearity among the OLS solutions as quantified by Hotelling's [30] canonical correlations.

3.1. Duality Results

Partition a generic 𝐗∈𝔽_𝑛×𝑝 as 𝐗=[𝐗₁,𝐗₂] with {𝐗,𝐗₁,𝐗₂} of orders {(𝑛×𝑝),(𝑛×𝑟),(𝑛×𝑠)}, respectively, having ranks {𝑝,𝑟,𝑠} such that 𝑟≤𝑠 and 𝑟+𝑠=𝑝<𝑛. Accordingly, write {𝐘=𝐗₁𝜷₁+𝐗₂𝜷₂+𝝐}, taking 𝜷^=[𝜷^1,𝜷^2], and denoting by 𝒮_𝑝(𝐗₁) and 𝒮_𝑝(𝐗₂), the subspaces of ℝ^𝑛 spanned by columns of 𝐗₁ and 𝐗₂. We seek a canonical form preserving these subspaces and linkage between (𝐗₁,𝐗₂), a geometric concept independent of bases for representing 𝒮_𝑝(𝐗₁) and 𝒮_𝑝(𝐗₂). Accordingly, let 𝐆₁=(𝐗^1𝐗₁)^−1/2𝐏 and 𝐆₂=(𝐗^2𝐗₂)^−1/2𝐐, with 𝐏∈𝒪(𝑟) and 𝐐∈𝒪(𝑠) to be stipulated. The original model becomes {𝐘=𝐙₁𝜶₁+𝐙₂𝜶₂+𝝐} with 𝐙=[𝐙₁,𝐙₂] and 𝜶^=[𝜶^1,𝜶^2], such that 𝐙₁=𝐗₁𝐆₁, 𝐙₂=𝐗₂𝐆₂, 𝜶₁=𝐆¹⁻¹𝜷₁, and 𝜶₂=𝐆²⁻¹𝜷₂.

Following [31], cosines of angles between 𝒮_𝑝(𝐗₁) and 𝒮_𝑝(𝐗₂) are found as singular values generated by (𝐗₁,𝐗₂), to be designated as design linkage parameters {𝛿₁,…,𝛿_𝑟}. To these ends, observe that 𝐗^𝐗 in partitioned form transitions into 𝐙^𝐙 through 𝐗^𝐗=⎡⎢⎢⎣𝐗^1𝐗₁𝐗^1𝐗₂𝐗^2𝐗₁𝐗^2𝐗₂⎤⎥⎥⎦⟶⎡⎢⎢⎣𝐆^1𝐗^1𝐗₁𝐆₁𝐆^1𝐗^1𝐗₂𝐆₂𝐆^2𝐗^2𝐗₁𝐆₁𝐆^2𝐗^2𝐗₂𝐆₂⎤⎥⎥⎦=⎡⎢⎢⎣𝐐𝐈_𝑟𝐏^𝐑𝐐_𝐑^𝐏𝐈_𝑠⎤⎥⎥⎦=⎡⎢⎢⎣𝐃𝐈_𝑟𝐃𝐈_𝑠⎤⎥⎥⎦=𝐙_𝐙.(3.1) Here 𝐑=(𝐗^1𝐗₁)^−1/2𝐗^1𝐗₂(𝐗^2𝐗₂)^−1/2; its singular decomposition is 𝐑 = 𝐏𝐃𝐐^, where 𝐃=[𝐃_𝛿,𝟎]; and elements of 𝐃_𝛿=Diag(𝛿₁,…,𝛿_𝑟) comprise the singular values of 𝐑. In particular, {𝜙_𝑗=arccos(𝛿_𝑗);1≤𝑗≤𝑟} defines the design linkage angles between 𝒮_𝑝(𝐗₁) and 𝒮_𝑝(𝐗₂) as subspaces of ℝ^𝑛.

To continue, partition 𝑉(𝜷)=𝚺=[𝚺_𝑖𝑗] conformably with 𝜷^=[𝜷^1,𝜷^2],  𝜷₁∈ℝ^𝑟,𝜷₂∈ℝ^𝑠; designate their inner product space as (ℝ^𝑟⊕ℝ^𝑠,(⋅,⋅)_𝚺), where ℝ^𝑟⊕ℝ^𝑠 is the direct sum and (⋅,⋅)_𝚺 their inner product, as in Eaton [32, page 409]. Denote by {𝜌₁,…,𝜌_𝑟} Hotelling’s [30] canonical correlations. Then by Proposition  10.2 of [32], {𝜌₁,…,𝜌_𝑟} are cosines of angles between (ℝ^𝑟,ℝ^𝑠) as subspaces of (ℝ^𝑟⊕ℝ^𝑠,(⋅,⋅)_𝚺). In keeping with earlier usage, identify {𝒮_𝑝(𝜷₁),𝒮_𝑝(𝜷₂)} with {ℝ^𝑟,ℝ^𝑠}. As Hotelling's canonical correlations are invariant under affine transformations {𝜷₁→𝐀𝜷₁+𝐜₁,𝜷₂→𝐁𝜷₂+𝐜₂}, parameters may be redefined linearly, preserving subspaces, thus leaving the canonical correlations invariant. Retracing steps leading to the canonical design model embodied in (3.1), but now to preserve {𝒮_𝑝(𝜷₁),𝒮_𝑝(𝜷₂)}, it thus suffices to begin with the canonical model {𝐘=𝐙₁𝜶₁+𝐙₂𝜶₂+𝝐}, where 𝑉(𝜶)=𝜎²(𝐙^𝐙)⁻¹ with 𝐙^𝐙 as the rightmost matrix of (3.1).

We next establish connections between the design linkage parameters 𝐃_𝛿 from (3.1), and the corresponding canonical correlations 𝐃_𝜌=Diag(𝜌₁,…,𝜌_𝑟), as derived eventually from 𝚺=(𝐗^𝐗)⁻¹. A critical duality result is encoded in the following.

Theorem 3.1. Consider the design linkage parameters  𝐃_𝛿 between {𝒮_𝑝(𝐗₁),𝒮_𝑝(𝐗₂)} as subspaces of ℝ^𝑛 and Hotelling’s [30] canonical correlations  𝐃_𝜌 between {𝒮_𝑝(𝜷₁),𝒮_𝑝(𝜷₂)} as subspaces of (ℝ^𝑟⊕ℝ^𝑠,(⋅,⋅)_𝚺). Then 𝐃_𝛿 and 𝐃_𝜌 coincide.

Proof. In view of invariance of {𝜌₁,…,𝜌_𝑟} under nonsingular linear transformations of 𝜷₁∈ℝ^𝑟 and of 𝜷₂∈ℝ^𝑠, canonical correlations between (𝜷₁,𝜷₂) proceed as in expression (3.1), but beginning instead on the left with 𝑉(𝜶) = (𝐙^𝐙)⁻¹  in lieu of 𝐗^𝐗. Specifically, with 𝐃 = [𝐃_𝛿,𝟎], and using rules for block-partitioned inverses, we have 𝐙^𝐙⁻¹=⎡⎢⎢⎣𝐈^𝑟𝐃𝐃_𝛿,𝟎_𝛿,𝟎^𝐈_𝑠⎤⎥⎥⎦⁻¹=⎡⎢⎢⎣𝐈^𝑟−𝐃^2𝛿⁻¹−𝐃𝐈_𝑠−𝐃₀⁻¹−𝐃^𝐈_𝑟−𝐃^2𝛿⁻¹𝐈_𝑠−𝐃₀⁻¹⎤⎥⎥⎦⟶⎡⎢⎢⎣𝐃𝐈_𝑟𝐃_𝛿,𝟎_𝛿,𝟎^𝐈_𝑠⎤⎥⎥⎦,(3.2) where equality at the first step follows using 𝐃𝐃^=𝐃^2𝛿 and 𝐃^𝐃=Diag(𝐃^2𝛿,𝟎)=𝐃₀. The succeeding step utilizes the factors (𝐈_𝑟−𝐃^2𝛿)^1/2 and (𝐈_𝑠−𝐃₀)^1/2, taking the principal diagonal blocks of (𝐙^𝐙)⁻¹ into (𝐈_𝑟,𝐈_𝑠) as in the rightmost matrix of (3.2), and its off-diagonal block from 𝐈_𝑟−𝐃^2𝛿^1/2𝐃_𝛿,𝟎𝐈_𝑠−𝐃₀⁻¹𝐈_𝑠−𝐃₀^1/2=𝐃_𝛿,𝟎,(3.3) since diagonal matrices commute. But the off-diagonal block is precisely [𝐃_𝜌,𝟎], the canonical correlations between (𝜷₁,𝜷₂), to complete our proof.

For subsequent reference, designate 𝜹(𝐗₁∶𝐗₂)=(𝛿₁,…,𝛿_𝑟) and 𝝆(𝜷₁∶𝜷₂)=(𝜌₁,…,𝜌_𝑟). Moreover, the foregoing analysis applies for models 𝐗₀=[𝟏_𝑛,𝐗] in M₀, where 𝑟=1 and 𝑠=𝑘 as partitioned. In short, we have the following equivalences.

Corollary 3.2. (i) Consider the design linkage parameters {cos(𝜙_𝑗)=𝛿_𝑗;1≤𝑗≤𝑟}, gaging collinearity between {𝒮_𝑝(𝐗₁),𝒮_𝑝(𝐗₂)} as subspaces of ℝ^𝑛 and the canonical correlations {cos(𝜙_𝑗)=𝜌_𝑗;1≤𝑗≤𝑟}, between {𝒮_𝑝(𝜷₁),𝒮_𝑝(𝜷₂)} as subspaces of (ℝ^𝑟⊕ℝ^𝑠,(⋅,⋅)_𝚺). Then angles between these pairs of subspaces correspond one-to-one, that is, {𝜙_𝑗=arccos(𝛿_𝑗)=arccos(𝜌_𝑗);1≤𝑗≤𝑟}.
(ii) For models 𝐗₀=[𝟏_𝑛,𝐗] in M₀, the element 𝜹(𝟏_𝑛∶𝐗)=𝛿₁ generates the angle cos(𝜙₁)=𝛿₁ between the regressor vectors and the constant vector. Equivalently, this is given by cos(𝜙₁)=𝜌₁=𝝆(𝛽̂₀∶𝜷) from duality.

3.2. Collinearity Indices

Stewart [33] reexamined numerical aspects of ill-conditioning, to the following effects for 𝐗₀=[𝟏_𝑛,𝐗]. Taking 𝐗^†0 = (𝐗^0𝐗₀)⁻¹𝐗^0 as the pseudoinverse of note, and letting 𝐱^†𝑗 be its 𝑗th row, each collinearity index in the collection 𝜅^𝑗=‖‖𝐱_𝑗‖‖⋅‖‖𝐱^†𝑗‖‖;𝑗=0,1,…,𝑘(3.4) is constructed to be scale-invariant. Clearly ‖𝐱^†𝑗‖² is found along the principal diagonal of [(𝐗^†0)(𝐗^†0)^] = (𝐗^0𝐗₀)⁻¹. In addition, the conventional VIF_𝑢s are squares of the collinearity indices, that is, {VIF_𝑢(𝛽̂_𝑗)=𝜅^2𝑗;𝑗=0,1,…,𝑘}. In particular, since 𝐱₀=𝟏_𝑛 in 𝐗₀, we have 𝜅²⁰ = 𝑛‖𝐱^†0‖².

Transcending Stewart’s analysis, we connect his collinearity indices to angles between subspaces as follows. Choose a typical 𝐱_𝑗 in 𝐗₀; rearrange 𝐗₀ as [𝐱_𝑗,𝐗_[𝑗]] and similarly 𝜷^ as [𝛽_𝑗,𝜷^[𝑗]]; and seek elements of ⎧⎪⎨⎪⎩𝐐_𝑗𝐗^0𝐗₀⁻¹𝐐_𝑗=⎡⎢⎢⎣𝐱^𝑗𝐱_𝑗𝐱^𝑗𝐗_[𝑗]𝐗^[𝑗]𝐱_𝑗𝐗^[𝑗]𝐗_[𝑗]⎤⎥⎥⎦⁻¹;𝑗=0,1,…,𝑘⎫⎪⎬⎪⎭(3.5) as reordered by each permutation matrix 𝐐_𝑗. From the clockwise rule, the (1,1) element of each inverse is 𝐱^𝑗𝐈_𝑛−𝐏_𝑗𝐱_𝑗⁻¹=𝐱^𝑗𝐱_𝑗−𝐱^𝑗𝐏_𝑗𝐱_𝑗⁻¹=‖‖𝐱^†𝑗‖‖²;𝑗=0,1,…,𝑘,(3.6) where 𝐏_𝑗 = 𝐗_[𝑗][𝐗^[𝑗]𝐗_[𝑗]]⁻¹𝐗^[𝑗] is the projection operator onto the subspace 𝒮_𝑝(𝐗_[𝑗])⊂ℝ^𝑛. These relationships in turn enable us to connect {𝜅^2𝑗;𝑗=0,1,…,𝑘} to the geometry of ill-conditioning as follows.

Theorem 3.3. For models in M₀, let {VIF_𝑢(𝛽̂_𝑗)=𝜅^2𝑗;𝑗=0,1,…,𝑘} be conventional VIF_𝑢s in terms of Stewart’s collinearity indices. These in turn quantify collinearities between subspaces through angles (in deg) as follows.
(i)Angles between [𝐱_𝑗,𝐗_[𝑗]] are given by 𝜙_𝑗=arccos[(1−1/𝜅^2𝑗)^1/2], in succession for {𝑗=0,1,…,𝑘}.(ii)Equivalently, {𝜅^2𝑗=1/[1−𝜹²(𝐱_𝑗∶𝐗_[𝑗])]=1/[1−𝝆²(𝛽̂_𝑗∶𝜷_[𝑗])];𝑗=0,1,…,𝑘}.(iii)In particular, 𝜙₀=arccos[(1−1/𝜅²⁰)^1/2] quantifies the degree of collinearity between the regressor vectors and the constant vector.

Proof. From the geometry of the right triangle formed by (𝐱_𝑗,𝐏_𝑗𝐱_𝑗), the squared lengths satisfy ‖𝐱_𝑗‖²=‖𝐏_𝑗𝐱_𝑗‖²+𝑅𝑆_𝑗, where 𝑅𝑆_𝑗=‖𝐱_𝑗−𝐏_𝑗𝐱_𝑗‖² is the residual sum of squares from the projection. Accordingly, the principal angle between (𝐱_𝑗,𝐏_𝑗𝐱_𝑗) is given by cos𝜙_𝑗=𝐱^𝑗𝐏_𝑗𝐱_𝑗‖‖𝐱_𝑗‖‖⋅‖‖𝐏_𝑗𝐱_𝑗‖‖=‖‖𝐏^𝑗𝐱_𝑗‖‖‖‖𝐱_𝑗‖‖=1−‖‖𝑅𝐱_𝑗𝑆‖‖^𝑗2^1/2=1−1𝜅^2𝑗^1/2(3.7) for {𝑗=0,1,…,𝑘}, to give conclusion (i) and conclusion (ii) by duality. Conclusion (iii) follows on specializing (𝐱₀,𝐏₀𝐱₀) with 𝐱₀=𝟏_𝑛 and 𝐏₀=𝐗(𝐗^𝐗)⁻¹𝐗^, to complete our proof.

Remark 3.4. The foregoing developments specialize from Section 3.1 in that the partition [𝐱_𝑗,𝐗_[𝑗]] always has 𝑟=1 and 𝑠=𝑘, giving a single angle 𝜙_𝑗. Rules-of-thumb in common use for problematic VIFs include those exceeding 10, as in [34], or even 4 as in [35], for example. In angular measure, these correspond respectively to 𝜙_𝑗<18.435deg and 𝜙_𝑗<30.0deg.

3.3. Case Study 1

Consider the model M₀∶{𝑌_𝑖=𝛽₀+𝛽₁𝑋₁+𝛽₂𝑋₂+𝜖_𝑖}, the design 𝐗₀=[𝟏₅,𝐗₁,𝐗₂] of order (5×3), and 𝐗^0𝐗₀ and its inverse as in 𝐗𝐗^0=⎡⎢⎢⎢⎢⎣−111011.5011.5110100⎤⎥⎥⎥⎥⎦,𝐗^0𝐗₀=⎡⎢⎢⎢⎢⎣531230.5103_0⎤⎥⎥⎥⎥⎦,𝐗₀⁻¹=⎡⎢⎢⎢⎢⎣−−100..13.9213527055−110.1..73257505000−000.3..34173257505⎤⎥⎥⎥⎥⎦.(3.8) Note first that VIF_𝑢(𝛽̂₀)=𝜅²⁰=0.9375×5=4.6875. Next apply first principles to find 𝜹²𝟏₅∶𝐗₁,𝐗₂=(5)⁻¹31⎡⎢⎢⎣20.503⎤⎥⎥⎦⁻¹𝝆⎡⎢⎢⎣31⎤⎥⎥⎦=0.786666,₂𝛽̂₀∶𝛽̂₁,𝛽̂₂=(0.9375)⁻¹−1.1250−0.3125⎡⎢⎢⎣10..7357050000..34735705⎥⎦⎤⎥⁻¹⎡⎢⎢⎣−−10..13215205⎤⎥⎥⎦=0.786666,(3.9) both equal to 1−(1/𝜅²⁰) as in Theorem 3.3(ii). The remaining VIF_𝑢s are found directly as VIF_𝑢(𝛽̂₁)=1.7500×2.5=4.3750 and VIF_𝑢(𝛽̂₂)=0.4375×3=1.3125. Using duality and earlier findings, we further compute 𝜹²𝐗₁∶𝟏₅,𝐗₂=𝝆²𝛽̂₁∶𝛽̂₀,𝛽̂₂=1−1𝜹𝜅²¹=1−4.31750=0.771429,₂𝐗₂∶𝟏₅,𝐗₁=𝝆²𝛽̂₂∶𝛽̂₀,𝛽̂₁=1−1𝜅²²=1−1.31125=0.238095,(3.10) thereby preempting the need to undertake singular decompositions as required heretofore.

4. Determinant Identities

4.1. Background

The generalized variance, as a design criterion for {𝐘=𝐗𝜷+𝝐}, rests in part on the geometry of ellipsoids of the type 𝑅(𝜷)=𝜷∈ℝ^𝑘∶𝜷−𝜷^𝐗^𝐗𝜷−𝜷≤𝑐².(4.1) Choices for 𝑐² in common usage give first (i) a confidence region for 𝜷, whose normal-theory confidence coefficient is 1−𝛼 on taking 𝑐²=𝑆²𝑐^2𝛼, with 𝑆² as the residual mean square and 𝑐^2𝛼 the 100(1−𝛼) percentage point of 𝐹(⋅;𝑘,𝑛−𝑘); and otherwise admitting a lower Chebychev bound as in [36, page  92]. The alternative choice 𝑐²=𝑘+2 gives (ii) Cramér’s [37] ellipsoid of concentration for 𝜷, that is, the measure uniform over 𝑅(𝜷) having the same mean and dispersion matrix as 𝜷. The generalized variance 𝐺𝑉(𝜷)=|𝑉(𝜷)| is proportional to the squared volumes of these ellipsoids, smaller volumes reflecting tighter concentrations.

4.2. Factorizations

To continue, let some 𝐓(𝐘)=𝜽∈ℝ^𝑘 be random having 𝐸(𝜽)=𝜽 and 𝑉(𝜽)=𝚺∈𝕊^+𝑘; partition 𝜽^=[𝜽^1,𝜽^2] and 𝚺=[𝚺_𝑖𝑗] conformably, with 𝜽₁∈ℝ^𝑟 and 𝜽₂∈ℝ^𝑠 such that 𝑟≤𝑠 and 𝑟+𝑠=𝑘; and let 𝐺(𝜽)=|𝚺|^1/𝑘. The canonical correlations [30], as singular values of 𝚺^−1/211𝚺₁₂𝚺^−1/222, are now to be designated as 𝝆(1∶2) = [𝜌₁,…,𝜌_𝑟],  in lieu of 𝝆(𝜷₁∶𝜷₂), and to be ordered as {𝜌₁≥𝜌₂≥⋯≥𝜌_𝑟≥0}. Moreover, the quantity 𝜸(1∶2)=Π^𝑟𝑖=1(1−𝜌^2𝑖) is the Vector Alienation Coefficient of Hotelling [30]. The factorization |𝚺|=|𝚺₁₁||𝚺₂₂| for 𝚺=Diag(𝚺₁₁,𝚺₂₂) extends directly as an upper bound for any 𝚺=[𝚺_𝑖𝑗], with further ramifications as follows.

Theorem 4.1. Consider 𝜽^=[𝜽^1,𝜽^2]∈ℝ^𝑘 having 𝐸(𝜽^)=[𝜽^1,𝜽^2] and 𝑉(𝜽)=[𝚺_𝑖𝑗], such that 𝜽₁∈ℝ^𝑟 and 𝜽₂∈ℝ^𝑠 with 𝑟≤𝑠 and 𝑟+𝑠=𝑘.
(i)The determinant of 𝚺 = [𝚺_𝑖𝑗] admits the factorization ||𝚺||=||𝚺₁₁||||𝚺₂₂||𝜸(1∶2)(4.2) such that |𝚺|≤|𝚺₁₁||𝚺₂₂| and 𝜸(1∶2)=Π^𝑟𝑖=1(1−𝜌^2𝑖)≤1.(ii)If 𝚺=Diag(𝚺₁₁,𝚺₂₂), then 𝐺(𝜽) is the geometric mean 𝐺(𝜽)=[𝐺(𝜽₁)]^𝑟/𝑘[𝐺(𝜽₂)]^𝑠/𝑘 of the quantities 𝐺(𝜽₁) and 𝐺(𝜽₂).(iii)Generally, for any 𝚺, the quantity 𝐺(𝜽) becomes 𝐺𝜽=𝐺𝜽₁^𝑟/𝑘𝐺𝜽²^𝑠/𝑘[𝜸(1∶2)]^1/𝑘(4.3) in terms of {𝐺(𝜽₁),𝐺(𝜽₂),𝜸(1∶2)}.(iv)If 𝜽=[𝜽^1,𝜽^2,𝜽₃]^ and 𝚺=[𝚺_𝑖𝑗;1≤𝑖,𝑗≤3] are partitioned conformably, with 𝜽₁∈ℝ^𝑟,  𝜽₂∈ℝ^𝑠, and 𝜽₃∈ℝ^𝑡, such that 𝑟+𝑠+𝑡=𝑘, then 𝐺(𝜽) admits the factorization 𝐺𝜽=𝐺𝜽₁^𝑟/𝑘𝐺𝜽²^𝑠/𝑘𝐺𝜽³^𝑡/𝑘[𝜸(1∶23)𝜸(2∶3)]^1/𝑘,(4.4) with 𝜸(1∶23) and 𝜸(2∶3) as the Vector Alienation Coefficients between {𝜽₁,[𝜽₂,𝜽₃]} and between {𝜽₂,𝜽₃}, respectively.

Proof. As in Section 3.1 with 𝐑=𝚺^−1/211𝚺₁₂𝚺^−1/222=𝐏𝐃𝐐^ and 𝐃=[𝐃_𝜌,𝟎], we have ⎡⎢⎢⎣𝚺₁₁𝚺₁₂𝚺₂₁𝚺₂₂⎤⎥⎥⎦=⎡⎢⎢⎣𝐖𝟎₁𝐖𝟎₂⎤⎥⎥⎦⎡⎢⎢⎣𝐃𝐈_𝑟𝐃𝐈_𝑠⎤⎥⎥⎦⎡⎢⎢⎣𝐖𝟎^1𝐖𝟎_2⎤⎥⎥⎦(4.5) with 𝐖₁=𝚺^1/211𝐏 and 𝐖₂=𝚺^1/222𝐐. The middle factor on the right has determinant |𝐈_𝑟−𝐃𝐃^|=Π^𝑟𝑖=1(1−𝜌^2𝑖) from the clockwise rule, so that |𝚺|=|𝚺₁₁||𝚺₂₂|Π^𝑟𝑖=1(1−𝜌^2𝑖) to give conclusion (i). Conclusion (ii) follows directly from 𝐺(𝜽)=[𝐺𝑉(𝜽)]^1/𝑘, and conclusion (iii) on combining (i) and (ii). Conclusion (iv) now follows on applying (iii) twice, first on partitioning 𝜽 into {𝜽₁,[𝜽₂,𝜽₃]}, whose canonical correlations are 𝝆(1∶23), then [𝜽₂,𝜽₃] into {𝜽₂,𝜽₃} having canonical correlations 𝝆(2∶3), to complete our proof.

Remark 4.2. In short, Theorem 4.1 links determinants and principal subdeterminants precisely through angles between subspaces. Moreover, arguments leading to conclusion (iv) may be iterated recursively to achieve a hierarchical decomposition for four or more factors, as in the following with 𝑘=𝑟+𝑠+𝑡+𝑣, namely, 𝐺𝜽=𝐺𝜽₁^𝑟/𝑘𝐺𝜽²^𝑠/𝑘𝐺𝜽³^𝑡/𝑘𝐺𝜽⁴^𝑣/𝑘[𝜸(1∶234)𝜸(2∶34)𝜸(3∶4)]^1/𝑘.(4.6)

Remark 4.3. Hotelling’s [30] Vector Alienation Coefficient 𝜸(1∶2)=Π^𝑟𝑖=1(1−𝜌^2𝑖) is a composite index of linkage between {𝒮_𝑝(𝜷₁),𝒮_𝑝(𝜷₂)} as subspaces of (ℝ^𝑟⊕ℝ^𝑠,(⋅,⋅)_𝚺), decreasing in each {𝜌^2𝑖;1≤𝑖≤𝑟}. Equivalently, duality asserts that 𝜸(1∶2)=Π^𝑟𝑖=1(1−𝛿^2𝑖) is the identical composite index of linkage between {𝒮_𝑝(𝐗₁),𝒮_𝑝(𝐗₂)} as subspaces of ℝ^𝑛.

Theorem 4.1 anticipates that 𝐷_𝑠-inefficient subset estimators may be masked in a design exhibiting good overall 𝐷-efficiency. Conversely, a 𝐷_𝑠-inefficient subset may contraindicate, incorrectly, the overall 𝐷-efficiency of a design. Details are provided in case studies to follow.

5. Case Studies

5.1. The Setting

Our tools are informative in input-output studies. In particular, specify {𝐘=𝐗₀𝜷+𝝐} as a second-order model 𝑌(𝑥₁,𝑥₂,𝑥₃) in three regressors and 𝑝=10 parameters, namely, 𝑌_𝑖=𝛽₀+𝛽₁𝑥_𝑖1+𝛽₂𝑥_𝑖2+𝛽₃𝑥_𝑖3+𝛽₁₁𝑥^2𝑖1+𝜷₂₂𝑥^2𝑖2+𝛽₃₃𝑥^2𝑖3+𝛽₁₂𝑥_𝑖1𝑥_𝑖2+𝛽₁₃𝑥_𝑖1𝑥_𝑖3+𝛽₂₃𝑥_𝑖2𝑥_𝑖3+𝜖_𝑖;𝑖=1,2,…,𝑛.(5.1) Next partition 𝜷^=[𝛽₀,𝜷^𝐿,𝜷^𝑄,𝜷^𝐼] with 𝜷^𝐿=[𝛽₁,𝛽₂,𝛽₃] as slopes; 𝜷^𝑄=[𝛽₁₁,𝛽₂₂,𝛽₃₃] as pure quadratic terms reflecting diminishing (−) or increasing (+) returns to inputs; and 𝜷^𝐼=[𝛽₁₂,𝛽₁₃,𝛽₂₃] as interactive terms reflecting synergistic (+) or antagonistic (−) effects for pairs of regressors in combination. Further let 𝜷^𝑀=[𝜷^𝐿,𝜷^𝑄,𝜷^𝐼] exclusive of 𝛽₀, the latter a base line for 𝑌(0,0,0). We proceed under conventional homogeneous and uncorrelated errors, the minimizing solution 𝜷=(𝐗^0𝐗₀)⁻¹𝐗^0𝐘 being unbiased with 𝑉(𝜷)=𝜎²(𝐗^0𝐗₀)⁻¹. We take 𝜎², although unknown, to reflect natural variability in experimental materials and protocol, and thus applicable in a given setting independently of the choice of design. Accordingly, for present purposes we may standardize to 𝜎²=1.0 for reasons cited earlier.

5.2. The Designs

Early polynomial response designs made use of factorial experiments, setting levels as needed to meet the required degree. For example, the second-order model (5.1) in three regressors would require 3³=27 runs. However, in the early 1950s such designs were seen to be excessive, in carrying redundant interactions beyond the pairs required in the model (5.1). In industrial and other settings where parsimony is desired, several small second-order designs have evolved, often on appending a few additional runs to two-level factorials or fractions thereof.

Eight such small designs of note here are the hybrids (H310,H311B) of [38], the small composite SCD [39], the BBD [40], the central composite rotatable design CCD [41], and designs ND [42], HD [43], and BDD [44]. The designs [H310,H311B,SCD,BBD,CCD,ND,HD,BDD] have numbers of runs as [11,11,11,13,15,11,11,11], respectively. These follow on adding a center run to all but design ND, rendering all as unsaturated having at least one degree of freedom for error. Specifically, the design ND of [42] already has 11 runs and is unsaturated. All designs have been scaled to span the same range for each regressor; and none strictly dominates another under the positive definite dispersion ordering. All determinants as listed derive from the respective 𝑉(𝜷) = (𝐗^0𝐗₀)⁻¹ and its submatrices. Subset efficiencies for {𝜷_𝐿,𝜷_𝑄,𝜷_𝐼} were examined in [45] for selected designs using criteria other than 𝐷- and 𝐷_𝑠-efficiencies. Our usage here, as elsewhere in the literature, considers 𝐺𝑉(𝜷) and 𝐺(𝜷) to be efficiency indices for 𝜷 specific to a particular design, to include subsets {𝜷_𝑖;𝑖∈𝖨}, and smaller values reflect greater efficiencies through smaller volumes of concentration ellipsoids. On the other hand, the comparative efficiencies of two designs for estimating 𝜷 or {𝜷_𝑖;𝑖∈𝖨} are found as ratios of these quantities.

5.3. Numerical Studies

Details for these designs are listed in the accompanying tables. Table 1 gives values 𝐺(⋅)=[𝐺𝑉(⋅)]^1/dim for 𝜷 and selected subsets, with dim as the order of the determinant. Also listed are angles 𝜙(𝟏_𝑛∶𝐗)deg between regressors and the constant, to be noted subsequently. Table 2 displays the squared canonical correlations 𝝆²(𝜷_𝑖∶𝜷_𝑗) between designated subsets, and Table 3 the corresponding Vector Alienation Coefficient 𝜸(𝜷_𝑖∶𝜷_𝑗)=Π^𝑟𝑖=1(1−𝜌^2𝑖), for specified pairs. Here {0L,QI} refers to the pair {[𝛽̂₀,𝜷_𝐿],[𝜷_𝑄,𝜷_𝐼]}, for example. Moreover, values of the composite indices 𝜸(𝜷_𝑖∶𝜷_𝑗) = 𝜸(𝐗_𝑖∶𝐗_𝑗), if much less than unity, serve to alert the user as to potential problems with ill-conditioning.

Table 1 

Roots 𝐺(𝜷) and 𝐺(𝜷_𝑖) of generalized variances for 𝜷 and subset estimators {𝜷_𝑖∈ℝ^𝑡}, and angles 𝜙(1_𝑛∶𝐗)𝑑𝑒𝑔 between regressors and the constant, against reference values 𝜙^∗∗<18.44 and 𝜙^∗<30.00, in eight small second-order designs.

Table 2 

Squared canonical correlations between designated subsets (𝜷_𝑖,𝜷_𝑗) of estimators for eight small second-order designs.

Table 3 

Vector Alienation Coefficients 𝛾(𝜷_𝑖∶𝜷_𝑗) between subsets (𝜷_𝑖,𝜷_𝑗) of estimators, and the factor 𝑎𝑏𝑐=[𝛾(0∶𝐿𝑄𝐼)𝛾(𝐿∶𝑄𝐼)𝛾(𝑄∶𝐼)]^1/10, for each of eight small second-order designs.

5.3.1. An Overview

To fix ideas, observe for the CCD that 𝐺(𝜷_𝑄,𝜷_𝐼)=1.13633, 𝐺(𝜷_𝑄)=1.03300, and 𝐺(𝜷_𝐼)=1.25000 from Table 1. These not only are comparable in magnitude, but are commensurate, in having been adjusted for dimensions and thus homogeneous of unit degree, as are all entries in Table 1. Moreover, since (𝜷_𝑄,𝜷_𝐼) are uncorrelated and 𝛾(𝜷_𝑄,𝜷_𝐼) = 1.0 for the CCD from Table 3, 𝐺(𝜷_𝑄,𝜷_𝐼) is the geometric mean 1.13633=(1.03300)^3/6(1.25000)^3/6 from Theorem 4.1(ii). A further rough spot check of Table 1 may be summarized as follows.

Summary Properties
(P1)Compared with 𝐺(𝜷), values for 𝐺(𝛽̂₀) appear excessive throughout.(P2)Values for 𝐺(𝜷_𝐿) are roughly comparable across designs.(P3)The eight designs sort essentially into two groups.(P4)Designs {H310,H311B,SCD,BBD,CCD} overall are comparatively 𝐷- and 𝐷_𝑠-efficient, with the noted exception being 𝐺(𝜷_𝐼)=4.16667 for the SCD.(P5)The designs {ND,HD,BDD} are considerably less 𝐷-efficient, with their generalized variances 𝐺𝑉(𝜷) being {1192.09,4768.37,2886.03}, respectively, in comparison with {57.342,11.852,74.422,2.722,0.523} for the remaining designs; and each of the former is burdened by unequivocal 𝐷_𝑠-inefficiency for 𝜷_𝑄, to be treated subsequently.

5.3.2. Further Details

We next examine Hartley’s [39] SCD in some detail, first in terms of generalized variances. Values for 𝐺𝑉(𝜷),  𝐺𝑉(𝛽̂₀), and 𝐺𝑉(𝜷_𝑀) appear in the first row of Table 4, along with 𝜸(𝛽̂₀∶𝜷_𝑀)=(1.0−0.909090)=0.090909 using 𝝆²(0∶𝑀)=0.909090 from Table 2. Theorem 4.1 (i) now asserts that 𝐺𝑉(𝜷)=𝐺𝑉(𝛽̂₀)𝐺𝑉(𝜷_𝑀)𝜸(𝛽̂₀∶𝜷_𝑀), as verified numerically through 74.4216=10.00(81.8638)(0.090909). In a similar manner, 𝜷_𝑀 partitions into {𝜷_𝐿,[𝜷_𝑄,𝜷_𝐼]}, where 𝐺𝑉(𝜷_𝐿)=4.6296 and 𝐺𝑉([𝜷_𝑄,𝜷_𝐼])=81.8638 from Table 4. The squared canonical correlations between {𝜷_𝐿,[𝜷_𝑄,𝜷_𝐼]} are 𝝆²(𝐿∶𝑄𝐼)=[0.4000,0.4000,0.4000]^ from Table 2, so that 𝜸(𝐿∶𝑄𝐼)=0.21600 as in Table 3. Theorem 4.1 (i) again recovers 𝐺𝑉(𝜷_𝑀) as 81.8638=4.6926(81.8638)(0.21600) since 𝐺(𝜷_𝐿) and 𝜸(𝐿∶𝑄𝐼) are reciprocals in this instance. Moreover, 𝐺𝑉(𝜷_𝑄,𝜷_𝐼)=𝐺𝑉(𝜷_𝑄)𝐺𝑉(𝜷_𝐼)𝜸(𝑄∶𝐼) translates into 81.8638=1.1317(72.3379)(1.0), where 𝜸(𝑄∶𝐼)=1.0 since elements of {𝜷_𝑄,𝜷_𝐼} are mutually uncorrelated from Table 2. In summary, the value 𝐺(𝜷) for the SCD admits the factorization of (4.6), on identifying {𝜽₁,𝜽₂,𝜽₃,𝜽₄} with {𝛽̂₀,𝜷_𝐿,𝜷_𝑄,𝜷_𝐼}, respectively, given numerically from Tables 1 and 3 as 1.53876=(10.00)^1/10(1.66667)^3/10(1.04210)^3/10(4.16667)^3/10[(0.090909)(0.21600)(1.0)]^1/10.(5.2)

Table 4 

Generalized variances and Vector Alienation Coefficients between designated subsets for Hartley’s [39] 𝑆𝐶𝐷 in 𝑘=3 regressors.

Corresponding factorizations proceed similarly for other designs. Details are left to the reader, but values for [𝜸(0∶𝐿𝑄𝐼)𝜸(𝐿∶𝑄𝐼)𝜸(𝑄∶𝐼)]^1/10, the rightmost factor of (5.2), are supplied for each design as the final row of Table 3. Although the tables, together with Theorem 4.1, support other factorizations, the one featured here seems most natural in terms of the parameters {𝛽₀,𝜷_𝐿,𝜷_𝑄,𝜷_𝐼}, together with their central roles in identifying noteworthy treatment effects in second-order models.

5.4. Masking

The 𝐷-efficiency index of the SCD, at 𝐺𝑉(𝜷)=74.4216, is larger but roughly comparable to that of H310 at 𝐺𝑉(𝜷)=57.3418. What cannot be anticipated from these facts alone, however, is that the (3×3) determinant 𝐺𝑉(𝜷_𝐼)=72.3379 for the SCD is comparable to its (10×10) determinant 𝐺𝑉(𝜷)=74.4216, despite their disparate dimensions. Adjusting for dimensions gives 𝐺(𝜷)=(74.4216)^1/10=1.53876 and 𝐺(𝜷_𝐼)=(72.3379)^1/3=4.16667 for the SCD. This illustrates the masking of a remarkably inefficient estimator for 𝜷_𝐼, despite the value 𝐺(𝜷)=1.53876 in estimating all parameters. This masking stems from the nonorthogonality of subset estimators as reflected in their canonical correlations and Vector Alienation Coefficients. In contrast are the corresponding commensurate values for the H310 design, namely, 𝐺(𝜷)=(57.3418)^1/10=1.49916 and 𝐺(𝜷_𝐼)=(4.1768)^1/3=1.61045. It may be noted that the condition number 𝑐₁(𝐗^0𝐗₀) is 21.59 for H310, with the somewhat larger value 54.01 for the SCD.

We next examine the 𝐷_𝑠-inefficiencies of ND and HD for 𝜷_𝑄 as noted earlier, with 𝐺(𝜷_𝑄) taking values 7.80031 and 6.61313, respectively. Our reference for masking is 𝐺(𝜷_𝐿,𝜷_𝑄). These values are not listed in Table 1, but may be recovered from Tables 2 and 3 as follows. Specifically, for ND we have 𝐺𝑉𝜷^𝐿𝐺(𝑉1.25𝜷0^𝑄00)𝛾_3/6(𝐿∶𝐐)^1/6=𝐺𝑉𝜷_𝐿,𝜷_𝑄^1/6=𝐺𝜷_𝐿,𝜷_𝑄,(7.80031)^3/6(0.72428)^1/6=2.95912,(5.3) where neither 𝐺(𝜷_𝐿)=1.25000 nor 𝐺(𝜷_𝐿,𝜷_𝑄)=2.95912 appears excessive. In consequence, that 𝐺(𝜷_𝑄)=7.80031 is excessive would be masked on examining 𝐺(𝜷_𝐿) and 𝐺(𝜷_𝐿,𝜷_𝑄) only. Parallel steps for HD give the factorization (1.91189)^3/6(6.61313)^3/6(0.78514)^1/6 = 3.41528, with similar conclusions in regard to masking.

5.5. Collinearity with the Constant

Advocates for these and other small designs have focused on 𝐷,𝐷_𝑠, and other efficiency criteria, as well as the parsimony of small designs and their advantage in industrial experiments. To the knowledge of this observer, none has considered prospects for ill-conditioning and its consequences, despite the fact that columns of 𝐗 are necessarily inter-linked as a consequence of second-order from first-order effects. Nonetheless, from Section 3.1 and Corollary 3.2, we may compute angles between the constant vector and the span of the regressors using duality together with the information at hand. This may prove to be critical in view of the admonition [29] that “collinearity with the intercept can quite generally corrupt the estimates of all parameters in the model.” As noted in Remark 3.4, rules-of-thumb for problematic VIFs include those exceeding 10 or 4 or, in angular measure, 𝜙^∗∗<18.435deg and 𝜙^∗<30.00deg. From the last row of Table 2, the angles 𝜙(𝟏_𝑛∶𝐗) have been computed for each of the eight designs, as listed in the final column of Table 1. For example, arccos(√0.81990)=25.1116deg for H310. It is seen that all designs are flagged as potentially problematic using rules-of-thumb as cited. This adds yet another layer of concerns, heretofore unrecognized, in seeking further to implement these designs already in wide usage.

6. Conclusions

Duality of (i) Hotelling’s [30] canonical correlations {𝜌₁,…,𝜌_𝑟} between the 𝑂𝐿𝑆 estimators {𝜷₁,𝜷₂} and (ii) the design linkage parameters {𝛿₁,…,𝛿_𝑟} between {𝐗₁,𝐗₂} is established at the outset. Stewart’s [33] collinearity indices are then extended to encompass angles {𝜙₀,𝜙₁,…,𝜙_𝑘} between each column of 𝐗₀=[𝟏_𝑛,𝐗₁,…,𝐗_𝑘] and remaining columns. In particular, 𝜙₀ quantifies numerically the collinearity of regressors with the intercept, of concern in the prospective corruption of all estimates due to ill-conditioning.

Matrix identities factorize a determinant in terms of principal subdeterminants and the Vector Alienation Coefficients of [30] between {𝜷₁,𝜷₂}. By duality, the latter also are Alienation Coefficients between {𝐗₁,𝐗₂}. These identities in turn are applied in the study of 𝐷_𝑠-efficiencies for the parameters {𝛽₀,𝜷_𝐿,𝜷_𝑄,𝜷_𝐼} in eight small second-order designs from the literature. Studies on 𝐷_𝑠- and 𝐷-efficiencies, as cited in our opening paragraph, confirm that designs are seldom efficient for both. Our determinant identities support a rational explanation. In particular, these identities unmask the propensity for 𝐷_𝑠-inefficient subset estimators to be masked through near collinearities in overall 𝐷-efficient designs.

Finally, the evidence suggests that all eight designs are vulnerable, to varying degrees, to the corruption of all estimates due to ill-conditioning. In short, we have exposed quantitatively the structural origins of masking through Hotelling’s [30] canonical correlations, and their equivalent design linkage parameters. This analysis in turn proceeds from the design matrix itself rather than empirical estimates, so that any design can be evaluated beforehand with regard to masking and possible subset inefficiencies, rather than retrospectively after having committed to a given design in a particular experiment.