Validity of Adult Psychopathology Model Using Psychiatric Patient Sample from a Developing Country: Confirmatory Factor Analysis

Alareqe, Naser Abdulhafeeth; Hassan, Siti Aishah; Kamarudin, Engku Mardiah Engku; Aljaberi, Musheer A.; Nordin, Mohamad Sahari; Ashureay, Nadeem Mohamed; Mohammed, Lubna Ali

doi:https://doi.org/10.1155/2022/9594914

Mental Illness

On this page

Abstract Introduction Results Discussion Conclusion Data Availability Ethical Approval Consent Disclosure Conflicts of Interest Authors’ Contributions Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2022 | Article ID 9594914 | https://doi.org/10.1155/2022/9594914

Validity of Adult Psychopathology Model Using Psychiatric Patient Sample from a Developing Country: Confirmatory Factor Analysis

Naser Abdulhafeeth Alareqe,¹Siti Aishah Hassan,^2,3Engku Mardiah Engku Kamarudin,²Musheer A. Aljaberi,^4,5,6Mohamad Sahari Nordin,⁷Nadeem Mohamed Ashureay,¹and Lubna Ali Mohammed⁸

Academic Editor: S M Yasir Arafat

Received24 Aug 2022

Revised29 Nov 2022

Accepted30 Nov 2022

Published21 Dec 2022

Abstract

Objectives. This study is aimed at testing and validating the two-factor measurement model of the Millon Clinical Multiaxial Inventory (MCMI). Specifically, this paper reported construct validity, particularly focusing on convergent and discriminant validities of the internalizing-externalizing MCMI model of adult psychopathology using a psychiatric sample from a developing country, the Republic of Yemen. Methods. MCMI was distributed among 232 outpatients from the Hospital of Taiz City and two private psychiatry clinics in Yemen; data were collected using structured interviews over four months. We used exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) to explore and confirm the latent structure MCMI and verify the evidence of convergent and discriminant validity. Results. The CFA results indicated that MCMI was a good fit for the internalizing-externalizing two-factor model of adult psychopathology, comparative fit index , and . The results of the CFA provide evidence of convergent and discriminant validity characterized by MCMI with the internalizing-externalizing model. Conclusion. The adult psychopathology of internalizing-externalizing is a valid measurement model of MCMI with ten personality disorders and eight clinical syndromes.

1. Introduction

Children’s behavioral issues are now conceptualized differently by researchers, clinicians, and educators as a result of Achenbach’s original findings on childhood psychopathology assessment [1, 2]. One of the fundamental aspects of the analyses is the hierarchical conceptualization of mental disorders in terms of two broad behavioral spectra, which Achenbach termed internalizing and externalizing. The spectra accounted for the systematic covariation among more narrowly defined behavioral problems [3]. Internalizing problems in childhood include anxiety, depression, and somatization-based disorders while externalizing problems include behavioral misconduct, anger, and attentional difficulties [2].

Another contribution of Achenbach’s work is its impact on approaches to conceptualizing adult psychopathology [4]. As a potential theoretical framework for organizing the structure of adult mental disorders, this internalizing-externalizing model has received a significant amount of attention [4–6]. Internalization is the propensity to express distress inwards; common internalizing disorders include major depression, dysthymia, generalized anxiety disorder, agoraphobia, social phobia, simple phobia, and obsessive-compulsive disorder. Conversely, externalization describes the propensity to express distress outwards; commonly recognized externalizing disorders include conduct disorder, antisocial personality disorder, marijuana dependence, and alcohol dependence [7–20].

The internalizing-externalizing spectra have been proposed as an organizational system applied in DSM-5 [5]. Regarding this change, several advantages have been highlighted, including the use of a more parsimonious and meaningful organizational model/schema, the ability to simplify problems caused by excessive comorbidity, aiding intervention and treatment development, and creating an important area for future research [6, 8, 21–23]. This internalizing-externalizing framework has been demonstrated across multiple disorders [17, 24], genders [25, 26], ethnicities [26], cultures [10], and over time [7]. It has also been highly successful in accounting for the relationships between psychopathology and other constructs [27] in explaining the etiology of psychopathology in twins’ studies [28, 29] and in studying adults with attention-deficit/hyperactivity disorders (ADHD) [30]. Although there is growing recognition that comorbidity among individual mental disorders is better understood by the broad psychiatric dimensions of internalization and externalization, expanding the scope of this framework is still needed [12, 24].

Previous studies indicated that the internalizing and externalizing spectra capture much of the genetic vulnerability to common mental disorders [31, 32]. In a severely extremely sick cohort, Kotov et al. [33] study also supported the essential internalizing-externalizing concept. These spectra seem to be comparable across cultures, ages, and inpatient samples. This knowledge can be useful because the new classification promises to improve our understanding of psychopathology in a variety of ways. However, assessments of internalizing-externalizing difficulties were based on ad hoc questions, scales, or scale combinations that were unique to each study. It is recommended to recognize that internalizing and externalizing problems are not mutually exclusive or completely independent of one another but are moderately correlated in many samples. Assessments of broadband problem groups, narrow-band syndromes, and individual problems can all be valuable for early clinical evaluation progress, outcome evaluations, and research on etiologies, treatments, outcomes, and epidemiology [34].

The Hierarchical Taxonomy of Psychopathology (HiTOP) is a scientific effort to address shortcomings of traditional mental disorder diagnoses, which suffer from arbitrary boundaries between psychopathology and normality, frequent disorder cooccurrence, heterogeneity within disorders, and diagnostic instability [33, 35–38]. The Hierarchical Taxonomy of Psychopathology (HiTOP) is a new classification of mental illness. It aims to address several major shortcomings of traditional taxonomies and provide a better framework for researchers and clinicians.

The HiTOP model included six spectra: internalizing (or negative affectivity), thought disorder (or psychoticism), disinhibited externalizing, antagonistic externalizing, detachment, and somatoform. Given the direct correspondence between internalizing and negative affectivity as well as between thought disorder and psychoticism, each of these pairs is represented by one dimension. Externalizing behavior has two personality counterparts: disinhibition and antagonism. Disinhibition is particularly prominent in substance-related disorders, and antagonism is especially significant in narcissistic, histrionic, paranoid, and borderline PDs. Both disinhibition and antagonism contribute to antisocial behavior, aggression, ODD, ADHD, and IED [33, 35–43].

1.1. Theoretical Gap

Millon et al. [44] theorized that psychopathology reflected by MCMI-III can be classified under four latent structures or the four-factor model of Millon: the 11 clinical personality disorders, the three severe personality disorders, the seven clinical syndromes, and the three severe clinical syndromes. However, both cluster analysis and exploratory/confirmatory factor analysis studies of mental disorders have found evidence that internalizing and externalizing dimensions have underlying common psychopathology in adults [4, 9, 27]. There is also evidence that a limited number of mental disorders represent the internalizing-externalizing spectrum of psychopathology. In particular, the internalizing dimension in previous studies of adult samples was composed of a few mental disorders. For example, Krueger and Markon [24] performed a meta-analysis of relevant literature, applied a CFA to pool data on 11 common mental disorders, and replicated the two general dimensions. The externalizing cluster was composed of substance use disorders, conduct disorders, and adult antisocial behavior. However, the internalizing conditions were only all depressive and anxiety disorders. Although adult psychopathology’s internalizing-externalizing forms are now reasonably established, this 2-spectrum framework is still far from a comprehensive structure, including several mental disorders [7, 33]. Krueger and Markon [24] concluded that expanding the scope of the internalizing-externalizing model by covering a greater diversity of psychopathological symptoms and personality constructs is required and would be a novel step for this model. For example, Krueger and Markon [24] carried out a meta-analysis of relevant literature, applied a CFA to pool data on 11 common mental disorders, and replicated the two general dimensions. The externalizing cluster was composed of substance use disorders, conduct disorders, and adult antisocial behavior. However, the internalizing conditions were only for depressive and anxiety disorders. Therefore, the current study is aimed at expanding the internalizing dimension by covering several more mental disorders, including personality disorders and clinical syndromes, totaling up to 18 internalizing psychopathologies.

1.2. Methodological Gap

In addition, the validity of the internalizing-externalizing model for psychopathology in adults has been replicated in population-based samples from several Western, industrialized nations [4, 9, 19, 27, 45]. However, the applicability of this model to mental disorders in non-Western countries remains unknown [10]. Hence, applying the internalizing-externalizing model of psychopathology in a non-Western country is in order. Methodologically, robust evidence for the internalizing-externalizing model for psychopathology has been supported by investigations with diagnostic interviews and multidimensional self-report measures [4, 9, 10, 45, 46]. Krueger and Markon [24] indicated that advanced statistical methods (e.g., structural equation models, item-response models, and growth-curve models) have allowed a tremendous increase in sophistication in our confidence regarding psychological theory conclusions tested using those methods. These methods also hold great promise for understanding psychopathology because they allow empirical comparison of different classification paradigms. Such paradigms can be represented by different quantitative models and can be rigorously compared by comparing the fit of those models to psychological data (p.114, 2006).

However, the Millon Clinical Multiaxial Inventory [47] is a widely used measure of personality disorders and psychopathology; it is evaluated as a third international scale in the clinical population [48–50]. The scale has provided good evidence of convergent and discriminant validity [44, 51] with several versions. It has been used in limited studies to examine (i) Millon’s four factors of psychopathology (11 clinical personality disorders, three severe personality disorders, seven clinical syndromes, and three severe clinical syndromes) and (ii) adequate fit of the internalizing-externalizing model using the maximum likelihood approach of CFA. One possibility of this advanced analysis is to provide powerful evidence of construct validity for the instrument [52].

Many researchers have employed exploratory techniques for MCMI [53–58], while few recent studies have used confirmatory methods with limited scales of MCMI. For example, Cuevas et al. [59] analyzed 14 personality disorders of MCMI using the maximum likelihood of confirmatory factor modeling among undergraduate students as the study sample. Another pioneering study by Rossi et al. [60] used the confirmatory factor method to analyze 14 disorders of MCMI, obtaining a four-factor solution including the internalizing-externalizing model in clinical patients’ forensic settings. Only Ruiz and Edens [61] used a CFA to analyze the 24 MCMI clinical scales. However, their model used a correlation matrix of MCMI, estimating the internalizing and externalizing dimensions and the general factor of personality disorders. Although their model demonstrated adequate fit, it did not provide MCMI’s construct validity evidence in convergent and discriminant validity. In summary, evidence of convergent and discriminant validity using the maximum likelihood of confirmatory factor modeling for MCMI has been limited; therefore, this research addressed this concern. Mainly, the current study is aimed at examining the Millon Clinical Multiaxial Inventory-III (MCMI) fit with the internalizing-externalizing model of psychopathology and, secondly, evaluating evidence of convergent and discriminant validity MCMI.

2. Methodology

2.1. Research Design

The research design of the current study was based on correlational structural equation modeling that was dependent on the covariance matrix analysis [52, 62]. The list of observable variables of personality disorders and psychopathology in MCMI was endogenous/dependent variables. In contrast, latent constructs, namely, internalizing and externalizing dimensions of psychopathology, were evaluated as exogenous variables. Subsequently, the structural model in the current study was under confirmatory factor modeling [52, 62–64].

2.2. Sample Size Determination

Hair et al. suggested that for covariance based on SEM (e.g., AMOS), the analysis needs a sample size greater than 100 [63]. In this study, 212 sample size was considered adequate for employing structural equation modeling (SEM) to address the research objectives. Concisely, the determined sample size was satisfactory for conducting confirmatory factor analysis (CFA) [64].

2.3. Participants and Procedures

The data was collected from 232 outpatients who were mainly diagnosed with mood disorders (71.9%), followed by substance abuse disorders (28.0%) in the Hospital of Taiz City and two private psychiatry clinics in Yemen. The inclusion criteria for selecting the outpatients were (i) Yemeni nationality, (ii) receiving a medical treatment in a psychiatric clinic for more than one year, and (iii) being more than eighteen years old and above. For the second criterion, the report of 12 months of treatment was used to clearly classify the patient into psychiatric and nonpsychiatric cases [65-67]. The final eligible sample size of 212 was used, considered the optimal sample size for CFA [68]. The data were collected over four months using a simple random sampling technique of the eligible registered patients. The response included males (86.3%). The age of the outpatients ranged from 20 to >50 years, with an average age of 30.81 years. Most of the participants were married (56.63%), whereas 34% were single and 9.4% were divorced. About 28.3% of the sample had primary education (writing and reading skills), 23.1% had primary education, 35.4% held secondary school certificates, and 13.2% were degree holders.

The study received ethical approval from the academic committees of the Department of Educational Psychology and Counseling at International Islamic University Malaysia (IIUM), Malaysia. In addition, every patient gave informed consent to participate in this research.

2.4. Patient and Public Involvement

In this study, the data was collected using Millon Clinical Multiaxial Inventory-III after consent had been taken from the outpatient to participate by answering the survey. However, no patient was involved in setting the research question or the outcome measures. Furthermore, the participants were also not involved in the design or conduct of the study. No patients were requested to provide advice on interpreting or reporting the results. The study results were not planned to be disseminated to the participants.

2.5. Millon Clinical Multiaxial Inventory (MCMI)

Psychopathology was measured using the Millon Clinical Multiaxial Inventory-III (MCMI-III) [47]. This self-report personality and diagnostic inventory consisted of 175 true-false items addressing axis I (10 clinical syndromes) and axis II (14 personality disorders) based on the Diagnostic and Statistical Manual of Mental Disorders (4th ed. (DSM–IV)). The MCMI had an adequate internal consistency (alphas ranged from 0.66 for the compulsive scale to 0.90 for the major depression scale) [44]. The MCMI-III was translated into Arabic by the first author of this study and two psychology professors who were proficient in the language with the help of the English Center of the International Islamic University Malaysia because there was no Arabic version of the test at the time the current study was conducted. The differences between the two versions were resolved when the first and back translations were done [69–72]. Moreover, the test-retest reliability of the MCMI scales was obtained by readministering the scale between 5 and 14 days after the initial administration to 0.87 subjects. The validity ranged from 0.84 for the anxiety scale to 0.96 for the somatoform scale. The median stability coefficient was 0.91, thus indicating that the MCMI results were highly stable over a short period [44, 53].

2.6. Statistical Analysis

According to Forbush and Watson [39], data analysis was conducted in three steps. First, in Alareqe et al. [53], the items were subjected to exploratory factor analysis (EFA) to create scales that would be used in the subsequent structural studies. Second, a baseline confirmatory factor analysis (CFA) model was developed using the results of EFAs. Third, we used a Monte Carlo-based simulation that created 100 random datasets with the same number of participants and variables as the actual dataset. This analysis allowed for deciding when factors may be considered insignificant and when additional factor extraction was no longer necessary.

2.6.1. Exploratory Factor Analysis (EFA)

Using the orthogonal rotation method, EFA was conducted to determine the number of underlying factors in MCMI. The extracted factors should meet several criteria, including theoretical justification, scree plot, Kaiser’s , and total variance explained ≥0.60 [72–76]. In addition, the direction and proportion of factor loading should be ≥0.55, and the corrected item-total correlation must be at least 0.30 and positive [77].

2.6.2. Confirmatory Factor Analysis

While EFA is used to explore the underlying dimensions of the measured variable, CFA is conducted to confirm the hypothesized measurement model. CFA is meant for measurement models within structural equation modeling (SEM) analysis, whereas full-fledge SEM consists of a measurement model and structural model. In other words, CFA deals with the relationships between observed measures or indicators (e.g., test items, test scores, and behavioral observation ratings) and latent variables or factors, but not the relationship between exogenous latent variables to endogenous latent variables [52, 69, 78–81]. Therefore, multiple tests were used in this study to evaluate the fit of the hypothesized model for the data. Kline [64] recommended a minimal set of four fit indexes, including the chi-square test (), the comparative fit index (CFI) [82], standardized root mean residuals (SRMR), and root mean square errors of approximation (RMSEA). Chi-square model included chi-square test, degree of freedom (DF), and nonsignificant value. However, it is well known that significant testing is greatly influenced by certain factors such as sample size, high correlations [64], and model complexity [83]. Both SRMR and RMSEA must be ≤0.08 [64, 84]. Additionally, normed chi-square () and incremental fit index (IFI) were also used to evaluate the hypothesized model fit [85]. The normed chi-square () should be ≤3. Both CFI and IFI must be ≥0.90 [64, 74, 78, 82, 86, 87].

2.6.3. Convergent Validity and Discriminant Validity

Convergent and discriminant validity of MCMI as components of construct validity was tested based on suggestions provided by several statisticians [52, 64, 77, 84, 85, 88]. This study refers to how well each indicator (disorder) represents or relates to its construct (internalizing or externalizing. For convergent validity, the threshold value for the factor loading is ≥0.4. The threshold value for average variance extraction (AVE) is ≥0.5 by using the following formula [89, 90]:

For construct reliability (CR), the threshold value is ≥0.7 by using the following formula [90]:

From the point of the construct level, the discriminant validity refers to the strength of the correlation coefficient among the latent constructs; that is, when two latent constructs are moderately correlated (not too high, e.g., 0.85, <0.90), the value of correlation coefficient is not too small (<0.20) [91]; and at the item level, it demonstrates the absence of cross-loading [90]. Discriminant validity is assessed by comparing the shared variance (squared correlation) between each pair of constructs against the average of the AVEs for these two constructs [90, 92]. In addition, Henseler et al. [93] suggested the heterotrait-monotrait ratio of correlations (HTMT) as the new approach to assessing discriminant validity; “If the indicators of two constructs, ξi, and ξj, exhibit an HTMT value that is clearly smaller than one, the true correlation between the two constructs is most likely different from one, and they should differ”[89].

3. Results

3.1. CFA and Millon’s Four Factors of Psychopathology

First, we accessed Millon’s hypothesized four psychopathology factors underlying the MCMI. The results of EFA provide evidence that the four-factor structure did not converge to an admissible solution, and the results of the covariance matrix were not definitely positive. Moreover, the four intercorrelation factors from the six relationships yielded offending estimates (). Subsequent attempts to solve this improper problem by eliminating the intercorrelations and fixing them at 1 also led to an inadmissible solution. Similarly, using the different methods of estimation such as generalized least squares, unweighted least squares, scale-free least squares, and asymptotically distribution-free produced an improper solution. Millon’s classification of psychopathology in the four-factor model did not fit the present data. Accordingly, to search for a more appropriate solution, we explored the number of factor structures underlying MCMI using EFA to obtain an initial validation, shown in Figure 1. The scree plot provides evidence for the three-factor solution of MCMI.

3.2. Exploratory Factor Analysis (EFA)

The eigenvalue of EFA produced an initial three-factor solution, which accounted for about 72.10% of the total variance, confirmed by the scree plot (Figure 1). The first factor, which explained 57.10% of the total variance, is named “internalizing psychopathology” and its . This factor consisted of the following eighteen factors with an excellent ratio of loading (>0.55): paranoid (0.83), anxiety (0.82), posttraumatic stress (0.80), depressive (0.80), thought disorder (0.81), schizotypal (0.80), dysthymia (0.80), avoidant (0.79), major depression (0.79), borderline (0.75), somatoform (0.73), delusional disorder (0.73), bipolar manic (0.70), masochistic (0.70), negativistic (0.70), dependent (0.66), sadistic (0.67), and schizoid (0.59).

The second factor, which modeled 7.49% of the total variance, is “externalizing psychopathology” with an . This term involved four factors with an excellent ratio of loading (>0.55) as follows: antisocial personality disorder (0.87), drug dependence (0.81), alcohol dependence (0.74), and compulsive personality disorder (-0.61) (Table 1). It was noted that compulsive personality disorder was related in the opposite direction across the orthogonal rotation with the other three disorders in the same factor. The third factor, which accounted for 4.40% of the total variance, is “emotional disturbances” and . This factor contained two factors with small loading (<0.55), which are narcissistic personality disorder (0.51) and histrionic personality disorder (0.43) as identified by orthogonal rotation. Therefore, the third factor was dropped in the subsequent analysis (CFA).

3.3. Confirmatory Factor Analysis (CFA)

The skewness and kurtosis indices demonstrated no evidence of nonnormality for the MCMI’s 21 factors (largest skewness for paranoid; largest kurtosis for posttraumatic stress disorder; a value of +/-1.0 is generally considered the cut-off for data nonnormality). The value of Mardia’s coefficient was 1.21, thus demonstrating evidence for multivariate normality. An estimate of ≤2.0 is usually reflected as the cut-off for multivariate data normality. Given that every variable was normally distributed, the ML estimation was used to analyze the confirmatory factor modelling.

CFA revealed that the two-factor model of the MCMI (internalizing and externalizing forms of psychopathology) was free from any improper solution of CFA. There were no offending estimates, loading ≥1.0, and no negative error variance value. The model converged to an admissible solution with all parameter estimations reaching a statistically significant level. However, its fit statistics were less satisfactory, thus suggesting the need for revising it. For instance, , , and failed to be reasonably adequate. After allowing the errors to be correlated between depressive and major depression and major depression and thought disorder (not shown in the figure for neat and clarity purposes), CFA results of the revised measurement model of the two-factor model of MCMI indicated that its fit statistics seemed reasonably good and plausible (e.g., , DF =175, , , , , , and ) (Figure 2).

The critical values indicate that the unstandardized factor loadings for all disorders were statistically significant (≥1.96), ranging from 10.16 for paranoid personality disorder to 20.02 for drug dependence. More specifically, the standardized factor loadings for each disorder in the internalizing psychopathology construct varied from good (0.68) for paranoid to excellent (0.90) for thought disorder. Similarly, the standardized factor loading for each indicator in the externalizing psychopathology construct was profound. Hence, the convergent validity for the two-factor model of MCMI seemed substantially credible.

The squared multiple correlations (SMC) represent the proportion of variance in each disorder, accounted for by its respective factor. Indicators with small SMC values represent potentially weak evidence of construct validity, while the squared multiple correlations (SMC) are 0.50 and above, which is an ideal value. Except for the paranoid ( as a good value), the squared multiple correlations of all disorders had an ideal value of , showing that both internalizing and externalizing psychopathology constructs explained a high proportion of variance in their respective disorders as shown in Figure 2.

3.4. Convergent Validity

The average variance extraction (AVE) indicated that the latent construct of internalizing psychopathology obtained a coefficient of 0.65, which fell above the recommended amount of 0.50. Likewise, the average variance extraction of externalizing psychopathology was 0.80, which reached the recommended threshold. This means that the large amounts of variance extracted for internalizing and externalizing constructs indicated an adequate percentage or greater than half of the variance for each set of the specified disorders modeled by each representative construct.

The construct reliability for internalizing and externalizing psychopathology constructs was 0.97 and 0.92. The values exceeded 0.70 as an acceptable level. Furthermore, the construct reliability based on standardized loading was 0.97 and 0.92 for internalizing and externalizing psychopathology constructs, exceeding the suggested threshold of 0.70. In conclusion, the construct reliability methods excellently indicated that the observed measures of disorders all adequately and consistently represented the same latent construct (internalizing and externalizing psychopathology).

3.5. Discriminant Validity

In this study, the finding showed that the interfactor correlation between internalizing and externalizing constructs of psychopathology was statistically significant: , , , (10.39/1.39), and (Figure 2). This correlation value was considered good evidence of the discriminant validity [90, 92, 93]. Next, the discriminant validity was estimated by comparing the AVE to squared correlation () or shared variance (SV) between internalizing psychopathology and externalizing psychopathology. The variance extracted was expected to be greater than the squared correlation value. Variance extractions of internalizing psychopathology (0.65) and externalizing psychopathology (0.80) were more than the squared correlation (0.53) between these two constructs, thus providing good evidence of the discriminant validity (Table 2).

At the item level, discriminant validity means that individual measured variables should represent only one latent construct. The absence of cross-loading for two constructs indicates discriminant validity. Factor score weights showed that antisocial, alcoholic, and drug disorders correlate highly with externalizing psychopathology. In contrast, all remaining disorders correlated highly with internalizing psychopathology, except the paranoid (Figure 2). At the construct level, discriminant validity can also be tested using chi-square difference tests. The unconstrained CFA model (with all factors freely correlated) was compared with a constrained model, and the correlation of two factors was set equal to one.

A significant change supports the discriminant validity. Table 3 shows that the chi-square difference and the goodness of fit were due to the added constraint, all getting significant (CFI and RMSEA) power; e.g., the constrained model fit is poorer than the unconstrained one. Thus, the two internalizing and externalizing psychopathology factors exhibited a discriminant validity. Next, the discriminant validity was determined by comparing the fit of the original model against the one-factor model. Table 3 shows that internalizing-externalizing psychopathology, as a two-factor model of MCMI, was more significantly plausible than the one-factor model of MCMI. Hence, the two-factor model of MCMI was sufficiently characterized by good evidence of the discriminant validity.

4. Discussion

The research objective of the present study was to examine the construct validity of MCMI with the two-factor model of psychopathology using the collected data from a psychiatric sample from a developing country, the Republic of Yemen. The result of CFA indicated that the MCMI was an adequate scale with a two-factor model, i.e., the internalizing-externalizing model of psychopathology. Precisely, fit statistics of CFA (CFI, IFI, and RMSEA) provided evidence for model fit data on MCMI with the internalizing and externalizing psychopathology from the psychiatric samples. Thus, it suggested that psychopathology is ideally classified into internalizing and externalizing. The eighteen disorders, schizoid, avoidant, depressive, dependent, sadistic, negativistic, masochistic, schizotypal, borderline, paranoid, anxiety, somatoform, bipolar manic, dysthymia, posttraumatic stress disorder, thought disorder, major depression, and delusional disorder, were ideally loaded on the internalizing form of psychopathology. The three disorders, antisocial, alcohol, and drug, were excellently loaded on the externalizing form of psychopathology. These findings on internalizing and externalizing are consistent with the results of previous studies [4, 7–19, 22, 23, 25, 30, 51, 60, 94].

However, it should be noted that the previous studies had found a small number of disorders (four to six disorders) representing and manifesting the internalizing model of psychopathology. The structure of internalizing psychopathology in this research was extended to ten personality disorders and eight clinical syndromes (a total of eighteen disorders). It captured the DSM-IV model of psychopathology: axis II (personality disorders) and axis I (clinical syndromes). Uniquely, the internalizing model in the current study not only validated mental disorders used in previous research on psychopathology but also included a wider range of mental disorders. This is congruent with Millon’s theory, which considers the importance and dominance of the broad general maladjustment factor with loadings from many disorders [44]. One plausible interpretation is that the expanded scope of internalizing disorders is justifiable since they are attributable to a shared set of genetic factors [4, 9]. Røysamb et al. [95] found that the internalizing factor contained six indicators from axis I (clinical disorders): anxiety disorder, major depressive disorder, dysthymia, anorexia nervosa, posttraumatic stress, and pain disorder. In addition, the two indicators from axis II (personality disorders), depressive and borderline, were loaded partly on this factor.

Theoretically, the expanded conceptualization of internalizing psychopathology is consistent with the ideas of Krueger and Markon [24], suggesting that the internalizing model can be tested with many mental disorders. Empirically, the result of the study is similar to the results of a previous study by Rushton and Irwing [16] who analyzed the correlation matrix of MCMI. Although the current study expands the model of internalizing-externalizing psychopathology, there are notable differences between the current model and previous models. For example, previous studies delineated internalizing psychopathology with two underlying factors: fear and distress [4, 10, 17, 22, 25], whereas the internalizing model of EFA [53] and the current study did not bifurcate into distinct distress and fear components of internalizing disorders. The bifurcation of two underlying factors was specified: fear and distress did not provide an admissible solution.

The externalizing psychopathology in this research included antisocial, alcoholic, and drug disorders. The present study results are consistent with the results of previous studies [4, 9, 12, 16, 17, 94, 95], which demonstrated that antisocial, alcoholic, and drug disorders all reflect an externalizing dimension. Substance abuse (e.g., alcohol and drug) and antisocial behavior are genetically linked to an impulsive personality [96]. In addition, these are both externalizing disorder indicators as the emotional release is directed outward without restriction [97], likely carried from a childhood of social values.

Additionally, the present study provides evidence of convergent and discriminant validities of MCMI with the internalizing-externalizing model of psychopathology. The result of CFA indicated the MCMI with the two-factor model (internalizing-externalizing model of psychopathology) was evidently characterized by convergent and discriminant validity. Given the ideal factor loadings of MCMI and the large proportion of variance explained, excellent construct reliability, and high average variance extraction, it is evident that the disorders within each construct of psychopathology measure consistently and sufficiently their respective construct: internalizing-externalizing of psychopathology. It can be concluded that a convergent validity adequately characterized MCMI with the internalizing-externalizing model of psychopathology. These findings are consistent with the results of the previous studies [16, 54, 58, 59]. Similarly, findings across five discriminant validity evidence confirmed that MCMI with the internalizing-externalizing model of psychopathology were characterized by discriminant validity.

The value of the latent correlation was 0.73 (), which was consistent with previous studies [4, 19]. The squared correlation between the two psychopathology constructs was less than their average variance extraction. These techniques of discriminant validity predict that each construct of psychopathology assesses distinct concepts and meanings in the hypothesized model. This finding supports previous studies that verified the distinction between internalizing and externalizing psychopathologies [24]. Therefore, it can be concluded that the two-factor model of MCMI was differently designed to be assessed, indicating that the internalizing psychopathology construct was distinct from the externalizing psychopathology construct; this was good evidence of discriminant validity. Next, the two factors for MCMI exhibited discriminant validity, which strongly confirmed that the two-factor model of psychopathology for MCMI appeared to be the most favorable. Psychopathology with more than two factors or single factor had unacceptable fit indices. These findings of the discriminant validity were consistent with a previous study that aimed to test the discriminant validity of MCMI regardless of the different methods of analysis [51].

4.1. Strengths/Originality and Limitations

The current research is the first to verify psychometric evaluations of construct validity comprised of convergent and discriminant validity, evidence of MCMI in a developing country using a sample of patients from outside Western culture, particularly in Yemen. This study is a novel attempt to offer evidence of the fit of the adult psychopathology model in terms of internalizing-externalizing forms using a psychiatric sample from a developing society. The study provides original information on MCMI in the clinical population using exploratory factor analysis (EFA) and confirmatory factor analysis (CFA).

Theoretically, this broadened the conceptualization of internalizing psychopathology, which is in line with the theories put out by Krueger et al. [32], who believe that the internalizing model will likely be tested with a variety of mental diseases. The research expanded the structure of internalizing psychopathology to capture simultaneously ten personality disorders and eight clinical syndromes. It is consistent with the ideal vision that new research would expand the scope of the structure of internalizing-externalizing psychopathology. Practically, the evaluation of psychopathology according to the internalizing-externalizing model in the developing country is considered a modern conceptual framework for adult psychopathology, which is more consistent with the recent DSM-5. There is no more demarcation between axis I (clinical syndromes) and axis II (personality disorders) as in DSM-IV. Psychotherapy as an add-on treatment to pharmacotherapy [98] is more likely to benefit from this new nosology.

Although the results of this study are significant, certain limitations should be addressed for future research. The first limitation is that the important findings from this study were based on correlational analyses, relying on participants’ self-report of psychopathology measures. Therefore, developmental longitudinal studies are needed to verify the findings reported here. Moreover, although the MCMI with the two-factor model, internalizing and externalizing psychopathology, was evident by its construct validation, replication of this study is needed, using different methodologies such as multitrait-multimethod [82, 83, 99] (e.g., self-reports, clinical observations, and interviews) and different samples (e.g., prisoners, patients, and community). Another cautionary note is that the sample was only from outpatients from the Hospital of Taiz City and two private psychiatry clinics in Yemen; hence, there are limitations on generalization and clinical implications. Future study is suggested using a bigger sample size and comparing the fit indices for the evidence of validity between the four-factor model (if possible after EFA) and the two-factor model to better explain the culture-sensitive item.

4.2. Implications and Recommendations

The MCMI is defined theoretically by internalizing-externalizing forms of psychopathology in clinical populations, which was significantly characterized by construct validity. Therefore, clinicians, clinical psychologists, and researchers can assess psychopathology/mental disorders and personality disorders using MCMI in clinical only; for nonclinical populations, an external validity study is needed in future research. In addition to the Mooney Problem Checklist of Measurement which is usually used in developing country educational settings [100], MCMI may be considered in the future. Moreover, MCMI is a clinically accurate assessment for diagnosing mental disorders. It can be used in treatment to track changes over time by looking at patients’ scores on MCMI-scale elevations.

5. Conclusion

This study provides evidence of model fit data of the adult psychopathology two-factor internalizing and externalizing using psychiatric samples from a developing country. More significantly, this study expanded the structure of internalizing psychopathology to identify ten personality disorders and eight clinical syndromes. It is consistent with the ideal vision that new research would expand the scope of the structure of internalizing and externalizing psychopathology. Concisely, it is a novel conceptual framework for adult psychopathology regarding internalizing-externalizing forms.

Data Availability

The data used in the current study are confidential and cannot be publicly shared, and this was also stated in the participants’ consent. However, it is available from the first or corresponding author on reasonable request.

Ethical Approval

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Research Ethics and Supervisory Committee of the Kulliyyah of Education, International Islamic University Malaysia (IIUM) with reference number IIUM/312/RNP/C/12/2 and date 26/3/2018.

All participants in this research voluntarily gave their informed consent to participate in this research.

Disclosure

A preprint of an earlier version of this manuscript has previously been published in Research Square [101].

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Authors’ Contributions

N.A.A., S.A.H., and M.S.N. have conceptualized the theoretical parts of the study, including theories and previous studies. N.A.A., S.A.H., N.M.A., and M.A.A. have formulated the section on Methodology. N.A.A., E.M.E.K., and M.S.N. participated in analyzing the results and statistical analysis. N.A.A., S.A.H., and E.M.E.K participated in the preparation of the discussion. N.A.A., S.A.H, M.A.A, and L.A.M contributed to drafting the paper and revising it critically to improve its important intellectual content. All authors read and approved the final manuscript.

Acknowledgments

The authors acknowledge the outpatients from the Hospital of Taiz City and two private psychiatry clinics in Yemen for their participation in this study.

References

T. M. Achenbach, “The classification of children's psychiatric symptoms: a factor-analytic study,” Psychological Monographs: General and Applied, vol. 80, no. 7, pp. 1–37, 1966.
View at: Publisher Site | Google Scholar
T. M. Achenbach and C. S. Edelbrock, “Psychopathology of childhood,” Annual Review of Psychology, vol. 35, no. 1, pp. 227–256, 1984.
View at: Publisher Site | Google Scholar
J. L. Tackett, “Toward an externalizing spectrum in DSM–V: incorporating developmental concerns,” Child Development Perspectives, vol. 4, no. 3, pp. 161–167, 2010.
View at: Publisher Site | Google Scholar
R. F. Krueger, “The structure of common mental disorders,” Archives of General Psychiatry, vol. 56, no. 10, pp. 921–926, 1999.
View at: Publisher Site | Google Scholar
G. Andrews, D. Goldberg, R. Krueger et al., “Exploring the feasibility of a meta-structure for DSM-V and ICD-11: could it improve utility and validity?” Psychological Medicine, vol. 39, no. 12, pp. 1993–2000, 2009.
View at: Publisher Site | Google Scholar
R. F. Krueger and S. C. South, “Externalizing disorders: cluster 5 of the proposed meta-structure for DSM-V and ICD-11,” Psychological Medicine, vol. 39, no. 12, pp. 2061–2070, 2009.
View at: Publisher Site | Google Scholar
N. R. Eaton, C. Rodriguez-Seijas, N. Carragher, and R. F. Krueger, “Transdiagnostic factors of psychopathology and substance use disorders: a review,” Social Psychiatry and Psychiatric Epidemiology, vol. 50, no. 2, pp. 171–182, 2015.
View at: Publisher Site | Google Scholar
N. R. Eaton, C. Rodriguez-Seijas, R. F. Krueger, W. K. Campbell, B. F. Grant, and D. S. Hasin, “Narcissistic personality disorder and the structure of common mental disorders,” Journal of Personality Disorders, vol. 31, no. 4, pp. 449–461, 2017.
View at: Publisher Site | Google Scholar
R. F. Krueger, A. Caspi, T. E. Moffitt, and P. A. Silva, “The structure and stability of common mental disorders (DSM-III-R): a longitudinal-epidemiological study,” Journal of Abnormal Psychology, vol. 107, no. 2, pp. 216–227, 1998.
View at: Publisher Site | Google Scholar
R. F. Krueger, Y. E. Chentsova-Dutton, K. E. Markon, D. Goldberg, and J. Ormel, “A cross-cultural study of the structure of comorbidity among common psychopathological syndromes in the general health care setting,” Journal of Abnormal Psychology, vol. 112, no. 3, pp. 437–447, 2003.
View at: Publisher Site | Google Scholar
R. F. Krueger, J. L. Tackett, and A. MacDonald, “Toward validation of a structural approach to conceptualizing psychopathology: a special section of the Journal of Abnormal Psychology,” Journal of Abnormal Psychology, vol. 125, no. 8, pp. 1023–1026, 2016.
View at: Publisher Site | Google Scholar
K. E. Markon, “Modeling psychopathology structure: a symptom-level analysis of axis I and II disorders,” Psychological Medicine, vol. 40, no. 2, pp. 273–288, 2010.
View at: Publisher Site | Google Scholar
M. W. Miller, J. L. Greif, and A. A. Smith, “Multidimensional personality questionnaire profiles of veterans with traumatic combat exposure: externalizing and internalizing subtypes,” Psychological Assessment, vol. 15, no. 2, pp. 205–215, 2003.
View at: Publisher Site | Google Scholar
M. W. Miller, D. G. Kaloupek, A. L. Dillon, and T. M. Keane, “Externalizing and internalizing subtypes of combat-related PTSD: a replication and extension using the PSY-5 scales,” Journal of Abnormal Psychology, vol. 113, no. 4, pp. 636–645, 2004.
View at: Publisher Site | Google Scholar
J. K. Rielage, T. Hoyt, and K. Renshaw, “Internalizing and externalizing personality styles and psychopathology in OEF–OIF veterans,” Journal of Traumatic Stress, vol. 23, no. 3, pp. 350–357, 2010.
View at: Publisher Site | Google Scholar
J. P. Rushton and P. Irwing, “A General Factor of Personality in the Millon Clinical Multiaxial Inventory- III, the Dimensional Assessment of Personality Pathology, and the Personality Assessment Inventory,” Journal of Research in Personality, vol. 43, no. 6, pp. 1091–1095, 2009.
View at: Publisher Site | Google Scholar
T. Slade and D. Watson, “The structure of common DSM-IV and ICD-10 mental disorders in the Australian general population,” Psychological Medicine, vol. 36, no. 11, pp. 1593–1600, 2006.
View at: Publisher Site | Google Scholar
E. Verona, N. Sachs-Ericsson, and T. E. Joiner, “Suicide attempts associated with externalizing psychopathology in an epidemiological sample,” American Journal of Psychiatry, vol. 161, no. 3, pp. 444–451, 2004.
View at: Publisher Site | Google Scholar
W. A. Vollebergh, J. Iedema, R. V. Bijl, R. de Graaf, F. Smit, and J. Ormel, “The structure and stability of common mental disorders,” Archives of General Psychiatry, vol. 58, no. 6, pp. 597–603, 2001.
View at: Publisher Site | Google Scholar
A. Wright, A. M. Beltz, K. M. Gates, P. Molenaar, and L. J. Simms, “Examining the dynamic structure of daily internalizing and externalizing behavior at multiple levels of analysis,” Frontiers in Psychology, vol. 6, pp. 1914–1914, 2015.
View at: Publisher Site | Google Scholar
N. Carragher, R. F. Krueger, N. R. Eaton, and T. Slade, “Disorders without borders: current and future directions in the meta-structure of mental disorders,” Social Psychiatry and Psychiatric Epidemiology, vol. 50, no. 3, pp. 339–350, 2015.
View at: Publisher Site | Google Scholar
N. R. Eaton, R. F. Krueger, and T. F. Oltmanns, “Aging and the structure and long-term stability of the internalizing spectrum of personality and psychopathology,” Psychology and Aging, vol. 26, no. 4, pp. 987–993, 2011.
View at: Publisher Site | Google Scholar
M. K. Forbes, J. L. Tackett, K. E. Markon, and R. F. Krueger, “Beyond comorbidity: toward a dimensional and hierarchical approach to understanding psychopathology across the life span,” Development and Psychopathology, vol. 28, no. 4part1, pp. 971–986, 2016.
View at: Publisher Site | Google Scholar
R. F. Krueger and K. E. Markon, “Reinterpreting comorbidity: a model-based approach to understanding and classifying psychopathology,” Annual Review of Clinical Psychology, vol. 2, no. 1, pp. 111–133, 2006.
View at: Publisher Site | Google Scholar
N. R. Eaton, K. M. Keyes, R. F. Krueger et al., “An invariant dimensional liability model of gender differences in mental disorder prevalence: evidence from a national sample,” Journal of Abnormal Psychology, vol. 121, no. 1, pp. 282–288, 2012.
View at: Publisher Site | Google Scholar
C. J. Hopwood and J. S. Moser, “Personality assessment inventory internalizing and externalizing structure in college students: invariance across sex and ethnicity,” Personality and Individual Differences, vol. 50, no. 1, pp. 116–119, 2011.
View at: Publisher Site | Google Scholar
R. F. Krueger, M. McGue, and W. G. Iacono, “The higher-order structure of common DSM mental disorders: internalization, externalization, and their connections to personality,” Personality and Individual Differences, vol. 30, no. 7, pp. 1245–1259, 2001.
View at: Publisher Site | Google Scholar
K. S. Kendler, C. A. Prescott, J. Myers, and M. C. Neale, “The structure of genetic and environmental risk factors for common psychiatric and substance use disorders in men and women,” Archives of General Psychiatry, vol. 60, no. 9, pp. 929–937, 2003.
View at: Publisher Site | Google Scholar
K. E. Markon, R. F. Krueger, T. J. Bouchard Jr., and I. I. Gottesman, “Normal and abnormal personality traits: evidence for genetic and environmental relationships in the Minnesota study of twins reared apart,” Journal of Personality, vol. 70, no. 5, pp. 661–693, 2002.
View at: Google Scholar
M. K. Forbes, A. J. Baillie, N. R. Eaton, and R. F. Krueger, “A place for sexual dysfunctions in an empirical taxonomy of psychopathology,” The Journal of Sex Research, vol. 54, no. 4-5, pp. 465–485, 2017.
View at: Publisher Site | Google Scholar
B. M. Hicks, R. F. Krueger, W. G. Iacono, M. Mcgue, and C. J. Patrick, “Family transmission and heritability of externalizing disorders: a twin-family study,” Archives of General Psychiatry, vol. 61, no. 9, pp. 922–928, 2004.
View at: Publisher Site | Google Scholar
R. F. Krueger, B. M. Hicks, C. J. Patrick, S. R. Carlson, W. G. Iacono, and M. Mcgue, Etiologic Connections among Substance Dependence, Antisocial Behavior, and Personality: Modeling the Externalizing Spectrum, DC, US, American Psychological Association, Washington, 2009.
R. Kotov, S.-W. Chang, L. J. Fochtmann et al., “Schizophrenia in the internalizing-externalizing framework: a third dimension?” Schizophrenia Bulletin, vol. 37, no. 6, pp. 1168–1178, 2011.
View at: Publisher Site | Google Scholar
T. M. Achenbach, M. Y. Ivanova, L. A. Rescorla, L. V. Turner, and R. R. Althoff, “Internalizing/externalizing problems: review and recommendations for clinical and research applications,” Journal of the American Academy of Child & Adolescent Psychiatry, vol. 55, no. 8, pp. 647–656, 2016.
View at: Publisher Site | Google Scholar
R. Kotov, R. F. Krueger, D. Watson et al., “The hierarchical taxonomy of psychopathology (HiTOP): a dimensional alternative to traditional nosologies,” Journal of Abnormal Psychology, vol. 126, no. 4, pp. 454–477, 2017.
View at: Publisher Site | Google Scholar
R. Kotov, R. F. Krueger, D. Watson et al., “The hierarchical taxonomy of psychopathology (HiTOP): a quantitative nosology based on consensus of evidence,” Annual Review of Clinical Psychology, vol. 17, no. 1, pp. 83–108, 2021.
View at: Publisher Site | Google Scholar
R. Kotov, C. J. Ruggero, R. F. Krueger, D. Watson, Q. Yuan, and M. Zimmerman, “New dimensions in the quantitative classification of mental illness,” Archives of General Psychiatry, vol. 68, no. 10, pp. 1003–1011, 2011.
View at: Publisher Site | Google Scholar
R. F. Krueger, K. A. Hobbs, C. C. Conway et al., “Validity and utility of hierarchical taxonomy of psychopathology (HiTOP): II. Externalizing superspectrum,” World Psychiatry, vol. 20, no. 2, pp. 171–193, 2021.
View at: Publisher Site | Google Scholar
K. T. Forbush and D. Watson, “The structure of common and uncommon mental disorders,” Psychological Medicine, vol. 43, no. 1, pp. 97–108, 2013.
View at: Publisher Site | Google Scholar
R. Gomez and P. J. Corr, “ADHD and personality: a meta-analytic review,” Clinical Psychology Review, vol. 34, no. 5, pp. 376–388, 2014.
View at: Publisher Site | Google Scholar
K. Herzhoff and J. L. Tackett, “Subfactors of oppositional defiant disorder: converging evidence from structural and latent class analyses,” Journal of Child Psychology and Psychiatry, vol. 57, no. 1, pp. 18–29, 2016.
View at: Publisher Site | Google Scholar
S. E. Jones, J. D. Miller, and D. R. Lynam, “Personality, antisocial behavior, and aggression: a meta-analytic review,” Journal of Criminal Justice, vol. 39, no. 4, pp. 329–337, 2011.
View at: Publisher Site | Google Scholar
A. G. Wright and L. J. Simms, “A metastructural model of mental disorders and pathological personality traits,” Psychological Medicine, vol. 45, no. 11, pp. 2309–2319, 2015.
View at: Publisher Site | Google Scholar
T. Millon, C. Millon, R. D. Davis, and S. D. Grossman, MCMI-III Manual, Pearson Education, Inc, Minneapolis, MN, 2006.
K. S. Kendler, C. G. Davis, and R. C. Kessler, “The familial aggregation of common psychiatric and substance use disorders in the National Comorbidity Survey: a family history study,” The British Journal of Psychiatry, vol. 170, no. 6, pp. 541–548, 1997.
View at: Publisher Site | Google Scholar
J. B. Hoelzle and G. J. Meyer, “The factor structure of the MMPI–2 restructured clinical (RC) scales,” Journal of Personality Assessment, vol. 90, no. 5, pp. 443–455, 2008.
View at: Publisher Site | Google Scholar
T. Millon, MCMI-III manual: Millon Clinical Multiaxial Inventory--III, National Computer Systems, Minneapolis. MN, 1994.
M. T. Boccaccini and S. L. Brodsky, “Attorney–client trust among convicted criminal defendants: preliminary examination of the attorney–client trust scale,” Behavioral Sciences & the Law, vol. 20, no. 1-2, pp. 69–87, 2002.
View at: Publisher Site | Google Scholar
R. J. Craig, “Essentials of MCMI-III assessment,” in Essentials of Millon inventories assessment, pp. 1–51, Wiley New York, 2002.
View at: Google Scholar
R. J. Craig, New Directions in Interpreting the Millon Clinical Multiaxial Inventory-III (MCMI-III), John Wiley & Sons, 2005.
M. R. Schoenberg, D. Dorr, and C. D. Morgan, “Development of discriminant functions to detect dissimulation for the Millon clinical multiaxial inventory (3rd edition),” The Journal of Forensic Psychiatry & Psychology, vol. 17, no. 3, pp. 405–416, 2006.
View at: Publisher Site | Google Scholar
T. A. Brown, Confirmatory Factor Analysis for Applied Research, Guilford Publications, 2014.
N. A. Alareqe, S. Roslan, M. S. Nordin, N. A. Ahmad, and S. M. Taresh, “Psychometric properties of the Millon clinical multiaxial inventory-III in an Arabic clinical sample compared with American, Italian, and Dutch cultures,” Frontiers in Psychology, vol. 12, pp. 562619–562619, 2021.
View at: Publisher Site | Google Scholar
R. J. Craig and A. Bivens, “Factor structure of the MCMI-III,” Journal of Personality Assessment, vol. 70, no. 1, pp. 190–196, 1998.
View at: Publisher Site | Google Scholar
J. B. Hoelzle and G. J. Meyer, “The invariant component structure of the personality assessment inventory (PAI) full scales,” Journal of Personality Assessment, vol. 91, no. 2, pp. 175–186, 2009.
View at: Publisher Site | Google Scholar
C. J. Hopwood, K. L. Baker, and L. C. Morey, “Personality and drugs of choice,” Personality and Individual Differences, vol. 44, no. 6, pp. 1413–1421, 2008.
View at: Publisher Site | Google Scholar
L. C. Morey, Personality Assessment Inventory (PAI), Wiley Online Library, 1999.
G. Rossi, L. A. van der Ark, and H. Sloore, “Factor analysis of the Dutch-language version of the MCMI–III,” Journal of Personality Assessment, vol. 88, no. 2, pp. 144–157, 2007.
View at: Publisher Site | Google Scholar
L. Cuevas, L. F. García, A. Aluja, and Ó. García, “Exploratory and confirmatory factorial structure of the MCMI-III personality disorders: overlapping versus non-overlapping scales,” The European Journal of Psychiatry, vol. 22, pp. 59–68, 2008.
View at: Google Scholar
G. Rossi, A. Elklit, and E. Simonsen, “Empirical evidence for a four factor framework of personality disorder organization: multigroup confirmatory factor analysis of the Millon clinical multiaxial inventory—III personality disorder scales across Belgian and Danish data samples,” Journal of Personality Disorders, vol. 24, no. 1, pp. 128–150, 2010.
View at: Publisher Site | Google Scholar
M. A. Ruiz and J. F. Edens, “Recovery and replication of internalizing and externalizing dimensions within the personality assessment inventory,” Journal of Personality Assessment, vol. 90, no. 6, pp. 585–592, 2008.
View at: Publisher Site | Google Scholar
M. A. Aljaberi, N. A. Alareqe, A. Alsalahi et al., “A cross-sectional study on the impact of the COVID-19 pandemic on psychological outcomes: multiple indicators and multiple causes modeling,” PLoS One, vol. 17, no. 11, article e0277368, 2022.
View at: Publisher Site | Google Scholar
J. Hair, W. Black, B. Babin, and R. Anderson, Multivariate Data Analysis, Prentice Hall, New Jersey, 2019.
R. B. Kline, Principles and practice of structural equation modeling, Guilford Publications, 2015.
P. S. Wang, M. Lane, M. Olfson, H. A. Pincus, K. B. Wells, and R. C. Kessler, “Twelve-month use of mental health services in the United States,” Archives of General Psychiatry, vol. 62, no. 6, pp. 629–640, 2005.
View at: Publisher Site | Google Scholar
R. Bruffaerts, P. Mortier, R. P. Auerbach et al., “Lifetime and 12-month treatment for mental disorders and suicidal thoughts and behaviors among first year college students,” International Journal of Methods in Psychiatric Research, vol. 28, no. 2, article e1764, 2019.
View at: Publisher Site | Google Scholar
E. G. Karam, Z. N. Mneimneh, A. N. Karam et al., “Prevalence and treatment of mental disorders in Lebanon: a national epidemiological survey,” The Lancet, vol. 367, no. 9515, pp. 1000–1006, 2006.
View at: Publisher Site | Google Scholar
D. L. Jackson, J. Voth, and M. P. Frey, “A note on sample size and solution propriety for confirmatory factor analytic models,” Structural Equation Modeling: A Multidisciplinary Journal, vol. 20, no. 1, pp. 86–97, 2013.
View at: Publisher Site | Google Scholar
S. Arafat, H. R. Chowdhury, M. Qusar, and M. Hafez, “Cross cultural adaptation & psychometric validation of research instruments: a methodological review,” Journal of Behavioral Health, vol. 5, no. 3, pp. 129–136, 2016.
View at: Publisher Site | Google Scholar
R. W. Brislin, “The wording and translation of research instruments,” in Field Methods in Cross-Cultural Research, Sage Publications, Inc, Thousand Oaks, CA, US, 1986.
View at: Google Scholar
D. W. Chapman and J. F. Carter, “Translation procedures for the cross cultural use of measurement instruments,” Educational Evaluation and Policy Analysis, vol. 1, no. 3, pp. 71–76, 1979.
View at: Publisher Site | Google Scholar
S. Noman, H. K. Shahar, H. A. Rahman, S. Ismail, M. A. Aljaberi, and M. N. Abdulrahman, “Factor structure and internal reliability of breast cancer screening champion’s health belief model scale in Yemeni women in Malaysia: a cross-sectional study,” BMC Women's Health, vol. 21, no. 1, p. 437, 2021.
View at: Publisher Site | Google Scholar
L. A. Clark and D. Watson, “Constructing validity: new developments in creating objective measuring instruments,” Psychological Assessment, vol. 31, no. 12, pp. 1412–1427, 2019.
View at: Publisher Site | Google Scholar
Z. E. Fares, B. Ala'a, H. F. Gadelrab et al., “Arabic COVID-19 psychological distress scale: development and initial validation,” BMJ Open, vol. 11, no. 6, article e046006, 2021.
View at: Publisher Site | Google Scholar
M. K. Forbes, A. L. Greene, H. F. Levin-Aspenson et al., “Three recommendations based on a comparison of the reliability and validity of the predominant models used in research on the empirical structure of psychopathology,” Journal of Abnormal Psychology, vol. 130, no. 3, pp. 297–317, 2021.
View at: Publisher Site | Google Scholar
M. Sellbom and A. Tellegen, “Factor analysis in psychological assessment research: common pitfalls and recommendations,” Psychological Assessment, vol. 31, no. 12, pp. 1428–1441, 2019.
View at: Publisher Site | Google Scholar
J. Hair, W. Black, B. Babin, and R. Anderson, Multivariate Data Analysis, Prentice Hall, New Jersey, 2010.
M. A. Aljaberi, K.-H. Lee, N. A. Alareqe et al., “Rasch modeling and multilevel confirmatory factor analysis for the usability of the impact of event scale-revised (IES-R) during the COVID-19 pandemic,” Healthcare, vol. 10, no. 10, p. 1858, 2022.
View at: Publisher Site | Google Scholar
S. Hassan and K. A. Bakar, “Exploratory factor analysis versus confirmatory factor analysis,” in Teachers’ Learning, Curriculum Innovations and Knowledge Applications, F. A. Rahman, F. Shafiee, and H. Elias, Eds., UPM Press, Malaysia, 2008.
View at: Google Scholar
M. F. Nasab, R. Abdul Kadir, S. A. Hassan, and S. Mohd. Noah, “Psychometric evaluation of the career decision scale with Iranian undergraduate students,” Journal of Counseling & Development, vol. 93, no. 3, pp. 344–351, 2015.
View at: Publisher Site | Google Scholar
M. U. H. Uzir, H. Al Halbusi, R. Thurasamy et al., “The effects of service quality, perceived value and trust in home delivery service personnel on customer satisfaction: evidence from a developing country,” Journal of Retailing and Consumer Services, vol. 63, article 102721, 2021.
View at: Publisher Site | Google Scholar
P. M. Bentler, “Comparative fit indexes in structural models,” Psychological Bulletin, vol. 107, no. 2, pp. 238–246, 1990.
View at: Publisher Site | Google Scholar
C. Distefano and B. Hess, “Using confirmatory factor analysis for construct validation: an empirical review,” Journal of Psychoeducational Assessment, vol. 23, no. 3, pp. 225–241, 2005.
View at: Publisher Site | Google Scholar
B. M. Byrne, Structural Equation Modeling with AMOS: Basic Concepts, Applications, and Programming, Routledge, New York, 2016.
View at: Publisher Site
K. Bollen, “Structural equations with latent variables,” Journal of Korean Academy of Nursing, vol. 29, 1989.
View at: Google Scholar
F. Z. E. Abiddine, M. A. Aljaberi, H. F. Gadelrab, C.-Y. Lin, and A. Muhammed, “Mediated effects of insomnia in the association between problematic social media use and subjective well-being among university students during COVID-19 pandemic,” Sleep Epidemiology, vol. 2, article 100030, 2022.
View at: Publisher Site | Google Scholar
M. A. Aljaberi, M. H. Juni, R. A. Al-Maqtari et al., “Relationships among perceived quality of healthcare services, satisfaction and behavioural intentions of international students in Kuala Lumpur, Malaysia: a cross-sectional study,” BMJ Open, vol. 8, no. 9, article e021180, 2018.
View at: Publisher Site | Google Scholar
C. Fornell and D. F. Larcker, “Evaluating structural equation models with unobservable variables and measurement error,” Journal of Marketing Research, vol. 18, no. 1, pp. 39–50, 1981.
View at: Publisher Site | Google Scholar
L. A. Mohammed, M. A. Aljaberi, A. Amidi et al., “Exploring factors affecting graduate Students’ satisfaction toward E-learning in the era of the COVID-19 crisis,” European Journal of Investigation in Health, Psychology and Education, vol. 12, no. 8, pp. 1121–1142, 2022.
View at: Publisher Site | Google Scholar
M. Rönkkö and E. Cho, “An updated guideline for assessing discriminant validity,” Organizational Research Methods, vol. 25, no. 1, pp. 6–14, 2022.
View at: Publisher Site | Google Scholar
E. J. Pedhazur and L. P. Schmelkin, Measurement, Design, and Analysis: An Integrated Approach, Psychology Press, 1991.
A. M. Farrell, “Insufficient discriminant validity: a comment on Bove, Pervan, Beatty, and Shiu (2009),” Journal of Business Research, vol. 63, no. 3, pp. 324–327, 2010.
View at: Publisher Site | Google Scholar
J. Henseler, C. M. Ringle, and M. Sarstedt, “A new criterion for assessing discriminant validity in variance-based structural equation modeling,” Journal of the Academy of Marketing Science, vol. 43, no. 1, pp. 115–135, 2015.
View at: Publisher Site | Google Scholar
D. M. Blonigen, C. J. Patrick, K. S. Douglas et al., “Multimethod assessment of psychopathy in relation to factors of internalizing and externalizing from the personality assessment inventory: the impact of method variance and suppressor effects,” Psychological Assessment, vol. 22, no. 1, pp. 96–107, 2010.
View at: Publisher Site | Google Scholar
E. Røysamb, K. S. Kendler, K. Tambs et al., “The joint structure of DSM-IV axis I and axis II disorders,” Journal of Abnormal Psychology, vol. 120, no. 1, pp. 198–209, 2011.
View at: Publisher Site | Google Scholar
R. F. Krueger, B. M. Hicks, C. J. Patrick, S. R. Carlson, W. G. Iacono, and M. McGue, “Etiologic connections among substance dependence, antisocial behavior and personality: modeling the externalizing spectrum,” Journal of Abnormal Psychology, vol. 111, no. 3, pp. 411–424, 2002.
View at: Publisher Site | Google Scholar
R. F. Krueger and J. L. Tackett, “Personality and psychopathology: working toward the bigger picture,” Journal of Personality Disorders, vol. 17, no. 2, pp. 109–128, 2003.
View at: Publisher Site | Google Scholar
T. L. Pheng and S. A. Hassan, “A review of psychotherapy as add-on treatment to pharmacotherapy for bipolar disorder,” Research Journal of Medical Science, vol. 8, pp. 99–108, 2014.
View at: Google Scholar
D. T. Campbell and D. W. Fiske, “Convergent and discriminant validation by the multitrait-multimethod matrix,” Psychological Bulletin, vol. 56, no. 2, pp. 81–105, 1959.
View at: Publisher Site | Google Scholar
O. Jailani, A. H. T. Adli, M. A. C. Amat, S. M. Othman, N. Deylami, and N. S. A. Rahim, “The self-perceived problems among Malaysian pre-university students: implications for college counselling,” Asian Journal of University Education, vol. 16, no. 3, pp. 112–124, 2020.
View at: Publisher Site | Google Scholar
N. A. Alareqe, S. A. Hassan, E. M. E. Kamarudin et al., Validity of Adult Psychopathology Model Using Psychiatric Patients Sample from a Developing Country: Confirmatory Factor Analysis, Research Square, 2021.
View at: Publisher Site

Copyright

Copyright © 2022 Naser Abdulhafeeth Alareqe et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

1658

Downloads

404

Citations