Abstract

The home questionnaire of the Progress in International Reading Literacy Study (PIRLS-HQ 2011) was designed to gather information from parents or primary caregivers of fourth-grade pupils on their reading literacy development related to aspects of pupils’ home lives across countries/districts. The questionnaire was translated into different languages for international comparison and research purposes. This study aims to assess the psychometric properties of the Chinese version of the PIRLS 2011 home questionnaire (PIRLS-HQCV 2011) and identify the underlying factor structure using exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) among Chinese fourth-grade pupils in Hong Kong. A 7-factor structure model has been identified by EFA and confirmed to resemble much to the original PIRLS structure by CFA. Additional conceptually important domains have been identified which add further insights into the inconclusive results in the literature regarding the relationship between home factors and reading achievement. Implications for further studies are discussed.

1. Introduction

It has long been known that students’ home background affects their reading skills. The influence of students’ home background on school achievement has been of interest to researchers for several decades [18]. Given the importance of reading and the goal of helping countries make informed decisions on ways to improve teaching and learning in reading, the Progress in International Reading Literacy Study (PIRLS) was established by the International Association for the Evaluation of Educational Achievement (IEA) to study the reading development of primary school students. Reading literacy is defined by PIRLS as “the ability to understand and use those written language forms required by society and valued by the individual. Young readers read to learn, to participate in communities of readers, and to enjoy things around them” [9].

Compared to PIRLS 2001 and 2006, Hong Kong raised her levels of reading achievement in 2011 and was the top-performing city among 49 participating countries/regions, with a mean score of 571 [10]. This remarkable jump in Hong Kong, as a city with a Chinese culture, has attracted much attention from researchers and educators all over the world. PIRLS examined four key factors affecting the reading performance which included “school,” “teacher,” “parent,” and “student” [11]. Among these four questionnaires, home factor was identified in this study because early literacy experiences at home would establish a foundation for learning long before children develop the cognitive and linguistic skills necessary for reading [1, 12, 13]. It is well documented that early literacy experiences are critical for later academic development [14]. In addition, home factor has been shown to be both directly and indirectly related to students’ reading achievement [15]. We identified home factors which have significantly contributed to this remarkable improvement in this study and are briefly reviewed here. Other factors have been and will be discussed elsewhere.

Parents are regarded as the best persons to help their children cultivate a good reading habit at an early age and build up a good reading environment at home [16]. The fourth grade is an important transition point in children’s development as readers, because at this stage most pupils should have learned to read and are now reading to learn [9]. Research on emergent literacy has recently highlighted the importance of a supportive home environment in developing children’s reading skills in the English-speaking societies [17, 18] and Asian context [19, 20]. Early home literacy activities will facilitate later literacy development [2124]. In contrast, lack of home literacy may become a vicious cycle of low literacy [25]. Sylva et al. demonstrated that home learning environment had significant positive effects on children’s prereading and language development [26].

While there has been much interest in the research on the association between home literacy practices and literacy in the west, attention paid to reading in nonalphabetic systems is scarce [18]. Given the differences and peculiarities of the Chinese language [2729], we wanted to gain more understanding if home literacy activities yielded the same impact as that of the West given that current knowledge on the differences between Chinese and English in this aspect is limited. This study therefore aims to assess the psychometric properties of the Chinese version of the home questionnaire (PIRLS-HQCV 2011) and identify the underlying factor structure using exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) among Chinese fourth-grade pupils in Hong Kong.

Although home factors are significantly related to reading, a review of the literature suggests that validated questionnaires of similar kind, especially for use in Chinese societies, are extremely limited [15]. The home questionnaire (PIRLS-HQ 2011) designed and compiled by PIRLS was used to gather information from children’s parents or primary caregivers of each child taking part in PIRLS2011 on their reading literacy development related to aspects of children’s home lives. The PIRLS-HQ 2011 asked economic, social, and educational resources at home, parents’ emphasis on children’s literacy development, and their reading behaviours and attitudes. The majority of the questionnaires were carried forward from the previous assessment cycle to measure trends. The content of the questionnaire was first reviewed by the National Research Coordinators (NRCs) from different countries by item and possible amendments were recommended. The updated drafts of questionnaires were then reviewed by the eight-member PIRLS 2011 Questionnaire Development Group (QDG) who believed that students’ reading behaviours and attitudes contributed to the full realization of the individual’s potential within a literate society. A substantial number of the items were designed to address parents’ attitudes toward reading and reading habits. This expert committee reviewed each questionnaire item for clarity and examined the data to make sure that the options provided useful information. The QDG also made recommendations so that all items and scales remaining in the questionnaire were useful for further analyses while items and scales that were found to be less useful were eliminated. The members also suggested improvements and provided additional new items as necessary to meet the goals of the upcoming assessment. Representatives from the IEA Data Processing and Research Center (DPC) met and ensured that the recommendations were amenable to efficient and error-free data collection. The updating of the PIRLS-HQ 2011 started in February 2008. The PIRLS-HQ 2011 consisted of two modules: a general module inquiring on parents or caregivers and their background, home literacy environment, parents’ attitudes toward school and reading, and the child’s numeracy and literacy skills. The PIRLS International Study Center prepared an international version of all updated questionnaires in English. Subsequently questionnaire instruments were translated by participating countries into their languages of instruction with the goal of creating high quality translations that are appropriately adapted for the national context and at the same time are internationally comparable. Therefore, a significant portion of the development and review effort of the Chinese version by NRC of Hong Kong was dedicated to ensuring that the questionnaires could be translated accurately. The NRC also checked any unusual national results that might be an indication of translation errors and corrected the translation as necessary or recommends revisions to accommodate translation.

The final version of the PIRLS-HQCV 2011 included demographic information, language spoken in the home, preschool experiences, homework activities, home-school involvement, books in the home, parents’ education and involvement, early literacy and numeracy abilities, reading and quantitative readiness, and parents’ reading activities and attitudes toward reading [9] (see Table 1). However, the factor structure and psychometric properties of the Chinese version of the home questionnaire have not yet been investigated. This study aims to assess the psychometric properties and explore the underlying factor structure of the home questionnaire using exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) among Chinese fourth-grade pupils in Hong Kong so that more detailed regression analysis and structure equation modelling may be done to explore the relationship between home factors and reading achievement at a later stage.

2. Method

2.1. Participants

The PIRLS 2011 followed the uniform sampling approach specified by the PIRLS National Research Coordinator (NRC) [30]. The probability proportional-to-size technique (PPS) [31] was used for school sampling and recruitment of participants. One hundred and thirty-two classes were randomly selected using this technique. A two-stage stratified cluster sampling was adopted. The first stage and the second stage composed of a sampling of schools and a sampling of intact classrooms from the target grade in the sampled schools, respectively. Schools were sampled using systematic random sampling with probability proportional to their measure of size (MOS). Within each sampled school, all fourth-grade classes were listed and one class was randomly sampled. One hundred and thirty-two primary schools located in the 18 districts of Hong Kong eventually took part in the PIRLS. There were 4,189 pupils sampled in participating schools. A total of 4,105 fourth-grade children’s parents or primary caregivers were eligible and thus recruited from 132 primary schools between March and May in the school year 2000/2011 in Hong Kong. Finally a total of 3,641 parents or caregivers completed the PIRLS-HQCV 2011. Among them, 71.3% of the questionnaires were completed by mothers, 22.1% were completed by fathers, 5.2% were completed jointly by both parents, and 1.4% were other caregivers. The average age of the student participants was 10.1 with 46% girls and 53% boys [10].

2.2. Measures

The PIRLS-HQCV 2011 was designed for completion by parents or primary caregivers of fourth-grade pupils. It consisted of a total of 21 questions with 13 questions on background information and 8 questions related to contextual factors stated in PIRLS 2011 assessment framework [9]. Background information on pupils’ preschool characteristics, books in the home, and parents’ characteristics were collected. Respondents were also asked to complete another 8 questions with 35 items comprising measurement of pupils’ performance on early literacy task (5 items), pupils’ performance on early numeracy task (4 items), homework activities (1 items), home-school involvement (8 items), parents’ involvement (8 items), reading and quantitative readiness (1 item), and parents’ reading activities and attitudes toward reading (8 items). While the items were grouped in eight questions in the questionnaire, they were considered as individual observations in the factor analyses. Items were presented as statements to be answered following a 4-point scale ranging from “not at all” to “very well.” Four items were scored in reverse, which were asterisk-marked in Table 2. The question items of the background information and contextual factors are listed in Table 1.

2.3. Procedure

Each sampled class was assigned a Test Administrator who followed the test procedures described in the Test Administrator Manual to distribute home questionnaire [32]. The Test Administrator was selected and trained by the School Coordinator using the Test Administrator Manual. The Test Administrator gave the home questionnaire to each pupil to take home to his or her parents or guardians with a consent letter. Parents or caregivers took 10 to 15 minutes to fill in the PIRLS-HQCV 2011 at home. A pupil identification code was used on each survey to ensure the linkage of the PIRLS-HQCV to the corresponding pupil’s reading assessment. The questionnaires were returned to the School Coordinator within one week of the testing date. If completed surveys were not returned, a follow-up telephone call was made.

2.4. Statistical Analyses

To follow usual practices [33], we randomly selected about one-third of the valid data to test the reliability of the questionnaire using Cronbach’s coefficient alpha and EFA () by Statistical Package for the Social Sciences (SPSS) version 20 and two-third of the valid data to test the construct validity of the questionnaire using CFA () by Amos version 21.0. Cronbach’s coefficient alpha was adopted to examine the internal consistencies of the overall scale and subscales from the questionnaire. The Kaiser-Meyer-Oklin value and Bartlett’s Test of Sphericity were used to test the suitability of the dataset for factor analysis. EFA was performed to identify a set of latent constructs underlying a battery of measured items. Fabrigar et al. suggested that maximum likelihood (ML) was the best extraction method if data were relatively normally distributed; principal axis factoring (PAF) method was recommended otherwise [34]. Promax with Kaiser Normalization was used for rotation given our assumption on the correlations among factors. In this study, factors with eigenvalue greater than 1 were extracted along the two methods separately (i.e., ML and PAF) with promax rotation (). The model was then tested by CFA based on previous empirical research on pupil’s reading achievement. Factors were allowed to be correlated and multiple goodness of fit tests including chi-squared statistics (), comparative fit index (CFI) [35], Tucher-Lewis index (TLI) [36], and the root mean squared error of approximation (RMSEA) with 90% confidence interval of RMSEA (CI) was used to evaluate the goodness of fit of the model. If the results of ML and PAF were the same at EFA, ML estimator was used to evaluate model fit at CFA. Satorra and Bentler robust correction was made to all model fit indices when necessary [37]. Values of TLI and CFI greater than 0.90 indicate a good fit to the data [38, 39]. Hu and Bentler have suggested a more conservative cut-off estimate of 0.95 for CFI [40]. A RMSEA values less than 0.05 indicated a good fit to the data while less than 0.08 indicated an acceptable fit to the data [41]. Based on the original structure of the questionnaire, the model was evaluated using the four-point Likert method (1–4). Descriptive and frequency statistics were used to summarize the demographic data of the participants (mean and SD) and the scales of the questionnaire. Subscale scores were computed from the average of the appropriate items.

3. Results

3.1. Factor Structure and Reliability of the Home Questionnaire

The data were found to be suitable for factor analysis by the Kaiser-Meyer-Oklin (KMO) value (0.85) and Barlett’s Test of Sphericity (0.000). A KMO value greater than 0.5 indicated sampling adequacy and suggested that EFA could be proceeded using the data set. EFA showed that 29 items of the 779 samples were categorized into 7 factors. This 7-factor solution accounted for 59.48% of the total variance. Factor loadings above .40 were considered acceptable. Based on the EFA result, 6 items below .40 were not included in the 7 factors and hence not considered for subsequent analyses. The item allocation was mainly based on the significant loadings from the EFA results and the nature of content. The seven factors were (1) parents’ engagement in children’s study, (2) performance on early literacy tasks, (3) parents like reading, (4) parents’ opinions on children’s progress, (5) performance on early numeracy tasks, (6) parents’ opinions of school involving parents, and (7) parents’ extrinsic motivation of reading.

All of the items were meaningfully interpreted in the assigned factors. Both the ML and PAF extraction methods with promax rotation yielded the same 7-factor solution with eigenvalue greater than 1. The finalized items and factor structure of the questionnaire using ML with promax rotation are shown in Table 2. The correlation between items and their corresponding factor was measured by Cronbach’s coefficients alpha. The internal consistency of the 7 factors/scales based on the EFA results was found to be satisfactory with a range from .60 to .91. The mean scores, standard deviation, and the coefficients alpha are summarized in Table 3.

3.2. Construct Validity of the Student Questionnaire

The 7-factor model was tested using CFA. The accompanying standardized path coefficients to the corresponding items are shown in Table 4. Examination of model fit indices revealed an acceptable fit to the data of the 7-factor model, with () = 3344.23, NFI = .904, CFI = .913, TLI = .894, and RMSEA = .054 (90% CI from .052 to .056). The standardized path coefficients of the factors to the corresponding items were all significant which ranged from .458 to .942. Correlations among the seven factors are shown in Table 5. Factor 2 “performance on early literacy tasks” was significantly correlated with all the other six factors. Factor 5 “performance on early numeracy tasks” was significantly correlated with all the other factors except factor 4 “parents’ opinions on children’s progress.” Factor 1 “parent’s engagement in children’s study” was significantly correlated with all the other factors except factor 7 “parents’ extrinsic motivation of reading.” Factor 6 “parents’ opinions of school involving parents” was correlated with all the other six factors. Factor 4 “parents’ opinions on children’s progress” was significantly correlated with all the other factors except factor 5 “performance on early numeracy tasks.” Factor 3 “parents like reading” was significantly correlated with all the other six factors. Factor 7 “parents’ extrinsic motivation of reading” was significantly correlated with all the other factors except factor 1 “parent’s engagement in children’s study.”

4. Discussion

Findings of the current study have provided compelling evidence as to the psychometric properties of the PIRLS-HQCV 2011. As a result, a number of implications can be made as to the use of the home questionnaire in assessing the underlying factors contribute to the Chinese student’s reading achievements. The study has had the obvious advantage of a large and representative sample size that facilitated the process of EFA and CFA. The most remarkable finding is that the 7-factor solution generated from the EFA was confirmed by the CFA. The Chinese version of the PIRLS-HQ was observed to have acceptable internal consistency for an educational assessment tool. The 7-factor solution resembles much to the original home questionnaire of the PIRLS 2011. Table 6 shows the factor structure following EFA and CFA of the PIRLS-HQCV 2011 and the original four PIRLS context questionnaire scales stated in the PIRLS 2011 assessment framework. Our findings of seven-factor solution revealed additional conceptually important domains which have not been described explicitly in the PIRLS 2011 assessment framework. Three context questionnaire scales had been mentioned in the PIRLS 2011 assessment framework, which included “Could do Early Literacy Tasks When Began Primary School Scale,” “Parents Like Reading Scale,” and “Early Literacy Activities before Beginning Primary School Scale.” Our factor analysis ascertained that these three factors aligned well with the original two scales but two factors aligned unequivocally with one of the original scales. These included factor 2 “performance on early literacy tasks” which aligned well with “Could do Early Literacy Tasks When Began Primary School Scale” and both factor 3 “parents like reading” and factor 7 “parents’ extrinsic motivation of reading” which aligned unequivocally with “Parents Like Reading Scale.” Our investigation on the correlation among each item under factors 3 and 7 allows us to observe that both items H13A and H13D constituted a significantly stronger correlation (, ) but very weak correlation with other items (H13B, H13C, H13E, H13F, and H13G). The correlation analysis further revealed that the two items under the same question (H13) were loaded on different factors. To follow the results of the EFA, the psychological concepts of both “intrinsic reading motivation” and “extrinsic reading motivation” should be considered separately in future studies when assessing the underlying factors which contribute to the Chinese student’s reading achievements. Our findings of the two psychological domains echo with the emerging literacy research that there exists two different concepts between parents’ intrinsic motivation and extrinsic motivation in promoting children’s reading motivation [42].

The results of EFA and CFA have derived four additional factors that were not explicitly described in the original PIRLS 2011 framework. These domains included factor 1 “parents’ engagement in children’s study,” factor 4 “parents’ opinions of school on children’s progress,” factor 5 “performance on early numeracy tasks,” and factor 6 “parents’ opinions of school involving parents.” These four domains reflected different concepts and have been recognized in recent studies [7, 4345].

This study has filled the gap for a validated instrument which may be used in Chinese communities including Hong Kong, Taiwan, Singapore, and mainland cities to assess factors pertaining to home and the relationship with reading achievement. Based on the results of EFA and CFA in the present study, the PIRLS-HQCV 2011 could be used as a validated tool for this specific research purpose. The 7-factor structure of this questionnaire basically fits with the PIRLS structure. The PIRLS-HQCV 2011 may be used for assessment of student factors of 4th grade pupils in Chinese societies. The factor constructs can be regarded as predictors of reading achievement in further studies using multiple regression or Structural Equation Modelling (SEM) to identify factors that may determine the reading accomplishment in reading. The findings will then add additional insight into the inconclusive results in the literature regarding the relationship between home factors and reading achievement. With minor revisions, the factor constructs show potentials to be used by educational researchers and teachers in Hong Kong and other Chinese societies such as Taiwan, Singapore, and mainland China. The results from the PIRLS-HQCV 2011 will in turn be used for comparison purposes with similar results obtained in the western world.

Although most of the results discussed above are positive in providing support to the psychometric properties of the scale, the study is by no means free from limitations. Although a two-stage stratified cluster sampling was adopted, the sample is not representative of all the 4th grade pupils in the international PIRLS. The results of EFA and CFA have both confirmed the 7-factor model for the PIRLS-HQ 2011. However, some of the factors were rather weak. These included factor 5 (performance on early numeracy tasks), factor 6 (parents’ opinions of school involving parents), and factor 7 (parents’ extrinsic motivation of reading). Each of these factors consisted only of two items which is the minimum for a particular factor. Item number H2 in the home questionnaire did not follow a 4-point scale and hence could not be used for further analysis even though it was considered as a scale of “Early literacy Activities Before Beginning Primary School Scale” in the original PIRLS assessment framework. Also, items 7A, 7B, 8, 10A, 10C, and 11 had factor loadings below .40 in the EFA result and could not be grouped under any of the seven factors. However, there are still insights coming out from the more negative findings. The lower factor loadings of items 7A and 7B suggested that counting and shape recognizing during early development were not of the same construct with items on numeracy activities although these two items were grouped under the same domain by PIRLS experts. The EFA results further suggested that numeracy activities are likely to be more closely related to reading competence of pupils. On the contrary, counting and shape recognition are less closely connected to reading. Further studies are needed in the future to explore the differential relationship between counting, shape recognition, and number recognition and writing on reading development. Both items 8 and 11 were related to time spent on homework and reading by pupils and parents, respectively. The lower factor loadings of these items may imply that frequency of doing homework and reading were not remarkably associated with academic accomplishment including reading although further studies are needed to further elucidate the relationship. Moreover, the low factor loading of item 10C indicated that parent’s opinion on school safety is not one of the home factors that may have close connection with the reading achievement. Finally, item 10A unexpectedly exhibited a low factor loading and did not contribute to factor 6 as similar as item 10B and 10E. The difference between items 10A as compared to 10B and 10E appeared to be on the use of “should” statement on the parent’s opinion of school involving parents in child’s education. However, the exact reason for the exclusion of 10A from factor 6 remains unknown. Further research is needed to gain more insight into this unexpected finding.

5. Conclusion

Although it is not perfect, the PIRLS-HQCV 2011 is still considered psychometrically acceptable based on the results of the current study. The seven factors pertaining to home environment may be used to explore the relationship with reading competence of pupils using the data of the PIRLS study. In addition, the development of the student questionnaire, namely, the Chinese version of the PIRLS 2011 Student Questionnaire (PIRLS-SQCV 2011), shared similar methodology as the home questionnaire and is reported elsewhere [46].

Competing Interests

The authors declared no potential competing interests with respect to the research, authorship, and/or publication of this paper.

Acknowledgments

The authors would like to acknowledge Ms. Christie Leung for her support in the initial data analysis process and Dr. Raymond Chung for his statistical advice. This study was supported by the General Research Fund from the Research Grants Council of Hong Kong under File no. 17606715, Hong Kong.