Abstract

The student questionnaire (PIRLS-SQ 2011) of the Progress in International Reading Literacy Study (PIRLS) was designed to gather information from pupils on reading literacy development as to aspects of pupils’ self-lives, home, and school lives across countries/districts. In order to serve the purposes of research and international comparison, the questionnaire was translated into various languages. Using exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), the current study investigates the psychometric properties of the Chinese version of the student questionnaire (PIRLS-SQCV 2011) and identifies its underlying factor structure among Chinese fourth-grade pupils in Hong Kong. A 10-factor structure model was identified and much resemblance could be drawn to the original PIRLS structure. While the similarity allows international comparisons of studies in different places following the PIRLS strategy, the findings of this study add to extant literature on the relationship between student factors and reading achievement.

1. Introduction

Reading is essential to learning in school which facilitates future activities in work and community and provides enjoyment from leisure reading [1]. The Progress in International Reading Literacy Study (PIRLS), the largest international reading achievement project, was first developed to explore the role of reading in educational achievement of fourth-grade pupils across 49 countries/districts in 2001. The PIRLS 2011 is the third cycle that includes a full complement of questionnaires to investigate the experiences of young children at home and school in learning to read. To assess culturally specific reading-learning model, Hong Kong has participated in this international study since the Curriculum Reform 2001, in which “Reading to Learn” is one of the Four Key Tasks in the Curriculum Reform as an entry point for achieving learning goals and developing pupils’ competence which is also the foundation for life-long learning [2].

PIRLS in each cycle collects a range of pieces of information about factors that may affect pupils’ learning by administering background questionnaires to pupils, parents, teachers, school principals, and curriculum experts, together with assessing pupils’ performance in reading. Our research team in Hong Kong modified the original PIRLS tool and altogether developed two Chinese questionnaires for use in Hong Kong, one for the home factors and one for the student. The development of the home questionnaire, namely, the Chinese version of the PIRLS 2011 Home Questionnaire (PIRLS-HQCV 2011), shared similar methodology to the student questionnaire and is reported elsewhere [3]. The development of the student questionnaire constitutes the focus of this paper. The student factors are reported to be important which contribute much to student achievement [46]. Unfortunately, there is a gap left in measurement of student attitudes [7]. Researchers have mainly focused their attention on the cognitive component including reading comprehension and acquisition of fluency [6]. Petscher’s [6] meta-analysis of 32 studies however suggested that the relationship between attitudes and achievement in reading is strong among pupils. Much research highlighted association of students’ individual characteristics with their reading achievement including self-beliefs or self-perceptions of reading competence [8, 9], reading self-efficacy [10], perception of reading easiness [10], and reading attitudes [6, 11]. However these studies are based on students in western countries. Validated questionnaires to assess student reading contexts [6, 7] and allow international comparison do not exist.

The student questionnaires (PIRLS-SQ 2011) encompassed two modules: (1) a general module inquiring on pupils’ home background and attitudes toward school and (2) a reading module inquiring on reading attitudes and behaviours, whereas the second module had the potential of filling the literature gap on educational measurement. An international version in English was first prepared by the PIRLS International Study Center. Subsequently, the questionnaires were translated by participating countries into their languages of instruction, with the endeavour of adapting individual national context and preserving the value of comparability across countries. Therefore, in the process of developing and reviewing the Chinese version, significant effort by the National Research Coordinators (NRC) of Hong Kong was allocated to ensure that the questionnaires were appropriately translated. After translation, the NRC also checked for any irregularities of translation and recommended corresponding revisions. Although the Chinese version of the PIRLS-SQ 2011 is widely used in different countries, its psychometric properties have not yet been investigated. By deploying exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), the present study explores the psychometric properties of the Chinese version of the PIRLS-SQ 2011 and investigates the underlying factor structure among Chinese fourth-grade pupils in Hong Kong.

It has been suggested that understanding Chinese reading is important for identifying the universal aspects of reading [12]. Share [13] argued that the research in past decades was largely studies of reading English. Reading science was distinctly preoccupied with the approach that focused only on Anglocentric research with limited generalizability at a universal level. The significance of our study on Chinese lies on the fact that it allows future comparisons across a wider spectrum of alphabetic and nonalphabetic languages. Most language-adapted measures, especially those adapted in Chinese, did not go through the process of factor analysis to ascertain the measures’ psychometric properties. The study has taken a significant step forward to validate the PIRLS-SQCV 2011 which may allow us to explain how research on pupils’ perspectives on Chinese reading informs universal reading science. In addition, we may address the important issue reviewed above as to the factors in Hong Kong that have contributed to the significant increase in the recent performance of pupils in the PIRLS which may have much implication to the reading-learning model as well as to teaching and learning process that may affect the reading competencies of Chinese pupils.

2. Method

2.1. Participants

Adopting the uniform sampling approach specified by the PIRLS National Research Coordinator (NRC) [14], the PIRLS 2011 applied probability proportional-to-size (PPS) technique for school sampling and recruiting participants [15]. The method of two-stage stratified cluster sampling was adopted. A sampling of schools was completed at the first stage. At the second stage, a sampling of intact fourth-grade classrooms in the sampled school was completed. Systematic random sampling with probability proportional to their measure of size (MOS) was used for sampling of schools. Within each sampled school, one of all fourth-grade classes was randomly sampled based on random assignment generated by SPSS. 132 primary schools, from 18 districts of Hong Kong, participated in the PIRLS. The initial sample size was 4,189 pupils, with 4,105 pupils found to be eligible and finally recruited between March and May in 2000/2011. 3,875 pupils completed PIRLS-SQCV 2011. The average age of the pupils was 10.1 years. 46% of the participants were girls and 53% were boys [16].

2.2. Measures

The PIRLS-SQCV 2011 consisted of a total of 18 questions with six questions on background information and twelve questions with 52 items on reading attitudes and behaviours. These 52 items comprise measurement of home environment (4 items), school climate for learning (9 items), out-of-school reading behaviours (11 items), lessons about reading (9 items), and attitudes toward reading (19 items). While the items were grouped into 12 questions in the questionnaire, they were considered as individual observations in the factor analyses. A total of 29 items were answered on a 4-point scale: “agree a lot,” “agree a little,” “disagree a little,” and “disagree a lot,” as 23 statements to be answered on a 4-point scale ranging from “never” to “everyday.” Six items were scored in reverse for consistency meaning. Table 1 tabulates the items, background information, and contextual factors.

2.3. Procedure

Each sampled class was assigned a Test Administrator who was selected and trained by the School Coordinator. Standardized procedures were followed to administer the Reading Assessment and Student Questionnaire [17]. After obtaining their informed consents, each pupil was given one booklet and asked to read two passages. The testing time was limited to 80 minutes, with 40 minutes for reading each passage. Pupils filled in the PIRLS-SQCV 2011 after the reading test in the classroom. To complete the questionnaire, pupils were given at least 30 minutes. The Test Administrator read the questionnaire items aloud together with the pupils to fill in the questionnaire. The NCR nominated an International Quality Control Monitor (ICQM) who was trained by PIRLS International Study Center to ensure quality of test administration. The ICQM conducted the site visits to a sample of 15 participating schools during data collection.

2.4. Statistical Analyses

To follow usual practices [18], one-third of the valid data of the questionnaire was randomly selected to undergo the reliability testing using Cronbach’s coefficient alpha and exploratory factor analysis (EFA) () by SPSS version 20; two-thirds of other valid data underwent the test of construct validity of the questionnaire using CFA () by Amos version 21.0. Cronbach’s coefficient alpha was calculated as an examination of the internal consistencies of the overall scale and subscales from the questionnaire. The Kaiser-Meyer-Oklin (KMO) value and Bartlett’s Test of Sphericity (BTS) were used to test whether the collected dataset was suitable for subsequent factor analysis. EFA was performed to identify a set of latent constructs underlying a battery of measured items. Fabrigar et al. (1999) suggested that maximum likelihood (ML) was the best extraction method if data were relatively normally distributed; otherwise, principal axis factoring (PAF) method was recommended [19]. Provided with the assumptions of correlations among factors, Promax with Kaiser Normalization was used for rotation. In the current study, factors with eigenvalue greater than 1 were extracted along the two methods separately (i.e., ML and PAF) with Promax rotation (). The model was then tested by CFA based on previous empirical research on pupil’s reading achievement. Factors were allowed to be correlated and multiple goodness of fit tests including chi-squared statistics (), comparative fit index (CFI) [20], Tucker-Lewis index (TLI) [21], and the root mean squared error of approximation (RMSEA) with 90% confidence interval (CI) of RMSEA were used to evaluate the model. If the results of ML and PAF were the same at EFA, ML estimator was used to evaluate model fit at CFA. Satorra and Bentler [22] suggested that robust correction should be made to all model fit indices if necessary. A TLI greater than 0.90 and a CFI greater than or equal to 0.95 indicate a good fit to the data [23, 24]. A RMSEA value less than 0.05 indicates a good fit to the data while a value greater than 0.10 indicates an unsatisfactory model fit [25]. The current Chinese model was evaluated using the four-point Likert scale (1–4) based on the original structure of the questionnaire. Demographic data of the participants and the scaling of the questionnaire were summarized with descriptive and frequency statistics (mean and SD). In addition, individual scores of the subscales scores were also computed using the average score of the appropriate items.

3. Results

3.1. Factor Structure and Reliability of the Student Questionnaire

Drawing from the KMO value (0.929) and the BTS (0.000), the data was found to be suitable for factor analysis. EFA showed that 38 items of the 886 samples were above 0.40 for factor loading and could be categorized into 10 factors. This 10-factor solution accounted for 54.37% of the total variance. However, 14 items below 0.40 were not segregated under any of the 10 factors and hence not considered for the subsequent analyses. The 38 items fell into the same factors as the original questionnaire. The item-factor allocation was based on the significant loading from the EFA results and the nature of content. The ten factors were (1) students motivated to read, (2) out-of-school reading behaviour, (3) students engaged in reading lessons, (4) students bullied at school, (5) perceived reading difficulties, (6) parents’ support for students’ studies, (7) school climate for learning, (8) reading self-efficacy, (9) reading enjoyment, and (10) independent reading behaviour at school.

All of the items could be meaningfully interpreted in the assigned factors. Both the ML and PAF extraction methods with Promax rotation yielded the same 10-factor solution with eigenvalue greater than 1. The finalized items and factor structure of the questionnaire using ML with Promax rotation are shown in Table 2. The correlation between items and their corresponding factor was measured by Cronbach’s alpha of coefficients. The correlations among the factors were also measured by Cronbach’s coefficients alpha. The internal consistency of the 10 factors/scales based on the EFA results was found to be satisfactory with a range from 0.71 to 0.86. The mean scores, standard deviation, and the coefficients alpha are summarized in Table 3.

3.2. Construct Validity of the Student Questionnaire

The 10-factor model was then tested using CFA. The accompanying standardized path coefficients to the corresponding items are shown in Table 3. Examination of model fit indices revealed the best-fit to the data of the 10-factor model, with (df = 620) = 2633.09, CFI = 0.95, TLI = 0.94, and RMSEA = 0.034 (90% CI from 0.032 to 0.035). The standardized path coefficients of the factors to the corresponding items were all significant, ranging from 0.49 to 0.89. Correlations among the ten factors were shown in Table 4. Factor 6 “Parents’ support for students’ studies” was significantly correlated with all the other factors except for factor 4 “Students bullied at school” and factor 5 “Perceived reading difficulties.” Factor 7 “School climate for learning” was significantly correlated with all the other factors except for factor 5 “Perceived reading difficulties.” Factor 4 “Students bullied at school” was significantly correlated with all the other factors except for factor 2 “Out-of-school reading behaviour” and factor 10 “Independent reading behaviour at school.” Factor 2 “Out-of-school reading behaviour” was significantly correlated with all the other factors except for factor 3 “Students engaged in reading lessons.” Factor 3 “Students engaged in reading lessons” was significantly correlated with all the other factors except for factor 5 “Perceived reading difficulties.” Factor 10 “Independent reading behaviour at school” was significantly correlated with all the other factors except for factor 4 “Students bullied at school.” Factor 9 “Reading enjoyment” was significantly correlated with all the other factors. Factor 8 “Reading self-efficacy” was significantly correlated with all the other factors except for factor 5 “Perceived reading difficulties.” Factor 5 “Perceived reading difficulties” was significantly correlated with all the other factors except for factor 6 “Parents’ support for students’ studies,” factor 7 “School climate for learning,” factor 3 “Students engaged in reading lessons,” and factor 8 “Reading self-efficacy.” Factor 1 “Students motivated to read” was significantly correlated with all the other factors.

4. Discussion

A large and representative sample size, which is the main advantage of the current study, facilitated analysis of EFA and CFA. The 10-factor solution suggested by EFA and then confirmed by CFA demonstrates much resemblance to the original student questionnaire of PIRLS 2011. The factor structure of the PIRLS-SQCV 2011 and the original six PIRLS context questionnaire scales pertaining to pupils is shown in Table 4. Table 5 shows the factor structure following EFA and CFA of the PIRLS-SQCV 2011 and the original six PIRLS context questionnaire scales pertaining to pupils.

Our factor analysis ascertained that five factors had perfect alignment with the original context questionnaire scales and contextual factors which was articulated in the PIRLS 2011 assessment framework. Specifically, factor 1 “Students motivated to read,” factor 4 “Students bullied at school,” and factor 7 “School climate for learning” aligned well with the original three context questionnaire scales of students on “Motivated to read scale,” “Students bullied at school scale,” and “School climate for learning scale,” respectively. Another two factors showed good alignment with two contextual factors which have not explicitly been considered scales but grouped under the same respective question. These included factor 6 “Parents’ support for students’ studies” under question G7 and factor 10 “Independent reading behaviour at school” under question R6. Factor 5 “Perceived reading difficulties” and factor 8 “Reading self-efficacy” of the present study parallel the PIRLS “students confident in reading scale.” This result reflects that confidence in the Chinese pupils consists of multiple meanings which are different from western pupils. This may be explained from the unique philosophical system of the Confucian teaching which stipulates that the Chinese children do not deny the existence of difficulties they encounter when pursuing their study, because they believe that effort is responsible for the ultimate level of achievement [26]. It is evidenced from the EFA that pupils’ confidence in reading in the Asian context spanned on two distinct concepts: reading self-efficacy and confidence in overcoming the perceived reading difficulties. However, items 8D and 8F exhibited factor loading below 0.4 and were not grouped under any of the factors. Both items were related to pupil’s willingness to read book at difficult levels and teacher’s appraisal on reading performance, respectively. Hence, further research should be done to gain more insight for these unexpected findings.

There are two factor constructs that showed only partial alignment with the PIRLS original context questionnaire scales. These included factor 3 “Students engaged in reading lessons” and factor 9 “Reading enjoyment.” Items R5A–G and R7A–F should be placed under “Students engaged in reading lessons scale” and “Students liked reading scale,” respectively. However, items R5A, R5D, and R7A–C had factor loading below 0.40 in the EFA result and were not grouped under any of these two corresponding factors. In fact, items R5B-C and R5E–G asked more about pupils’ engagement in teacher’s instruction during reading lessons. Items R5A “Like the reading in school” and R5D “Think of unrelated things in class (reversely coded)” were more related to pupils’ intrinsic motivation although these items were grouped under the same question by the PIRLS expert. In a similar context, items R7A–C were about “Read only have to (reversely coded),” “Like talking about reading,” and “Happy to receive books as gifts,” respectively. These items were not purely related to reading enjoyment. Our CFA analysis concluded that “Students liked reading scale” only reflected a single dimension of “reading enjoyment,” Further investigation is needed to affirm the dimensionality. Items R1–R4 were not explicitly defined as scales but considered an important student contextual factor of “out-of-school reading behaviour” by the PIRLS expert. However, items R1, R3B–F, and R4 were more related to out-of-school reading frequency on materials borrowed from library or materials on magazine/newspaper/TV. As a result, our EFA analysis generated that only R2A–C and R3A were grouped under factor 2 “Out-of-school reading behaviour.”

This study fills the knowledge gap for validating an instrument to assess factors pertaining to pupils. According to the results of EFA and CFA in the present study, the PIRLS-SQCV 2011 could be used as a validated tool for various research purposes pertaining to reading of Chinese pupils around the world. The results showed that the 10-factor structure of this questionnaire resembles the original PIRLS structure. This similarity allows international comparisons of studies carried out in different places of the world following the PIRLS strategy.

Among the participating countries/districts, Hong Kong has shown significant improvement in pupils’ reading achievement [27] for the study period. In 2001, Hong Kong ranked 14th and jumped to 2nd in 2006; and it became the first on the list in 2011. Researchers are impressed by the remarkable improvement and have become interested in identifying factors that have contributed to this upward trend. The PIRLS-SQCV 2011 could be used to assess the factors that have contributed to the top performance of Hong Kong pupils in PIRLS 2011, so that teaching strategies may be developed to help improve the reading competencies of pupils not only in Hong Kong butin other places in the world, especially communities that are predominately Chinese.

Even though most of the results discussed above are supportive in disentangling the psychometric properties of the scale, the study is not without limitations. Although a two-stage stratified cluster sampling was adopted, the collected sample could not represent all Hong Kong fourth-grade pupils in the international PIRLS. Even though the results of EFA and CFA both confirmed the 10-factor model for the student questionnaire (PIRLS-SQ 2011) of the Progress in International Reading Literacy Study (PIRLS), some of the factors were a bit weak, such as factor 8 (Reading self-efficacy) and factor 10 (Independent reading behaviour at school), being only composed of two items, the minimum for a particular factor. Among all the items in CFA, two items’ standardized path coefficient is lower than 0.5. They are G7d “My parents check if I do my homework” (0.486) and G9a “I was made fun of or called nicknames” (0.488).

Nevertheless, the factor constructs could still be seen as predictors of reading achievement after implementation of multiple regression or Structural Equation Modelling (SEM) to identify key factors that contribute to reading development. The findings could then fill the gap of existing literature concerning the relationship between student factors and reading achievement. The factor constructs could potentially be used by educational researchers and teachers in Hong Kong and other Chinese societies such as Taiwan, Singapore, and mainland China with minor adjustments.

Competing Interests

The authors declared no potential conflict of interests with respect to the research, authorship, and/or publication of this paper.

Acknowledgments

The authors would like to acknowledge Ms. Claire Qiuzhi Xie for her support in the initial data analysis process and Dr. Raymond Chung for his statistical advice. This work was supported by the General Research Fund from the Research Grants Council of Hong Kong under File no. 17606715, Hong Kong.