Abstract
A well-tested comprehensive Chinese positive youth development program (Project P.A.T.H.S.) developed in Hong Kong has been modified and adapted for use in Macau. This program aims to help adolescent school children develop positively and to be better prepared for their future. The present study investigated the effectiveness of the Tier 1 Program of “P.A.T.H.S.” for Secondary 2 students of two pilot schools. Since there were “repeating” and “transferring” students joining the program, the effectiveness of the program on these particular groups of participants was also examined. The subjective outcome evaluations including participants' perceptions of the program, program instructors, benefits from the program, and overall satisfaction were positive. Although the longitudinal data from the objective outcome evaluation did not show any notable improvement, the overall effect of the program was found to be positive to the new comers in the junior secondary years. The existing evaluation findings suggest that the Secondary 2 program is especially effective to those newly joining the program. In view of the paucity of youth studies in Macau, the present study can contribute to evidence-based youth work and provide baseline data for the program to be evaluated in the Secondary 3 periods in the future.
1. Introduction
Macau is a small city located near Hong Kong in South East Asia, famous for tourism and an increasing gaming industry. In 2010, the estimated population of Macau was 542,400, and it has a comparatively young population with those aged between 10 and 24 years contributing 22 percents to the total population [1]. The Macau Government opened the gaming licensure in 2002, and this leads to a rapid development of this industry, generating a big increase of revenue and contributing to the economic growth of Macau. However, this may also bring with it potentially negative influences on adolescents. Due to employment opportunities and perks in the gaming industry, many adults work within the casinos, and this requires them to work long and irregular hours. One potential implication of this development was highlighted in a recent government report which clearly identified the problem on lack of communication of parents with their children and the adverse effect on their adolescent development [2]. The Youth Indicators published by Education and Youth Affairs Bureau in 2009 revealed that many teenagers lacked social norms and their participation in social functions/affairs, and the sense of belongingness to Macau had deteriorated in comparison to an earlier study conducted in 2006 [3]. This research also reflects a dramatic increase in youths stress levels, originating from pressure at school as well as family conflicts. A recent study has revealed that over half of 744 respondents (54%) agreed that gambling was a common phenomenon in young people in Macau. It is suggested that, by building up positive social norms and a sense of morality in adolescents, a more harmonious society in Macau may be resulted [4].
A well-structured local youth program can potentially help adolescents develop positive growth and ensure that they are better prepared for future challenges in life. At present, youth studies and theoretically sound and comprehensive programs for adolescent positive growth and development in Macau are lacking. [5, 6]. Hong Kong and Macau share a similar Chinese culture; therefore, the well-tested comprehensive Chinese positive youth development program “P.A.T.H.S.” developed in Hong Kong [7, 8] has been modified and adapted for use in Macau. The acronym “P.A.T.H.S.” denotes Positive Adolescent Training through Holistic Social Programs. There are two tiers of programs (Tier 1 and Tier 2) in this project. Tier 1 program is a universal positive youth development program in which students in Secondary 1 to 3 participate normally 20 hours of duration or at least with 10 hours of training of the core program during each academic year. With the support from the Education and Youth Affairs Bureau, a local research team was formed by the author and his colleagues who modified the program with some changes of the content relating to local terminology, government structure, and indigenous customs [9]. The team also monitored the implementation of the program and evaluated its effectiveness after the completion of the program for three consecutive academic years. Two secondary schools were invited to participate as pilot schools to run the program starting from their Secondary 1 students. Training for teachers and school social workers was also organized both in Hong Kong and Macau. The initial findings of this Secondary 1 program evaluation was completed during the academic year 2009-2010 and reported positively [9].
Macau has a history of a 15-year free nontertiary education system with direct promotion from primary to secondary school without any public examination. Individual admission examination is required after secondary school education for entry into local universities. Consequently, there is a rather relaxed and less competitive learning atmosphere in the education system. It is not uncommon to see students repeating their studying in some classes in the primary and secondary schools if they find their academic performances unsatisfactory. It is roughly estimated that about 20% of students may repeat their studying in the junior secondary years in many schools [10]. In this paper, the effectiveness of the Macau version of the Tier 1 Program of “P.A.T.H.S.” for Secondary 2 students in 2 pilot schools during the academic year 2010-2011 is evaluated. Both subjective and objective outcomes were measured. Since there were imported students either repeating the Secondary 2 class or transferred from other school joining the Secondary 2 classes of the pilot schools, the effectiveness of the program on these new participants was also examined.
2. Method
2.1. Participants and Procedures
The study participants included all Secondary 1 students in the two chosen schools, totally 232 starting from year 2009, these were followed up for three years up to Secondary 3. When this group of students was promoted to Secondary 2 year, 43 of them dropped out of their classes and 79 new students either repeating in Secondary 2 or transferred from other schools had joined the program. Data was collected at two time-points during each year. Firstly, before the program started, pretest self-reported questionnaire was completed within 1-2 weeks after the start of the school year. The second data collection time-point occurred after the students finished the program at the end of that academic year. The pretest data Wave 1 (W1) and posttest data Wave 2 (W2) of Secondary 1 had been collected and analyzed [9]. This data was used as baseline for the longitudinal assessment for the Secondary 2 year. The pretest data Wave 3 (W3) and posttest data Wave 4 (W4) of Secondary 2 were collected in this study to assess the effectiveness of the Secondary 2 program.
At pre- and posttests, the participants were invited to complete a valid and reliable questionnaire including measures of positive youth development, life satisfaction, school adjustment, adolescent problem behaviors, and demographic information. An identical questionnaire was used in the pre- and posttests, the paired-samples t-test were performed to examine the pre- and posttest differences between the scales, providing an objective outcome measures for evaluation. After completion of the program each year, an evaluation questionnaire was also completed by the participants to assess their satisfaction with the course and perceived benefits of the program, providing subjective outcome measures for evaluation.
2.2. Instruments
The two sets of questionnaires which were used in Year 1 were used again when the students were promoted to Year 2. The components in these questionnaires are described below.
2.3. The Chinese Positive Youth Development Scale (CPYDS)
The Chinese Positive Youth Development Scale (CPYDS) is a self-administrated questionnaire developed by Shek et al. [11]. It consists of 15 subscales (90 items) which address the 15 constructs of the program. These subscales include bonding (BO, 6 items), social competence (SO, 7 items), emotional competence (EC, 6 items), cognitive competence (CC, 6 items), behavioral competence (BC, 6 items), moral competence (MC, 6 items), self efficacy (SE, 7 items), prosocial norms (PN, 5 items), resilience (SE, 6 items), self-determination (SD, 5 items), spirituality (SP, 7 items), identity or clear and positive identity (ID or CPI, 7 items), beliefs in the future (BF, 7 items), prosocial involvement (PI, 5 items), and recognition for positive behavior (RP or PB, 4 items). The instrument had a good reliability (alpha = 0.91) and ranged from 0.63 to 0.86 [12]. The Cronbach’s alpha of the present study is 0.93 and ranged from 0.62 to 0.87. A higher score indicates a higher level of positive youth development.
2.4. Life Satisfaction Scale (LIFE)
Life satisfaction is another important indicator of positive youth development [13]. The five-item LIFE was developed by Diener et al. [14] to assess one’s own global judgment of one’s quality of life. The Chinese version was translated by Shek [15] with acceptable psychometric properties. The Cronbach’s alpha of the present study is 0.80. A higher LIFE scale score indicates a higher level of life satisfaction.
2.5. Behavioral Intention Scale (BI)
The five-item scale was used to assess the adolescents’ behavioral intention to engage in problem behavior, including drinking, smoking, taking drugs, having sex with others, and participation in gambling. The scale has a good reliability (alpha = 0.84) [16]. The Cronbach’s alpha of the present study is 0.71. A higher BI scale score indicates a higher behavioral intention.
2.6. School Adjustment Measures (SA)
The school adjustment measures include three items. Two items assess the participant’s perception of his/her academic performance. The third item assesses the participant’s perception of his/her conduct. Previous studies showed that these measures were temporally stable and valid [17, 18]. The Cronbach’s alpha of the present study is 0.84. In line with other measures, a higher scale score indicates a higher level of school adjustment in this study.
2.7. Subjective Outcomes Scale (Form A)
The Subjective Outcome Evaluation Form (Form A) was designed by Shek and Siu [19]. The Form consists of totally 39 items and four open questions which are divided into five parts. The first part asks for the participants’ views on the program (10 items). The second part looks at the views of the participants towards those involved in delivering the program, including teachers and/or social workers (10 items). The third portion examines the participants’ views about their perceived effectiveness of the program (16 items). Three items ask about tendency of participants to join a similar program in the future, their overall satisfaction with the program, and whether they would recommend the program to others. The final part consists of four open questions on things that participants have learned and appreciated most, as well as opinions about the instructors and areas for improvement, respectively. The form had a good reliability on 39 items (alpha = 0.99, mean interitem correlation = 0.80) [20]. The present study has a Cronbach’s alpha of 0.98 with mean interitem correlation 0.55.
3. Results
3.1. Demographic Information
In total, there were 268 and 244 students who participated in the Wave 3 pretest and Wave 4 posttest, respectively. Out of 268 Secondary 2 students, almost one-third of them, 79 (29.48%) were students who were new to the program. Once invalid questionnaires were discarded, mainly due to missing data, 236 questionnaires were appropriately completed for analysis. There were 173 old participants and 63 new participants (Table 1). Results showed that, with the exception of age, there were no statistically significant differences in sociodemographic background between the old and new students using the chi-square test. The mean age of the new students () was higher than that of the old group (). Details can be seen in Table 2.
3.2. Subjective Outcome Evaluation
Table 3 highlights the participants’ perception of the program and program instructors. Firstly, more than two-thirds of the participants viewed the program positively. For example, 77.4% of the students indicated that the objectives of the program were very clear; 77.8% felt that they had participated actively during the program; 69.0% had a very positive evaluation of the program. Secondly, more than four-fifths of the students provided a positive evaluation of their program instructors. The vast majority of participants (84.7%) felt they were well cared for, and 83.1% of them provided overall positive evaluation of the instructors. The participant’s perception of the benefits of the Tier 1 program revealed the following findings. Firstly, more than two-thirds of the participants perceived the program was effective for their personal development and enhanced their psychosocial competences (Table 4). In addition, they thought it also enriched their overall development (73.9%). Secondly, a majority of respondents (63.2%) indicated that they would recommend the program to their friends, and more than a half of them (51.3%) would join similar programs in the future (Table 5). Finally, more than two-thirds of the participants indicated that they were satisfied with the program (Table 5). The qualitative analysis of the four open-ended questions will not be reported in this paper.
3.3. Objective Outcome Evaluation
The overall evaluation of all Secondary 2 students, using the paired t-test to compare the W3 and W4 data of the program, revealed that there was a nonsignificant decrease in the score of CYPDS, SA or increase of LIFE. Besides, there was a significant slight increase in the score of BI (Table 6). To assess the continual effect of the program from Secondary 1 to Secondary 2, the W1 data of the Secondary 1 program was used as a baseline to measure the longitudinal effect after the participants had completed the Secondary 2 program. Only 189 participants completed Secondary 1 and Secondary 2 programs, using the repeated ANOVA for W1, W3, and W4 data revealed similar nonsignificant results. However, comparison between the W3 data of old participants with new participants revealed that the old students demonstrated better development, life satisfaction, and school adjustment than the new students. Significant differences were reported within the subscales of CYPDS, LIFE, and SA scores (Table 7). As there was a significant difference of age between the groups, analysis of covariance was conducted using age as a covariate. A statistical significant difference of ID subscale and LIFE score was also found. As shown in Table 8, when using the pair t-test to compare the W3 and W4 data of the new participants, results showed that there was significant improvement in EC, CC and ID subscales, LIFE, and SA scores. However, there was a significant decrease in BO subscale and an increase of score of the BI scale.
4. Discussion
In this study, the participants’ perception of the program and their program instructors were evaluated positively. With regard to their perception to the effectiveness of the program, and the overall satisfaction, results provided a positive feedback. Overall, subjective outcome evaluation generally supported positive perceptions to the program, program instructors, benefits from the program, and overall satisfaction with the whole course, these findings are consistent with those reported in Hong Kong [20, 21]. However, when asked about whether they would join similar program in the future, just over half of the participants replied positively. There may be several possible reasons for this dissociation between individuals’ perception towards the program and their behavioral intention to participate. Firstly, as suggested, students might think that similar programs may contain similar elements and not be motivated to join such programs again [20]. In fact, the three-year program is designed with the same 15 constructs but with more in-depth exploration of the constructs for senior classes. Secondly, presently students are more prone to be interactive in class. As revealed in the previous study by Luk et al. [9] on the Secondary 1 students, many of them felt that the program was boring. Therefore, if they have a choice, they may not be willing to join even if it may be beneficial for them. Thirdly, when students move to Secondary 3 classes, there are likely to be higher academic demands placed on them. Moral class or personal growth courses like P.A.T.H.S. may not be seen by the students as a high priority.
With regard to the objective outcome evaluation, our findings showed no significant improvement in the Secondary 2 program at posttest, which is inconsistent with findings in Hong Kong [22]. There may be several possible reasons. Firstly, the number of participants in this study is relatively small in comparison to the Hong Kong study which had 20 experimental schools with 2,784 students. This potentially increases the chances of sample bias in the study. Secondly, instructors in the Macau program included social workers and teachers, and these professions are relatively new to this type of interactive youth program, which may affect their effectiveness in conducting the program. Thirdly, the instructors also have to adapt the modified program to meet their student needs, and they may take time to practice and become familiarized with the program. Finally, a certain number of students dropped out and exchanged with a group of new students joined in may affect the group cohesion which in turn may affect their involvement in the program.
However, when the W3 pretest data of the old participants were compared with those of the new inexperienced participants, the findings generally showed that the old students performed better than the new ones. This was evidenced in terms of the global positive youth development and school adjustment indicators. Even with control for age, results still revealed that there was an improvement in positive identity subscale and life satisfaction scale. This comparison may demonstrate the effectiveness of the Secondary 1 program. Furthermore, when comparing the W3 and W4 data of this new group of students, significant improvement in different youth subscales, life satisfaction, and school adjustment measures was noted. This evidence supports the effectiveness of the Secondary 2 program in this group of students. One possible explanation may relate to the nonacademic nature of the program, so it may be more receptive to those whose academic performance is of average or below academic standard. As this group of students is older than the students who were promoted from their original schools, most of them are probably of average or lower standard. Another reason may relate to the fact that this is a new program and may attract their interest more than the older students. Conversely, the decrease in bonding score could be attributed to taking more time for them to develop new friendship, because of the repeating in Secondary 2 or adapting to a new school. More attention and further study are warranted in relation to the behavioral intention of alcohol consumption in this group.
Subjective outcome evaluation highlighted positive results from all participants. Although the longitudinal data from the objective outcome evaluation did not support any improvement, the effect of the program was found to be positive to the new comers in the junior secondary years. Therefore, in conjunction with the previous findings and based on the objective outcome evaluation, the Secondary 2 program is effective, particularly for new comers of the program. There are several limitations of this study. Firstly, only two schools are involved and the sample size was relatively small, raising the potential for sample bias, and it makes any generalization of the findings difficult. Secondly, the present study is based on a one-group pre- /posttest design which may not be the most appropriate. Other approaches such as the randomized control trial which can provide a more rigorous design to give more insight on the effectiveness of intervention program could have been considered. Thirdly, a comparative portion of students dropped out of the program which may affect longitudinal observation and observation of long-term effects. Finally, if the program is more beneficial to those with below average academic performances, and if some of them may have to repeat studying in Secondary 1 and lose the chance to be followed up, the evaluation of the program will be incomplete. Nevertheless, with the current paucity of youth studies in Macau, the present study can contribute to the understanding of the potential benefits of evidence-based youth work and provide further baseline data for evaluation of Secondary 3 periods in the future.
Acknowledgments
The research team would like to thank professor Daniel Shek and his research team in Hong Kong for their support in launching this study in Macau. Furthermore, this study was financially supported by the Education and Youth Affair Bureau of the Macau Government.