Abstract
This study concerns the permanence of the basic arithmetical skills of Finnish students by investigating how a group () of the eighth and eleventh year students and the university students of humanities perform in problems that are slightly modified versions of certain PISA 2003 mathematics test items. The investigation also aimed at finding out what the impact of motivation-related constructs, for example, students’ achievement goal orientations, is and what their perceived competence beliefs and task value on their performance in mathematics are. According to our findings, the younger students’ arithmetical skills have declined through the course of ten years but the older students’ skills have become generic to a greater extent. Further, three motivational clusters could be identified accounting for 7.5 per cent of students’ performance in the given assignments. These results are compatible with the outcomes of the recent assessments of the Finnish students’ mathematical skills and support the previous research on the benefits of learning orientation combined with the high expectation of success and the valuing of mathematics learning.
1. Introduction
In this millennium, the success in PISA (Programme for International Student Assessment) surveys has induced much positive interest in Finnish school and mathematics education. However, critical voices have also been heard. Already in 2005, based on the experience of their regular work, 206 Finnish mathematics professors and other mathematics educators from all Finnish universities and polytechnics published a letter [1] in a leading Finnish newspaper where they expressed their deep concern over Finnish students’ real mathematical skills, claiming that PISA only tells a half truth about students’ mathematical knowledge. The publication of the latest PISA results in December 2013 has only increased this criticism; the scores for Finland in the PISA 2012 mathematics test fell significantly [2].
Further, The Finnish National Board of Education, which is responsible for drawing up the national core curricula for primary and secondary education in Finland, regularly surveys primary and secondary students’ mathematical achievement with quite extensive samples. In the most recent reports [3–5], the general view is that the Finnish students’ mathematical skills have darkened in all areas of school mathematics. The most significant changes are related to arithmetic although basic arithmetic operations are still managed rather well [4, 5]. Geometry and the operations with the concept of per cent seem to be especially problematic issues for Finnish students.
The principal purpose of the present study is to provide a complementary view of the quality of Finnish mathematics education by examining how permanent or generic the results of teaching primary and lower secondary mathematics in Finland are. Since Finnish students have traditionally been quite good in the basic arithmetic operations and statistical reasoning, we decided to focus on such test items that merely require applying these skills. In order to make the recent development comparable with the situation a decade ago, at least, at an approximate level, some of the test items on our questionnaire are slightly modified versions (i.e., translated and recontextualized in the Finnish setting) of certain questions in the PISA 2003 mathematics test; for further details, see Sections 3 and 4.
The participants of the study consist of a group of the eighth grade students (), eleventh grade students of upper secondary school () studying the advanced () or basic () syllabus in mathematics, and university students () majoring in other than a mathematical subject. Consequently, our data do not represent a genuine longitudinal study and hence we cannot measure the permanence of the students’ arithmetical skills at an individual level. However, the comparison of the participating groups with one another and with the Finnish students in PISA 2003 is sensible because the contemporary national core curriculum for primary and lower secondary school has been implemented since August 2006 and the differences between that and the previous version from 1994 are very modest. In practice, all participants both in our study and in PISA 2003 have essentially studied the same mathematical contents in comprehensive school.
Another goal of the present paper is to draw an overview of the participants’ motivation in mathematics in order to better understand their performance in the test problems. Consequently, our research questions are as follows.(1)How does students’ performance in the given mathematical assignments (of which three correspond to the test items M413, M467, and M468 in PISA 2003) vary across the educational level?(2)How does the students’ motivation in mathematics relate to their performance in the given mathematical assignments?
2. Theoretical Framework
Although there is much empirical evidence indicating the important role of motivation in the study of achievement in mathematics, the relations between motivation-related constructs, such as students’ achievement goals and motivational self-perceptions and success in mathematics, call for further research (cf. [6]). A contemporary view in research on motivation emphasizes the role of achievement goals. A large body of research on achievement motivation during the past few decades has demonstrated that goals, values, and competence beliefs are primary effects on achievement motivation [7]. Still, according to Conley [8], little is known about how these components of motivation function as a coherent set within individual students. In her recent study, Conley [8] suggested a model of combining achievement goals and expectancy-value perspectives. This is also a focus in the present study.
Goal orientation theory has emerged as an important theoretical perspective regarding students’ motivation in school [9]. This theory provides a framework for extensive research on motivational orientations that contributes to students’ academic achievement, persistence, and performance. Researchers have differed about the number and the definitions of the orientations that people may adopt in achievement situations [10]. However, despite these differences, most researchers have mostly focused on two main orientations, a mastery orientation and a performance orientation, which were found to differently relate to adaptive and maladaptive engagements. In the literature, there is a diverse terminology used in labelling them. Often, the words “approach” and “avoidance” have been used as affixes to the performance orientation.
A mastery goal orientation (also called learning orientation and task orientation) has been defined as a student’s incentive to learn and develop competence. In the engagement in academic tasks, the focus is on learning per se or on task mastery instead of, for example, a competition with other students. In performance-approach orientation, the focus is on demonstrating one’s competence relative to others or in competition for good grades and on outperforming others. In performance-approach orientation, labelled as “self-enhancing ego orientation” by Skaalvik [11], the student’s goal is directed toward attaining favourable judgments of competence, whereas in performance-avoidance orientation, labelled as “self-defeating ego orientation” in [11], the aim is at avoiding unfavourable judgments of competence. Skaalvik [11] has also identified a fourth goal orientation, particularly in mathematics learning, which has been named as work avoidance orientation, or avoidance orientation, because a student’s ultimate goal is to invest a minimum amount of effort, gain an easy success, and only reach a passing grade [9, 11, 12].
As for contemporary research on goals, it has focused on a “multiple goals perspective.” This approach on goals has demonstrated that individuals can and do operate according to multiple salient goals and that both mastery and performance-approach orientation result in higher achievement (e.g., [13]), whereas performance-avoidance and work avoidance orientation consistently lead to maladaptive outcomes such as low perceived competence and poor performance. However, findings for performance-approach goals have been much less consistent [9, 11, 12].
In achievement motivation research, student performance is assumed to be mediated also by their motivational beliefs such as efficacy expectations and task values [7, 14]. Theoretically, the expectancy of success is closely related to other conceptions of self and ability beliefs, such as self-efficacy [15]. Eccles and her colleagues’ model of achievement is known as the expectancy-value model of achievement and is comprised of two related components: expectancy for success and subjective task value. Accordingly, expectancy for success is defined as individuals’ beliefs about how well they will do in an upcoming task. It also relates to their perception of being able to carry out their academic tasks successfully. Subjective task value refers to the qualities of tasks that increase or decrease the probability of their selecting the task or putting effort on learning it [14]. Subjective task value has four subcomponents: intrinsic or interest value, attainment value, utility value, and cost.
Intrinsic value refers to the student’s enjoyment of performing the task or the subjective interest they have in the subject. Attainment value is the importance of doing well in the task. Similarly, utility (instrumentality) value refers to how useful and important a school subject is for the students’ future goals, such as career plans. Cost value refers to negative (undesirable) aspects in engaging in a task, such as performance anxiety, the fear of failure. Cost can also be conceptualized in terms of the loss of time and energy for other activities [7].
Evidence is quite strong that the expectancy of success, interest, intrinsic motivation, and intrinsic value predict greater academic engagement and success in learning (see [7]). For example, in their recent study, Trautwein et al. [16] showed that expectancy and value components predict significantly German upper secondary school students’ () performance in mathematics based on TIMSS (Trends in International Mathematics and Science Study) standardized test and that the prediction power of expectancy of success is larger than that of value components.
3. Materials and Methods
The data of this survey were collected from four different lower secondary schools, four upper secondary schools from the different parts of Finland, and one university. The size of the sample () in this survey is not as large as in the surveys mentioned in the introductory section, yet it can be seen as sufficient to provide a quite realistic overview of Finnish students’ arithmetic skills. The participating university students represent the humanities: the students from three different courses at one typical multifaculty Finnish university were invited to participate. Those who did, did so on a voluntary basis without any reward.
We have organized the participants into four groups and, in the following sections, use the following abbreviations of them: PK, the eighth graders (“peruskoululaiset”); MAA, the eleventh graders studying the advanced syllabus in mathematics (“pitkä matematiikka”); MAB, the eleventh graders studying the basic syllabus in mathematics (“lyhyt matematiikka”); UNIV, the university students. The difference between the MAA and MAB syllabi in upper secondary school is that the MAA students have ten compulsory courses but they often take three or four specialisation courses, too, and the MAB students study six compulsory courses and typically 0–2 specialisation courses. Moreover, the MAB courses are more focused on basic arithmetic, problem solving, and the real life phenomena.
In the PISA, the Finnish participants are 15-year-olds from the eighth and ninth grades. In order to minimize the organizational troubles at the participating schools, we focused only on the eighth graders. Since most children start school in Finland at the age of seven, the PK students are fourteen or fifteen years old. Most of the MAA and MAB students are seventeen years old. The age of the UNIV students was not asked but mostly they are second or third year university students.
The participants were given 60-minute time to answer the form. The use of a calculator was allowed. The form contained, first, a section surveying the participants’ educational history and other typical background information. In the second section, the students’ motivation-related beliefs were measured using a modified questionnaire based on a well-established achievement goal instrument by Skaalvik [11] and Lukin [6] and an instrument targeted to measure students’ competence and task value beliefs which was adapted from Wigfield’s and Eccles’s [14] expectancy-value model of motivation.
The students’ goal orientations were measured using a 27-item Likert-type instrument with a five-point scale ranging from 1 which is “Strongly disagree” to 5 which is “Strongly agree.” The students’ perceived competence (the expectancy of success) and the value of learning mathematics were measured in the same range using a 19-item scale. Both measures were adapted to the Finnish educational context. The items were written in a domain-specific form (see [10]), for example, “In my mathematics class, I like to solve problems by working hard.” Other exemplar assertions will be given in Section 4.2.
The data of this section were analysed in sequential steps consisting of an exploratory factor analysis, -means cluster analysis, and two-way analysis of variance (two-way ANOVA). First, an exploratory factor analysis was applied to develop two sets of scales of the students’ motivation-related beliefs and to serve in identifying two sets of latent constructs underlying the batteries of measured variables. Second, a centroid-based clustering, that is, -means clustering, was used to identify the groups of students based on the six scales derived from the factor structures of the motivation instruments. The third phase of the data analysis was to test mean differences in students’ mathematics performance using two-way ANOVA with a motivation group (three clusters) of a student and his/her educational level as independent variables.
The final section of the form consisted of altogether seven mathematical assignments. The items were given to the students in Finnish but we now give their English translations.(1) (Corresponding to M413Q01 in PISA 2003) Anne and her two friends decided to spend their holidays in Sweden.(a) How many Swedish Kronas did they get for 2000 Euros as the exchange rate was 1 EUR = 9.3 SEK?(b) A month later the rate was 1 EUR = 9.5 SEK. How many Kronas more or less would they have gotten with this rate?(2) (The first part corresponding to M467Q01 in PISA 2003) There are differently coloured balls of the same size in a box. The number of balls of each colour is given in the following. Red: 6 Yellow: 5 Orange: 3 Green: 3 Blue: 2 White: 4 Purple: 2 Brown: 5 Bertil takes one ball out of the box without looking at the colour.(a) What is the probability that the ball is red?(b) How many red balls should you insert into the box in order to guarantee that the probability is at least 25 per cent?(3) (The first part corresponding to M468Q01 in PISA 2003) Camilla takes part in a quiz, which consisted of five rounds, each one giving 0–100 points. After four rounds, the average of her points is 60. Her scores for the final round are 80 points.(a) What is the average of Camilla’s points in the whole quiz?(b) What could have she been able to deduce on that average already before the final round?(4) On a summer day, you are at the market square with Diana and Edward. Edward proposes: “If I buy us 1/2 kg of strawberries, which we all share one by one, how about you two together pay me the price of the portion multiplied by the remainder of the division?” Should you suggest to Diana that you accept this offer?
The fundamental idea in designing the test items was as follows. First we selected three mathematics questions from the PISA 2003 test (M413Q01, M467Q01, and M468Q01). They were only translated and recontextualized in the Finnish setting. Then we generated three follow-up assignments to measure the same content knowledge. The purpose of this manoeuvre was to measure to what degree success in the given task depends on that how it is given. The last test question was intentionally designed to be more challenging than the others in order to see how many students are motivated enough to take on the challenge.
The first six test items were scored on the scale 0–3, where 0 is a wrong conclusion with no explanation or with several serious defects in the explanation or a missing answer, 1 is a correct conclusion with no explanation or an incorrect conclusion with a partly reasonable explanation, 2 is a correct conclusion with a few minor defects in the explanation, and 3 is a correct conclusion with a reasonable explanation. In the corresponding PISA 2003 test items, the evaluation of the responses was dichotomous (1 is full credit; 0 is no credit); that is, no explanations were required. Therefore, in the comparison of our data with that of PISA 2003, we recoded and . The scale on the last and a more challenging assignment was 0–6.
4. Results
4.1. Analysis of Students’ Mathematical Performance
We begin with answering the first research question. The distributions of the means and the percentage of the full credit responses of each group (the figures in the parenthesis) are given in Table 1.
As Table 1 shows, the participants’ success in general was merely satisfactory. The average of total scores is only 9.99 (the maximum being 24). The PK students performed well only in 1a, the other groups quite well also in 1b, 2a, and 3a, and the MAA students rather well in 2b, too.
More precisely, in the one-way ANOVA and Bonferroni’s post hoc test, the eighth graders’ performance (total scores) was very highly significantly weaker than that of other groups (, ). Also the difference between the means for MAA and MAB is significant () in the same test. The mean of total scores of MAA is the highest but the difference between MAA and the second best group UNIV barely remains insignificant.
The schools which compose PK are all not equal: the difference of the means for the best (8.25) and the weakest (6.04) school is significant in one-way ANOVA and Dunnett’s T3 post hoc test (, ).
If we look at the differences in each item in more detail (one-way ANOVA and Dunnett’s T3), in the comparison of MAA and UNIV, the difference is highly significant () in item 1a and, in 2b, significant () in favour of MAA. There are no remarkable differences between MAB and UNIV although UNIV succeeded slightly better in 2a, 2b, and 3b and MAB in 3a. In 3a, the MAB students even performed a little better than the MAA students, yet the difference is not significant. A possible reason for the good success of the MAB students may be that elementary statistics is quite widely discussed in their curriculum. The mean of PK is highly or very highly significantly lower than other groups’ means in each item except in the last one.
All in all, these findings suggest that the students’ basic arithmetic and statistical skills still improve in upper secondary school and they are also quite permanent (cf. [3]). On the other hand, they are rather superficial. As the comparison between the scores in 1a and 1b, 2a and 2b, and 3a and 3b, respectively, shows, even a minor modification of a task easily results in a significant decrease in the rate of success. Concerning the last item, students’ performance was even worse than expected. Only two MAA students succeeded in solving it; 334 students gave a nonempty answer but 317 of them were virtually completely wrong or nonsensical.
We complete this section by comparing the PK students’ success in 1a, 2a, and 3a with the Finnish students’ results in the original PISA 2003 test items. We remark that because the PK group consists only of the eighth graders and there also were ninth graders in the Finnish sample in PISA 2003, the results of the comparison must be read only as approximate.
For Finland, in items M413Q01, M467Q01, and M468Q01 in PISA 2003, the percentages of the correct answers are 90%, 60%, and 53%. The OECD average percentages in the same items are 80%, 50%, and 47%, respectively [17, Table 3.2, page 301].
Now, for the PK students, the percentages of the zero point answers in 1a, 2a, and 3a are 20%, 55%, and 63%, respectively. In other words, on the PISA 2003 scale, the percentages of the correct answers in 1a, 2a, and 3a are 80%, 45%, and 37%. In these items, the PK students’ performance seems not to exceed the OECD average level in PISA 2003. Their success in 1b, 2b, and 3b is even worse which only reasserts the above conclusion. This is in line with the findings of [2, 4].
4.2. Analysis of Students’ Motivation
Data on the motivation-related beliefs, the achievement goal measure (27 Likert-type scales) and the expectancy-value measure (19 Likert-type scales), were reduced to fewer new variables using an exploratory factor analysis. For the achievement goal measure, a four-factor solution accounting for 52% of the common variance was computed using Maximum Likelihood extraction with Direct Oblimin rotation. The four factors extracted were labelled as “mastery orientation,” “performance-approach orientation,” “performance-avoidance orientation,” and “avoidance orientation.”
The first factor focuses on the individual’s development of competence, understanding, and task mastery as well as learning new and interesting things. An example out of the seven items included in the factor is as follows. “In math classes it is important for me to learn something new.” The second factor comprised of seven items describes an achievement goal where an individual is directed toward attaining the favourable judgments of competence compared with the peers in the classroom. A sample item is as follows. “In my math classes I try to score higher on tests than other students.” The third factor with six items focuses on what students are concerned with at school and emphasizes the attainment of the favourable judgments of their ability from others and avoids unfavourable ones (e.g., “If I give a wrong answer in a math class I am most concerned with what other students think about me”). The four items on the last factor describe a goal orientation where a student hopes to avoid hard work (e.g., “I like math classes best when there is no hard work”). Reliability estimates (Cronbach’s alphas) for the subscales from first to fourth were 0.83, 0.86, 0.89, and 0.74, respectively. Based on the four-factor solution, factor coefficient variables were computed.
Using the same procedure of factor analysis as above, the measure of students’ perceived competence and value of learning mathematics were reduced to two factors accounting for 58% of the common variance. The first factor includes ten statements such as “Mathematics has always been easy to me” and “I’m sure that I am capable of solving the most difficult assignments in my math class” and is therefore labelled as “perceived competence.” The second factor is extracted from the remaining nine statements consisting of three value subcomponents: intrinsic value (“I like the things I learn in a math class”), attainment value (“I’m interested in learning math”), and utility value (“Math skills are useful in everyday life”). The reliability estimates for both subscales were reasonably high (Cronbach’s alphas were 0.93 and 0.91, resp.).
Table 2 presents the intercorrelation matrix (Pearson) between the six factor score coefficient variables (four for achievement goals and two for expectancy-value components) and the mathematics performance, that is, the total scores in the last section of the form.
The correlation matrix shows a reasonable positive correlation between a mastery orientation () and a performance-approach orientation and a negative correlation between a mastery orientation () and an avoidance orientation, which is in line with the findings in [6, page 138]. The coefficient of correlation between the competence and the value components () demonstrated a moderate overlap of these constructs which has also been verified in [16]. Three out of four achievement goal variables correlated moderately with perceived competence and task value. The correlations between perceived competence and task value with mastery orientation and performance-approach orientation were positive, ranging from 0.42 to 0.79, and with avoidance orientation negative (−0.47 and −0.63, resp.). Mathematics performance correlated positively with mastery orientation (), performance-approach orientation (), perceived competence (), and task value () and negatively with avoidance orientation () supporting Lukin’s [6] results.
The -means cluster analysis produced an interpretative solution of three clusters of students with statistically very highly significant () differences in each of the variables (cf. Table 2).
Table 3 represents the final cluster centres as standardized scores based on factor analyses. The first group of students () with a reasonable high positive value on the avoidance orientation scale and negative -values on the rest of the scales can be defined as “Avoidant Oriented.” The second group of students is quite an opposite to the first one with positive means on both scales of expectancy-value measure and on two favourable scales of motivational goal orientations, namely, mastery orientation and performance-approach orientation. These students perceive their competence positively and emphasize the value of learning mathematics. Consequently, this group () of students can be called “Learning Oriented and Self-Confident.” Students in the third group () can be described as “Performance-Avoidant Oriented” due to the highest positive mean on the corresponding variable and average level of means on all other variables.
Differences in mathematics performance between the three motivation clusters and the four groups based on students’ educational level as well as their interaction effect are shown in a two-way ANOVA table (Table 4) and the table of means (Table 5).
Table 4 shows that both motivation cluster and education level statistically have a very highly significant effect () on mathematics performance. The partial eta square for the motivation cluster was 7.5% and, for the education level, 26.5%. The interaction effect on the motivation cluster and the educational level was statistically insignificant.
Multiple comparisons for the mean differences between the groups in both independent variables were conducted using Bonferroni post hoc tests. These showed statistically significant () differences between all motivation clusters, indicating that cluster two students (Learning Oriented and Self-Confident) scored the highest in mathematics (M = 12.26, SD = 4.17) and cluster one students (Avoidant Oriented) the lowest (M = 7.51, SD = 4.99). Cluster three students (Performance-Avoidant Oriented) were in the middle (M = 9.86, SD = 4.66); see Table 5. For the educational level, the Bonferroni post hoc showed statistically significant differences between all other groups except MAB (M = 11.82, SD = 3.88) and UNIV (M = 11.46, SD = 4.11).
5. Discussion and Conclusions
In the light of our data, Finnish students perform quite well in standard arithmetic tasks (1a, 2a, and 3a), but already a minor modification (especially 2b and 3b) of the assignment results in a remarkable decline in the solving rate. Similar findings have recently been reported by Häkkinen et al. [18] who studied the arithmetic skills of applicants for primary school teacher education at a Finnish university. A possible reason for this phenomenon may be the students’ linguistic difficulties in understanding the given assignment; for instance, Tossavainen et al. [19] noticed that university mathematics students’ success in studying the monotonicity of the square function remarkably depended on how this task was given.
The comparison of the PK students’ success in 1a, 2a, and 3a with the Finnish results in the corresponding PISA 2003 test items seems to confirm what we already know: the Finnish lower secondary students’ proficiency in mathematics is declining (cf. [2, 4]). On the other hand, Table 1 suggests that students’ mathematical literacy seems still to develop in upper secondary school. The scores for the university students of the humanities were highly significantly or very highly significantly higher than those for the eighth graders in each item. This outcome suggests that the permanence of the university students’ arithmetic skills is strong.
Concerning our findings on the participants’ motivation, previous research has brought up similar results. Results of the cluster analysis showed that two of the goal orientations, mastery orientation, and performance-approach orientation go hand in hand with both of the components of the expectancy-value model. The pattern was similar in each of the three clusters: high mastery orientation and high performance-approach orientation were linked to the high expectancy of success and high task value in cluster two (Learning Oriented and Self-Confident). The corresponding cluster means were the lowest of all in the first cluster (Avoidant Oriented) and in the middle in the third cluster (Performance-Avoidance Oriented). Hence, the present study supports Conley’s [8] suggestion that achievement goals and expectancy-value perspectives can be combined.
Furthermore, the results support the multiple goals view of achievement goal theory because cluster two students, Learning Oriented and Self-Confident, got almost equally high positive means on the goals of mastery orientation and performance-approach orientation. The students in this group also scored the highest of all in the last section of the form as the results of the two-way ANOVA indicated (M = 12.3) and the students in the first, unfavorable, group the lowest (M = 7.5) supporting previous research findings by Huang [13].
This result is anticipated because, for work-avoidant students, the ultimate goal is to invest the minimum of effort aiming only at reaching a passing grade. The standardized difference between the means (Cohen’s ) of these two groups was 1.07 showing a strong effect size. Performance-avoidant oriented students were about in the middle in their mathematics performance which is in concordance with goal orientation theory. According to Skaalvik [11], students with strong “self-defeating ego orientation” (which is a corresponding construct to performance-avoidance orientation) are particularly concerned not to be the poorest performer, not to look stupid, and not to be perceived negatively by others.
Motivation clusters accounted for 7.5% of the variance in mathematics performance which is lower than that of students’ educational level factor’s explanation rate (26%). The ANOVA model revealed no interaction effect between the motivation and education level variables but showed that the relationship between motivation and mathematics performance is parallel in each of the educational level groups. This suggests that this relationship is independent of the students’ age.
The results of the present study cannot take a stand on the direction of influence or the causality of the relations between achievement goals and expectancy-value components. On the basis of earlier research (see [20]), we can expect a reciprocal relation between the variables. However, more research using a longitudinal research strategy and sophisticated statistical analyses in order to confirm these relations is needed.
Why is it important to study students’ motivation as an affecting factor in their mathematics performance? Further, is it possible to influence in the school setting these and their interrelationships? Lukin [6] conducted a longitudinal three-year survey () in a Finnish secondary school (grades 7–9) and found out that teachers and parents play a remarkable role in adolescents’ goal orientations and self-efficacy. These together with perceived support from these stakeholders influence their achievement in mathematics, thus corroborating previous research. In conclusion, an important message of our study for schools and teachers is that it is necessary to help students to develop positive perceptions of their competence and foster their learning-oriented motivation instead of concentrating solely on improving their cognitive abilities and skills.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
The authors thank Fr. Vladimir (William P. Lysak) for proofreading the language of the paper.