A few insights from PISA 2022 – Wonder in Mathematics

In my last blog post, I focused on students’ beliefs about themselves and mathematics, commonly known as a “productive disposition”. To help us as teachers in fostering them in our students, I suggested considering four dimensions. Three are student-centred and relate to students’ thoughts about themselves mathematically, the mathematics they are learning, and how they engage with it. The fourth dimension, a productive environment, recognises that factors beyond students’ control—such as the classroom, home, and societal attitudes—can significantly affect their disposition toward mathematics.

I noted in the blog post that the 2012 Programme for International Student Assessment (PISA) results for mathematical literacy indicated that each of the following student-centred affective factors was important in relation to achievement in mathematics:

• intrinsic motivation (i.e. being interested in and enjoying mathematics)
• instrumental motivation (i.e. believing that mathematics is useful for their future)
• self-concept (i.e. believing that they are good at mathematics)
• self-efficacy (i.e. believing they can succeed with a particular mathematics task)
• responsibility for one’s own success in mathematics.

PISA is conducted every three years, with each cycle examining 15-year-olds’ literacy in mathematics, science and reading, and with a deeper focus on each core domain every three cycles. Consequently, mathematics has been a primary focus only three times: 2003, 2012, and 2022 (deferred from 2021 due to the pandemic).

So what did PISA 2022 tell us about students’ affective factors in relation to productive dispositions?

Frustratingly, PISA 2022 did not ask the same questions as PISA 2012 in relation to learning mathematics, meaning that we can’t directly examine changes in the affective factors mentioned above. However, it did ask about the following three student-centred factors in relation to their learning in general:

• Student resistance to stress
• Students’ perseverance
• Students’ curiosity

To measure each factor, an index was constructed using the responses of students to ten statements specific to each factor. All three factors were positively correlated with mathematical performance for Australian students (see the ACER report by de Bortoli et al., 2024 ). In other words, high-performing students in mathematics reported greater levels on these factors than low-performing students.

PISA 2022 also examined seven factors that may contribute to what I’ve termed environmental influences on productive dispositions. The first five relate to learning in general, and the last two are mathematics specific.

• Sense of belonging at school
• Student-teacher relationships
• Exposure to bullying
• Feeling safe at school
• Parental involvement in school activities
• Teacher support in mathematics classes
• Disciplinary climate in mathematics classrooms (i.e. classroom conditions)

Again, an index was constructed for each factor from students’ responses to a number of factor-specific statements. Six of the seven factors were positively correlated with mathematical performance for Australian students. The exception is that high-performing students in mathematics reporting less exposure to bullying than low-performing students. All except for parental involvement in school activities were found to be statistically significant.

While these results are perhaps unsurprising, they emphasise, yet again, the importance of addressing affective and environmental factors in relation to students learning mathematics.

What is alarming, though, are some of the gaps between high-performing and low-performing students in mathematics. To understand the disparity, it is important to outline how PISA results are reported. The scale was first designed in 2000 with a mean of 500 and a standard deviation of 100 across OECD countries. This gives a baseline for comparison across PISA cycles. In PISA 2022, Australian students achieved an average of 487 points (standard deviation of 96) in mathematical literacy.

In an Australian context, ACER have reported the mean mathematics scores for students in the highest and lowest quartiles of each index, as well as the difference. For example, students in the highest quartile for feeling safe at school have a mean mathematics score of 512 (25 above the average) whereas students in the lowest quartile have a mean mathematics score of 464 (23 below the average), resulting in a difference of 48 points.

I’ve put this information on the chart below. Each bar corresponds to one of the factors. (Note that I’ve excluded exposure to bullying and parental involvement.) The bar starts at the mean score for the lowest quartile and extends to the mean score for the highest quartile. The number on the right of the bar shows the difference. You’ll notice I’ve left off labelling the bars (that’s deliberate!).

In the spirit of a ‘slow reveal’ graph, can you work out which factor corresponds to which bar? The factors are below:

• Student resistance to stress
• Students’ perseverance
• Students’ curiosity
• Sense of belonging at school
• Student-teacher relationships
• Feeling safe at school
• Teacher support in mathematics classes
• Disciplinary climate in mathematics classrooms

When you are ready, scroll down to see the complete chart.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

Here’s the complete chart. How did you go? What surprises you? What do you notice? What do you wonder? You might like to add these in the comments.

Before I comment on the chart, it’s worth mentioning that I had to make some decisions about how to present the data. I wondered whether to order the factors by the size of the difference (as shown here), the mean mathematics score of the highest quartile, or the mean mathematics score of the lowest quartile. Each approach highlights different aspects of the data. (If you’d like to see the data organised in those alternative ways, click the links in the last sentence.)

My first takeaway is the importance of nurturing students’ curiosity. Although this factor is not specifically about mathematical curiosity, it clearly shows that curiosity and mathematics performance are intertwined. Students’ curiosity exhibits the largest disparity of all factors, with an 81-point difference between the mean mathematics scores of the highest and lowest quartiles, fairly evenly balanced around the Australian mean. This significant gap suggests that fostering curiosity is crucial. Furthermore, the scientific literature on curiosity shows that it is “more state than trait”, meaning it is highly responsive to the situation or environment we are in. Given that teachers play a pivotal role in creating these environments, how might we foster curiosity in a mathematics classroom?

Students’ perseverance also stands out as important, with a considerable difference of 61 points between the mean mathematics scores of the highest and lowest quartiles. The importance of this factor is unsurprising in relation to mathematics, particularly when examining the statements from which the index is constructed, for example: “I give up after making mistakes”, “I apply additional effort when work becomes challenging”, “I finish tasks that I started even when they become boring”, “I stop when work becomes too difficult”, “I give up easily”. (See the report for the other five statements.)

In relation to the other student-centred affective factor, I was surprised to see that resistance to stress was comparatively less important. While those in the highest quartile for resistance to stress performed, on average, 24 points above the Australian mean, those in the lowest quartile were only 1 point below. This isn’t to say that resistance to stress isn’t important for students, but it appears to be less of a factor in relation to differences in mathematics performance as measured by PISA.

Of the environmental factors, the disciplinary climate in mathematics classrooms is significant, with a 63-point difference. It’s important to note that ‘discipline’ here relates to classroom conditions such as noise, disorder, waiting for students to quieten, student attention, and distraction. I am hesitant to suggest implications for mathematics teaching from this data alone, but it is noteworthy that Australian students reported one of the least favourable disciplinary climates among the comparison countries that performed the same or better than Australia in mathematics in PISA 2022.

The other environmental factor that caught my attention was the importance of student-teacher relationships, which showed a 60-point difference, highlighting the critical role of having a teacher who students consider to be genuinely concerned about their wellbeing, friendly towards them, and interested in their lives.

While it’s worth keeping in mind that nearly 80% of Australian students reported they ‘didn’t fully try’ in PISA 2022, there is still a great deal to be learned from this data, particularly when examining different demographic groups. Overall, the results underscore the importance of fostering supportive and engaging learning environments in mathematics and encouraging curiosity and perseverance among students.

What are your key takeaways? I’d love to hear your thoughts.

References

De Bortoli, L., Underwood, C., Friedman, T., & Gebhardt, E. (2024). PISA 2022. Reporting Australia’s results. Volume II: Student and school characteristics. Australian Council for Educational Research. https://doi.org/10.37517/978-1-74286-726-7