An assessment of numeracy in Brazil: a NeuroMat op-ed

Jan 31, 2020

by Fernando Jorge da Paixão


The international assessment made in 2018, promoted by the OECD, PISA, was widely disseminated in December 2019. It is a three-year exam that consists of three subjects, reading, mathematics and science. 600,000 15-year-old participated, chosen as a sample representing the 32 million students from the 79 countries participating in the test.

As in all previous occasions, our result was not good, and we are among the last positions among the participants --this result was massively covered by the media. The OECD, in its considerations, comments that Brazil, Mexico, Indonesia, Turkey and Uruguay have recently opened their secondary schools to include many students without loss of quality. We put more students in high school, and our performance remained loosely equivalent. This stagnation can be seen in the specific report on Brazil. It brings many comments on various results of our students and presents a figure with three graphs with our performance between 2000 and 2018 on each one of the topics. They point out that we have remained still.

To understand how the exam is done is important to make sense of a performance. Each test consists of questions designed for six increasing levels of difficulty. The first two, 1 and 2, are the simplest, the next two, 3 and 4, the medium, and the last two, 5 and 6, are advanced. One of the results that we should look at is the distribution of our students according to these levels. More than our general position among the last in the ranking, this distribution within our population is our major issue. Most of our students are concentrated in the lower levels, few are in the middle and only some are in the advanced positions. This distribution impacts several areas, including the economy.

Let's look at our performance on some questions chosen to represent complexity at each level. They are made available by PISA, here. We chose questions on mathematics, a topic of direct relevance to the RIDC NeuroMat, from the test that took place in 2012. For each of the questions chosen, the success rate of each participating country is presented.

The level 1 question requires the reading of a graph, and the student must realize in which month one of the bars is larger than the other. Success rates show us that there are 17 countries and 3 regions in China where the level of accuracy is between 99% and 95%; Brazil has 65%. At the next level, 2,the question involves calculating the average speed of a cyclist, with divisions and the manipulation of small numbers. 20 countries and regions have a score between 96% and 81%; Brazil has 33%. Question 3 requires one to read the engine capacity in a table and choose the smallest value, the difficulty is that the numbers have figures after the comma. 20 countries and regions have a score between 89% and 59%; Brazil has 13%. In question 4, one is asked to calculate the possible flow of people in a revolving door. The success rate of 20 countries and regions ranges from 76% to 33%; we reach 4%. Number 5 involves calculating the walking time on a mountain, including climbing and descending time in a situation in which the distance and speed are known. Success among the 20 first goes from 55% to 14%; Brazil has 1%. On question 6, again there is a situation with a cyclist, but more complex than number 2; 20 countries and regions reach a success score between 31% and 4%; Brazil does not show up among as a significant level of success.

Comparing the success rates of the top group of countries and ours, we see that as the level of difficulty of questions increases, the success in all countries decreases, but ours falls much more. Our drop is such that there is no performance on the level-6 question.

An assessment based on some questions is not precise. A more accurate measure is the average calculation of each decile of the distribution of our students. The result shows that our top 10% compare to the worst 10% in Canada or Vietnam.

Mathematics and Science are an invention of humanity, which can be understood as a language with which we can better understand the nature and events of our life. It helps to understand interest rates on a loan or an eclipse. Developing these skills is also directly related to an economic concept, human capital. This was emphasized by Theodore Schultz showing the importance of education in agricultural productivity, a research recognized with the Nobel Prize in Economics in 1979. More recently, a work by Hanushek and Woessmann showed the importance of the quality of education to the economic growth of several Asian countries.

How may the scientific community collaborate to improve our educational indicators? In recent years, funding agencies have required research groups to run scientific dissemination activities. Research activities can be a great motivation for learning. However, it is necessary to understand the relative capacity of a group of researchers to influence basic education. For example, the National Confederation of Municipalities estimated that in 2019 Fundeb, the fund for basic education, would reach the value of 156 billion reais. Even with much smaller resources, research groups can contribute.

NeuroMat has been dedicated to dissemination activities and I mention two of them here. The first activity is the improvement of entries related to our research on Wikipedia. As it is an open access environment, students can consult it for free. The quality of the entries is guaranteed by the involvement of researchers in their formulation and as a result the number of views of some of them reach up a few hundred thousand a year. In addition, it promoted in May 2017 at the Faculty of Education at USP an event on "Numeracy in Brazil: diagnoses and perspectives". This event was attended by Dr. Esther Carvalhaes representing the OECD-PISA whose presentation already indicated the stagnation perceived in the 2018 exam. In the context of Dr. Carvalhaes' visit, there were talks at the Faculty of Education at Unicamp and FEA-USP, seeking to motivate the community around this issue pertaining to education and also economics. 

How are we going to overcome this challenge? We have several successful experiences in Brazil. What is lacking is coordination, so that based on empirical evidence we manage to overcome this great difficulty. Unfortunately we are lacking this.

* Fernando Jorge da Paixão, a professor at the UNICAMP Institute of Physics, is the RIDC NeuroMat Scientific Dissemination coordinator. 

Share on Twitter Share on Facebook