On 7 December 2010, the results of PISA 2009 were released. PISA or the Programme for International Student Assessment is an international achievement test conducted once every three years by the Organization for Economic Co-operation and Development (OECD). First introduced in 2000, this study measures students’ ability to apply knowledge and skills in mathematics, science, and reading as they go about analyzing, interpreting and solving problems. The study is administered to a sample of 15-year-old students from each participating country. Two other prominent international achievement tests are the Trends in International Mathematics and Science Study (TIMSS) and Progress in International Reading Literacy Study (PIRLS). TIMSS is run by the International Association for the Evaluation of Educational Achievement (IEA) and is a study concerned with improving the teaching and learning of mathematics and science. The study involves Grades 4 and 8 students, and has been conducted every four years since 1995. PIRLS provides information on trends in reading literacy development among Grade 4 students. The study is into its third cycle, and measures reading comprehension as well as children’s home and school experiences when learning to read.
There is widespread international interest on the relative performance of countries following each test, leading to debates and discussions. Following the results of PISA 2009 in which Shanghai, participating in the study for the first time, became the top performing country in the reading, mathematics, and science literacy scales, there were intense debates in the USA, ranging from speculation on the reasons Shanghai was able to top the PISA ranks to whether certain interest groups were using the USA’s performance to launch a panic attack in their efforts to further a particular political agenda for education. Chester E. Finn, Jr., a former member of President Ronald Reagan’s education team, is said to have likened Shanghai’s stellar performance to a “sputnik” moment, a reference to 1953 when the launch of this satellite gave America reason to fear that the Russians had surpassed them in space technology. Other pundits were more blasé and less apprehensive about the ‘threat’ posed by the Shanghainese students; they attributed the surprising performance of Shanghainese children to an education system that conducts relentless test preparation under a highly centralized, highly pressurized education system.
Barely after the waves caused by the release of the results of PISA 2009 have subsided, another two international achievement studies in mathematics and science, and literacy are currently underway in 2011, this time involving Grades 4 and 8 students. This year, 2011, marks the convergence of the four-year TIMSS and five-year PIRLS cycles, where there will be a “comprehensive assessment” of Grade 4 students from participating countries in all three areas of learning. Over 50 education systems have committed themselves to this once-in-twenty-years international testing event. While a number of systems are assessing the same students for both TIMSS and PIRLS, others are using the same sample of schools but arranging for different classes of students to take TIMSS and PIRLS. Interspersed between the cycles of international achievement tests, most education systems would typically conduct internal assessments, either informally as classroom tests, or more formally as standardized tests like the International General Certificate for Secondary Education in the United Kingdom, or the mandated state tests under the No Child Left Behind law in the USA. Overall, it is evident that large numbers of students around the world are subject to frequent and numerous tests. The questions I ask myself are why these international tests generate so much interest, and more importantly, given the frequency, scale and extent of these studies, how can policy makers and educators best make sense of and use the data. Making cautious inferences and deliberate decisions from the test information is a way to account for the large sums of public money spent on the tests – for instance, the total participation cost per country for PIRLS 2006 was US$120,000 for the duration of study. Evidently, there are other intangible costs involved as well, such as time and human resources spent implementing the tests.
I was prompted to ruminate on the influence of these international tests because of two recent articles which I read over the last fortnight. Both articles make direct or indirect reference to the significance and implications of the East Asian education systems topping the international tests, but with similar purposes: the authors contribute suggestions for improving education systems, recommending broad approaches that countries may adopt. Second, they also emphasize that the end goals of educational change should be supporting all students to attain high levels of achievement. Ben Jensen (The Australian, 29 July 2011) makes the case for Australian policymakers undertaking educational change to learn from the East Asian education systems, namely Hong Kong, South Korea, Singapore and Shanghai, and from Finland, the only non-East Asian jurisdiction mentioned in the article. Finland has topped the PISA tables for the 2000, 2003, and 2006 cycles. Jensen draws on data from PISA 2009 to raise the alarming news that the average 15-year old student in Australia was lagging behind his Shanghainese counterpart by about two years of learning. Despite Australia being geographically close to the East Asian nations, he laments that Australian policymakers appear reluctant to learn from her high performing neighbours because of a lack of understanding of the nature and design of the achievement tests, as well as ignorance about the educational reforms taking place in recent years in the East Asian systems. To this end, educational reform in Australia is needed, and he urges Australian policy makers not to dismiss lessons which may be drawn from the East Asian systems. He praises the fact that in the East Asian education systems, there is a teacher career structure which sees the most effective teachers desiring to work in the least advantaged schools.
While not dismissing the validity of international tests, Yong Zhao (Leading the Change Series, August 2011) urges U.S. policymakers not to simply rely on single measures of achievement, and in particular, he presses educators to look beyond the current “test-driven incentive system”. In his view, it is not appropriate or feasible to transplant models of educational reform from one country to the next; rather, he asserts that thorough knowledge of the context is critical. For instance, while he recognizes that Finland, South Korea and Singapore are high performing countries in PISA 2009, their education systems are roughly the size of the state of Massachusetts and hence, their strategies may not be appropriate for the USA. In addition, the cultural, economic, social and geographic contexts of these systems are vastly different from the USA. Related to this, he asserts that policymakers need to develop a deep awareness of the different systems rather than persist with stereotypical views of other systems, a point also raised by Jensen. One of Zhao’s proposals is for researchers to look broader beyond a narrow emphasis on instructional interventions to focus on larger, systemic reform. In his view, educational change must be accompanied by longer term objectives; currently, there is myopic reverence for short term outcomes based on a narrow set of indicators.
A common observation from both authors is that education change in some OECD countries (USA and Australia in the two articles) is moving towards increasing centralization (and standardized testing) while education change in the Asian countries is headed for more flexibility, with more autonomy being devolved to schools. The articles resonated with me because the Singapore education system – one that I grew up in, and later taught in – was mentioned in both articles. Further, I am also interested in educational change and reform. Reflecting on the two articles, I pondered over several topics related to educational change, as well as data from the international tests. Responding to the two authors, as well as drawing from findings in PISA 2009, I have further suggestions for the Australian and US policymakers.
The first topic is related to issues of equity and the achievement gap. Zhao attributes the persistent achievement gap in the USA to disparity issues contingent on where students live and on their social economic status. In this respect, I suggest that the contextual data collected in addition to the achievement booklets in the international tests play an important role in educational change. The contextual data comprise self-report responses from students, teachers, principals, and parents with respect to a wide range of educational indicators, including experience of learning, use of instructional and assessment activities, details of homework activities, home and school provision of educational resources, and teacher efficacy. I also contend that trend analysis of the contextual data will help countries determine patterns of educational change in their systems. The analysis of the contextual data collected from PISA 2009 (and also from previous cycles) shows that across countries, students from lower SES backgrounds spend less time learning science than their more wealthy classmates. These trends are especially pertinent in light of the disparities resulting from globalization. However in a number of countries, “resilient” students in the bottom SES strata were also predicted to score within the top quarter of the achievement scale. As reported in the PISA 2009 Results: Executive Summary, such a pattern was evident based on the statistical analyses conducted for Finland, Japan, Turkey, Canada, Portugal and Singapore where 39 to 48% of disadvantaged students were “resilient”. In Hong Kong and Shanghai, 72% and 76% of students were “resilient” respectively. These findings suggest that despite coming from less privileged backgrounds, students from these countries had access to quality education, as well as were given opportunities to excel academically in mathematical and scientific literacy. Further, there were also countries which devoted a larger proportion of teachers to schools serving low SES populations than to the advantaged schools. Such data will be of interest to countries like Australia where, as Jensen mentions, students from the lower SES backgrounds are more likely to drop out of school, and have deteriorating grades than their East Asian peers. In fact, in my view, given the widening disparities resulting from globalization, studying the reform strategies used by countries with high “resilience” will help to ensure greater equity and accessibility in the distribution of education resources to help every child succeed.
The second topic relates to the within-country use of international test data. Rather than be preoccupied with international comparisons, I suggest that countries scrutinize trend data and use them as indicators when conducting internal reviews of their own systems. Such data are available every 4-5 years if an education system participates in each successive test cycle. Schleicher presents the case for the international achievement data to be used as indicators of widening disparities as well as levers for educational change. For example, he argued that following judicious statistical analyses of the achievement results and other contextual data from PISA 2000, the German government was made cognizant of the deepening impact of socio-economic segregation in German schools: privileged students from higher SES backgrounds were channeled into the elite academic track while students from lower SES families were directed towards the vocational schools. This finding raised awareness that the sorting approach in German’s education system was deepening the socio-economic divide rather than moderating socioeconomic background factors. International benchmarks may also be used as critical levers to spur educational change. Using Japan’s performance in PISA 2003 as an example, Schleicher illustrated how the analysis of student performance led Japanese policymakers to review the curriculum and propose changes for modifying the country’s assessment system. The outcome was a recommendation to incorporate more open-ended tasks in the national examinations because Japanese students were weak in this aspect of literacy. A similar approach was undertaken in Korea which recently increased the emphasis on literacy tasks for its university entrance examinations, arguing that these tasks were critical as part of 21st century skills. In both instances, Schleicher contends that the international test and contextual data brought attention to areas requiring review within the education system, and became the impetus for urgent and immediate education change; such revisions might have faced fierce opposition and resistance if proposed without the evidence provided in international benchmarking studies.
Finally, Zhao’s recommendation for policymakers and researchers to concentrate on the purpose of education is especially pertinent. Education is certainly more than achievement test scores and the meeting of policy targets like Adequate Yearly Progress set by the U.S. Department of Education. In his interview, Zhao speaks of the need for countries to embrace or adopt an educational philosophy. He cites the objectives of China’s recent education reform which is shifting the emphasis from knowledge acquisition to focus on four areas: ability to create, ability to apply, move towards flexibility, and desire for students to develop global perspectives. Likewise, Jensen urges Australian stakeholders in education to cast away their stereotype (mis)perceptions of East Asia education; in particular, the general (mis)conception that there is undue emphasis on rote learning in East Asia. While this may have been so in the past, both Jensen and Zhao note that recent educational change efforts in the past decade in these education systems have seen a move towards more progressive views: as the new policies are more child-centered, there is a recognition that each child is different, as well as an overarching effort to reduce the reliance on test scores as a measure of success. In response to Zhao, I agree that developing an educational philosophy is useful, but I also suggest that this must be complemented by deliberate efforts to ensure continuity in the policy. In many countries, this permanence in policy is typically not easily achieved because with each successive government change, the pendulum swing in education reform may take diametrically opposite philosophies, objectives and approaches.
With respect to the continuity of educational policy, I would like to illustrate my thoughts using Singapore as an example. In Singapore, the outcomes of education are underpinned by the Desired Outcomes of Education (DOEs), which are key attributes that society aspires for each Singaporean student to have at the end of formal education. These outcomes are the indicators of the vision of education, Thinking Schools Learning Nation. Since 1997, these DOEs have guided policymakers and educators when conceptualizing policies, initiatives and programmes. For a teacher like me teaching in the system, it is important that the overall education vision has been consistent over the last fifteen years. However, the strategies for meeting these goals continue to be refined and reviewed based on feedback from schools, teachers, researchers, students, and the public. To this end, the constant review of processes and targets is a means of ensuring that goals are realistic, and that schools and teachers have the necessary support to implement the policy intent. For teachers who have to implement the policies, the constant renewal and review may be a bane because they have to keep making new adjustments. Nevertheless, because the DOEs have been in existence since 1997, a sufficiently long period of time has transpired for the policy instruments to take shape. To date, the three phases in Singapore’s young education system – survival-driven, efficiency-driven, and ability-driven – have progressed and transitioned seamlessly, with each phase lasting 15 to 20 years. This consistency in the system may be attributed to a stable social and economic arena because the same political party has been leading the country since independence in 1965. As such, developments within each phase are part of a deliberate and concerted movement towards the overarching education philosophy. For example, since 1979, there has been a consistent shift from autocracy to more school autonomy, and from uniformity in the system to more diversity. Finally, because of the consistent reviews, curriculum and education planners are more aware of the support structures required to help teachers realize the policy intent. To this end, where previously teachers had to grope in the dark to decipher and interpret the intent and meaning of particular policies, with new initiatives like Teach Less Learn More which were introduced in 2006, there are mini-descriptors that help teachers understand the spirit and intent of the policy. As mentioned in both articles, Singapore’s education system is now increasing its emphasis on creativity and independent thinking, the present challenge for planners is to locate the balance between central and local control of curriculum and assessment.
Educational change is a complex process. While student achievement in international test data provides an indication of each education system’s relative standing with respect to others, there are many other indicators which need to be thoroughly examined before implementing changes. Although valuable lessons can be drawn from the high performing education systems, it is also important to bear in mind that what works in one jurisdiction is not transferable or replicable in another due to different economic, social, cultural, and political contexts.