June 2, 2016, by Editor
Survey Fatigue and the Search for ‘Good’ Data: post-disaster strategies
Written by Claire L. Berja.
Leyte in the Eastern Visayas of the Philippines was one of the areas hardest hit by Typhoon Yolanda in 2013. Tacloban, the city that became the ‘poster town’ of the disaster, is located in Leyte facing the Pacific Ocean at the head of the Leyte Gulf. Leyte is one of the poorest provinces in the Philippines. There is a high incidence of poverty and many people also move in and out of poverty (transient poverty) due to a high degree of vulnerability to shocks.
Typhoon Yolanda left many people devastated. It did not discriminate by class. In the aftermath of disasters the wealthy tend to be able to rehabilitate themselves more quickly, as they may have savings or extended family support to fall back on. However in the immediate aftermath of Typhoon Yolanda the devastation was total in many areas. In the longer term the disaster increased poverty overall.
Post disaster relief efforts require good data. Good data helps aid agencies efficiently target the victims of disaster. It also helps to scale up rehabilitation efforts in a sustainable fashion. The need for good data probably explains why aid agencies conducted their own surveys in areas devastated by Typhoon Yolanda. The specific data needs of aid agencies were not otherwise readily available. Immediately after the typhoon the relief operations of UNICEF and the World Food Programme (WFP) made use of the national household targeting system of the Department of Social Welfare and Development (DSWD). However, the list included only the beneficiaries of the government run conditional cash transfer program. There was no list of non-beneficiaries.
Two years after the Yolanda disaster our team gathered data on poverty alleviation in selected communities in Tacloban, Palo and Tanauan. We selected 20 barangays, 17 most affected and three least affected. We surveyed 800 households and conducted a number of focus groups. I expected that data gathering would be challenging because of the sensitivity of the subject matter but in general people were willing to participate in our study. Many of respondents were able to answer questions swiftly but some expressed survey ‘fatigue’. I observed that surveys had become a routine for beneficiaries of Yolanda aid. Many respondents considered surveys as their “ticket” to aid or a way to be included in the “listahan” (list of those who will be given aid) so they participated even with hesitance.
It is important to address survey fatigue because it could severely affect the quality of the data gathered. Having gone through different types of surveys, including rapid needs assessment surveys, social surveys, impact surveys and evaluation surveys, in all post disaster phases respondents can understandably develop survey fatigue. This can lead to overly brief answers or responses that simply follow the path of least resistance.
Survey fatigue can also be experienced during the course of an interview. Considering that the average attention span of adults is only 20 minutes, long interviews must be avoided. In our survey, to minimize the length of interviews, we included only questions relevant to the research objectives. Questions were worded simply so that they were easy to understand. The logical and chronological sequencing of questions also helped our respondents to answer the survey. This also helped respondents to remember facts. Interviews rely a lot on memory recall so this also impacts on data quality. It is the responsibility of the field researchers to come prepared for the interviews. Survey interviews go smoothly when the interviewer knows what questions to ask. Clear questions result in clear answers and comparable data.
The quality of the research is also contingent on trust. Building a relationship of trust is important for community based research. Our project is set to run over three years (2015-2018). Our fieldworkers will be regular visitors so it is important that we are accepted in and trusted by the communities. People share their experiences, thoughts and feelings only to those whom they trust. Honesty and integrity throughout the study is important. We make it clear to the respondents that they will not receive any aid or grant, bar a token gift for their time, for participating in the research. We are also aware that the best people to conduct surveys at the community level are members of the communities themselves. For this reason we have hired recently graduated students from the communities to help with data gathering. Foreigners and even researchers from Manila could distort survey respondent answers simply by their presence. There is the danger that respondents could exaggerate their plight in the expectation that aid would be forthcoming.
During training sessions we were also able to tap into the fieldworkers’ local knowledge of how questions would be perceived. For instance they pointed out that the distinction between community and family is blurred in many communities as extended families are the norm. Therefore the distinction between helping your family and helping your community is a blurred line. Many respondents are also most comfortable speaking in the local Waray language so it made sense to hire fieldworkers with approprate language skills.
We also deliberately avoided working in our chosen communities during the recentelection period as respondents could have tailored their answers to favour their preferred candidates or condemn those that they did not support. The distribution of goods to voters during the election campaign could also have rendered our token gifts less attractive to respondents.
It is important that aid agencies and academics consider the issues listed above as ‘rubbish in’ leads to ‘rubbish out’ for data gathering. Good data on the other hand can help identify real need and craft sustaianble rehabilitation strategies that are guided by the beneficiaries themselves.
Claire L. Berja is an Assistant Professor at the University of the Philippines, Manila. She is currently working on the ESRC/DFID funded project ‘Poverty Alleviation in the Wake of Typhoon Yolanda’. You can follow this project on Facebook as Project_Yolanda and Twitter @Project_Yolanda. Image credit: CC by Dvidshub/Flickr.