Steps in validating a questionnaire
how can we ensure that the survey questions are valid and are really measuring what they intend to measure?Validity refers to the extent to which a construct (elements of information sought by a researcher) is adequately captured; this is sometimes referred to as a specification error.This provided further information on any uncomfortable or difficult questions. Using Cognitive Interviewing to Improve Medical Education Surveys.For this particular survey, 12 changes were made as a result of piloting, despite the survey having undergone cognitive interviews and expert reviews prior to that. I tend to send the survey out for another round of expert reviews, but this depends on whether the survey has been heavily modified since undergoing cognitive interviews.Although pilot testing may be useful for face-to-face surveys, this should not be used as the only validating method.
Expert reviews are where both questionnaire design experts and subject matter experts evaluate the content, cognitive, and usability standards of a questionnaire (Groves et al. This can highlight any problems the questions and scales may have in relation to its scope for example, or the way the questions may be understood or perceived by its participants, and evaluate whether the questions are clear and easy for participants to understand and answer (De Maio and Landreth 2004; Groves et al. Cognitive interviewing is a process in which participants respond to draft survey questions, and are asked to reveal their thought processes at the same time (Farrall et al. The premise is that knowing these thought processes and cognitive understanding can help the researcher evaluate the quality of the survey questions and responses. (2005) and Willis and Artino (2013), there is a growing body of literature that demonstrates that only a very few cognitive interviews can allow for the identification of problems with questions, and help in making revisions, so that the quality of the survey data can be greatly improved.Pilot testing tends to be used more to help with predicting the analysis of a large-scale survey.