B1419
Title: Identifying periods of careless responding in surveys: A deep learning approach
Authors: Max Welz - Erasmus University Rotterdam (Netherlands) [presenting]
Andreas Alfons - Erasmus University Rotterdam (Netherlands)
Abstract: Rating-scale datasets collected from surveys are paramount to empirical research. However, some respondents may not comply with survey instructions due to deficiencies in survey design or lack of motivation. This phenomenon is known as careless responding (CR). CR is a major threat to internal validity and should therefore be screened for. Existing methods for detecting CR are designed to identify respondents who respond carelessly throughout the survey. However, recent work suggests that the longer a survey takes, the higher the likelihood that a large proportion of all respondents will eventually start responding carelessly. Thus, we are interested in identifying when a respondent becomes careless (if at all) rather than trying to detect respondents who respond carelessly throughout the survey. Correspondingly, we propose a novel method for identifying the periods of carelessness (or a lack thereof) of each respondent. The proposed method uses the deep learning technique of autoencoders in combination with response times. By means of extensive numerical experiments, we find that our proposed method achieves high reliability in correctly identifying periods of careless responding and discriminates well between careless and regular respondents. Our method seems to perform particularly well in long surveys, which are common in psychology and health sciences, where it is likely that a large proportion of all respondents eventually responds carelessly due to fatigue.
