Last night a doctor from the School of Labor and Human Resources called me to ask for advice on their research project. Judging from her voice and tone, I guess she was almost completely at a loss then (even a little bit angry).
The reason is clear – they’ve encountered great obstacles ever since the beginning of data analysis, and what’s worse, these obstacles are NOT from statistical models, but rises from the data collected using an ill-designed questionnaire.
If you have no hand you can't make a fist.
Many people tend to believe that designing a questionnaire is just something on tactics (how to get data instead of what to get). I can understand this belief, because they’re short-sighted from a view of the complete procedure of statistical analysis: collection, classification, analysis and interpretation. All of these four components are equally important.
I don’t know why they didn’t pay enough attention to this simple philosophy in designing their questionnaire: just connect the research purposes with questions/items (and options). When they got on with the analysis, they found the data had nothing to do with their aims. Ironic, isn’t it?
This is not the first case I’ve met. And they are doctors and teachers, which indicates a great lack of basic recognition on the position of data collection: strategy or tactics?