search this site.

200225P - QUESTIONNAIRE DESIGN

Copyright 1995-2023 © by Prof Omar Hasan Kasule - Admin
Category:
Print Friendly and PDFPrint Friendly

Prepared for a course on February 25, 2020 by Prof Omar Hasan Kasule Sr Professor of Epidemiology and Bioethics, King Fahad Medical City. 


SCOPE OF QUESTIONNAIRE DESIGN

  • Content
  • Wording
  • Format and layout
  • Pilot testing


CONTENT

  • Enough time and effort to prepare a questionnaire
  • Decisions on what to include in the questionnaire based on the hypothesis: independent variable, dependent variable, and confounding variable
  • Start by reviewing questionnaires of similar studies but remember to cite if you use their items and/or ideas
  • Each item must have an objective:


VARIABLES OF THE QUESTIONNAIRE

Types of variables

  • Potential confounding variables: socio-demographics (age, gender, SES etc)
  • Independent variables
  • Dependent variables


Classification of variables

  • Qualitative
  • Quantitative discrete
  • Quantitative continuous


OBJECTIVES OF QUESTIONNAIRE ITEMS

  • Knowledge
  • Attitude
  • Belief
  • Experience
  • Behavior
  • Attributes


UNIQUE OBJECTIVE FOR EACH ITEM

  • An item must ask from only one objective
  • No multiple objectives in an item
  • Each item must have a unique answer; no multiple valid answers
  • It is better to use many items than combining related items in one
  • The gold standard is that the respondent should not have to guess or work out what you want.

 

WORDING OF QUESTIONS

  • The objectives of questionnaire wording are clarity, comprehensibility, neutrality, and proper scaling. 
  • The questions must be worded properly to make sure they are easy to understand. 
  • The wording of the questionnaire items should leave no room for ambiguity. 
  • Words used must be easy and common
  • Technical jargon must be avoided. 
  • The wording should be neutral, neither positive nor negative so that the respondent is free to answer. 
  • Biased questions (leading, threatening, value-laden, or assumptions) should be avoided.
  • Each question should contain only one concept or item of information; questions should not be double-barreled. 
  • The responses must be scaled appropriately. 
  • Double negatives should be avoided.


FORMAT AND LAYOUT

  • The format and layout of the questionnaire are important and have an impact on response rates. 
  • The order of the questions must be logical moving from the superficial to the more detailed. Embarrassing questions should be kept towards the end because they may spoil the whole interview. 
  • Closed questions are preferred to open questions. 
  • Questions should not be too long. 
  • The total number of questions must be appropriate. 
  • The questionnaire should be designed for easy reading. 
  • The use of boxes and different colors helps. 
  • The font print size must be readable. 
  • Skip patterns should be worked out carefully and exhaustively. 


PILOT TESTING

The reliability and validity of the questionnaire should be tested during the pilot study. 

  • Pilot testing also has the additional advantage of testing the study procedures and administrative bottlenecks. 
  • The following are common problems in questionnaires that can be uncovered during pilot testing: 
    • Ambiguous questions, 
    • Questions that are not self-explanatory, 
    • Two questions in one, 
    • Use of unfamiliar words, 
    • Asking for events that are difficult to remember, 
    • Insufficient number of response categories, 
    • Overlapping categories, 
    • Questions that are too long, 
    • Questions that have too many ideas, 
    • Questions that require too much detail, 
    • Leading questions, 
    • improper use of rating scales.


ETHICAL and CONFIDENTIALITY ISSUES

• Informed consent must be obtained. 

• The information provided could be subpoenaed by a court of law and the investigator cannot refuse to release it. 

• In the course of the interview the investigator may get information that requires taking life-saving measures. Taking these measures will however compromise confidentiality. 

• What do you do if the respondent tells you he is planning to kill himself or kill someone else?


PREPARATION FOR DATA COLLECTION - 1

• Data collection processes must be clearly defined in a written protocol which is the operational document of the study. 

• The protocol should include the initial version of the questionnaire. This can be updated and improved after the pilot study. 

• If a paper questionnaire is used data transfer into the electronic form will be necessary. The need for this could be obviated by the direct online entry of data. 

• The objectives of the data collection must be defined clearly. Operational decisions and planning depend on the definition of objectives. It is wrong to collect more data than what is necessary to satisfy the objectives. 

• It is also wrong to collect data just in case it may turn out to be useful. 


PREPARATION FOR DATA COLLECTION - 2

• The study population is identified. The method of sampling and the size of the sample are determined. 

• Staff to be used must be trained. The training should go beyond telling them what they will do. They must have a sufficient understanding of the study that they can detect serious mistakes and deviations. 

• A pilot study to test methods and procedures should be carried out. However well a study is planned, things could go wrong once fieldwork starts. A pilot study helps detect and correct such pitfalls. 

• A quality control program must be part of the protocol from the beginning. 


PREPARATION FOR DATA COLLECTION - 3

• Proxy or surrogate respondents must be used when the subject is handicapped or is not available. The next of kin is usually selected for this. Sometimes the subject and the proxy may disagree. 

• In some case-control studies, dead controls are selected for dead cases and proxies are interviewed for both series. 

• Response can be increased by obtaining sponsorship by the government or some other official body that credibility to the study. 

• Participation is also maximized by short follow-up periods, regular feedback, and causing the participants as little inconvenience as possible. 


QUESTIONNAIRE ADMINISTRATION BY FACE-TO-FACE INTERVIEW - 1

• In a face-to-face interview, the interviewer reads out questions to the interviewee and completes the questionnaire. 

• The interview may be structured or unstructured. 

• The interviewer should make sure that circumstances of the interview are optimal in terms of place and time. 

• The interviewers should be selected carefully and adequately trained. They should be given an interviewer’s manual to guide them. 

• It is important that interviewers are continuously monitored. 


ADVANTAGES OF QUESTIONNAIRE ADMINISTRATION BY FACE-TO-FACE INTERVIEW - 2

• The interviewer can establish the identity of the respondent. In the mailed questionnaire, the answers may be from another person other than the intended respondent. 

• There are fewer item non-responses because of the presence of the interviewer who will encourage and may coax the respondent to answer all items. The interviewer can clarify items that the respondent does not understand or is likely to misunderstand.

• There is flexibility in the sequence of the items. 

• Open-ended questions are possible 

• Items irrelevant to the particular interviewee can be dropped thus saving time. 


DISADVANTAGES OF QUESTIONNAIRE ADMINISTRATION BY FACE-TO-FACE INTERVIEW - 2

• It costs more in terms of time and money. The interviewer has to travel, search for, and spend time with the respondent. 

• A prior appointment is needed to ensure that the respondent will be available at the place and time of the proposed interview. 

• Personal chemistry may not work well. The interviewee may resent the interviewer on the basis of gender, ethnicity, or any other personal and behavioral characteristic.

• The presence of the interviewer may influence interviewee responses in a subtle way. The interviewee may try to give responses that he thinks are acceptable to the interviewer on the basis of the interviewer's gender, race, SES, and suggestive questioning. (

• The common errors in face-to-face interviews are omitting a question, too much or too little probing, failure to record information, and cheating by the interviewer.


ADVANTAGES OF QUESTIONNAIRE ADMINISTRATION BY TELEPHONE

• Considerable savings in time and money. It is possible to conduct a nationwide survey sitting in one office. 

• Has fewer items non-responses because of the personal contact involved. (c) Skip patterns can be followed to save time. 

• Difficult questions can be explained. 

• Interviewer bias is less than in face-to-face interviews. 


DISADVANTAGES OF QUESTIONNAIRE ADMINISTRATION BY TELEPHONE

• Selection bias may operate when the study sample includes only those who have telephones and the telephone numbers are listed. The problem of unlisted numbers can be overcome by the use of random digit dialing. 

• Selection bias may arise due to the day and time of day that the telephone call is placed. Office workers will be missed in early morning calls. Workers on night shifts will be missed in evening calls.  

• It is not possible to be sure whether the person at the other end of the line is the actual intended respondent. 

• Telephone interviews must also be supervised for optimal results. The supervisor should listen in as the interview is conducted. 


ADVANTAGES OF QUESTIONNAIRE ADMINISTRATION BY TELEPHONE AIDED BY THE COMPUTER

• Telephone interviews can be improved by the use of computers. 

• Computer-assisted telephone interviews can make the process quicker when the interviewer is prompted by the computer. 

• The computer will work out the skip patterns and will alert the interviewer to responses that are inappropriate or contradictory. 


QUESTIONNAIRE ADMINISTRATION BY E-MAIL

• In the method of questionnaire administration by mail, a questionnaire is e-mailed to the respondent's email address. The respondent completes and returns the questionnaire by email

• Questionnaire administration by e-mail has 2 main advantages: 

o It is the cheapest method of data collection 

o There is no bias due to interviewer involvement. 

• Questionnaire administration by e-mail has main disadvantages: 

o Low overall response 

o Higher item non-response 

o Delays in returning the questionnaire. 

• Measures are undertaken to increase response to e-mailed questionnaires: 

o Sending the questionnaire with a personalized cover letter. 

o Promising a token of appreciation for return of the questionnaire. 

o Making the questionnaire anonymous by not including any information on the returned questionnaire that can be used to identify a particular individual. 

o Using pre-coded questionnaires so that all the respondent has to do is to select responses.

o Follow up e-mail for those who delay in returning the questionnaires.


ADVANTAGES OF COMPUTER-ADMINISTERED QUESTIONNAIRE:

• It frees the interviewer's time.  

• There are no transcription errors because the information is entered online. 

• No items are missed because the computer will not allow the respondent to move to the next item before answering the previous one. 

• The respondent can give more honest responses when facing an anonymous computer than when faced by a human interviewer. 


ADVANTAGES OF COMPUTER-ADMINISTERED QUESTIONNAIRE:

• The respondent does not have the opportunity to vary the order of questions to his convenience