Home 
Contact 
Profile 
Videos 
Workshop at a pre-conference workshop on Medical Education Research March 13 2010 at the American College of Physicians Conference under the theme ' ' Prepared by Dr Omar Hasan Kasule MB ChB (MUK), MPH (Harvard) DrPH (Harvard) Professor or Epidemiology and Bioethics faculty of Medicine King Fahad Medical City, Visiting Professor of Epidemiology University of Malaya
ABSTRACT
CONTENTS
1.0 Introduction
1.1 Objectives of the workshop
1.2 Scope of the workshop
2.0 Principles of questionnaire design
2.1 Content
2.2 Wording
2.3 Format and layout
2.4 Pilot testing
2.5 Ethical considerations
3.0 Methods of questionnaire administration
3.1 Questionnaire administration by face-to-face interview
3.2 Questionnaire administration by telephone
3.3 Questionnaire administration by mail
3.4 Computer-administered questionnaire:
4.0 Preparation for data collection:
4.1 Research Protocol
4.2 The objectives of the data collection
4.3 The study population
4.4 Staff training
4.5 A pilot study
4.6 A quality control program
4.0 CHECK LIST FOR INSTRUMENT DESIGN
4.1 Basic Questions for Design
4.2 Format
4.3 Content of questions
4.4 Wording of questions
4.4 Sequence of questions
4.5 Form of responses
4.6 Pretesting the instrument
5.0 SCALES OF QUESTIONNAIRES
5.1 Construction of the agreement scale
5.2 Steps for constructing and using a semantic differential
5.2 The ordered scale
6.0 VALIDATING AN ASSESSMENT TOOL
6.1 Validity and reliability
6.2 Specific actions or responses
6.3 Utilize the definition to operationalize the construct
6.4 Analyze
6.5 Consider defining appropriate outcomes and evaluating for these.
6.6 Make the assessment methodology as close to reality as possible.
6.7 Summary of process (five steps)
7.0 ASSESSING RELIABILITY OF AN INSTRUMENT
7.1 Use the same instrument with the same people taking it on 2 occasions
7.2 Administer two equivalent forms of the instrument to the same population.
7.3 Compare groups' performance
8.0 QUALITY CHECKLIST FOR INSTRUMENTS
8.1 Introduction
8.2 Item stems
8.3 Responses
8.4 Format
8.5 Directions
9.0 DATA COLLECTION (From ACP MERC)
9.1 Collection and presentation of data
9.2 Pros and Cons
9.3 Getting data back
9.4 Structured interviews
9.5 Information collectors:
10.0 PRACTICAL EXERCISES IN QUESTIONNAIRE CONSTRUCTION
10.1 Basic Class data
10.2 Attitudes, experiences, and professional expectations of general surgery residents
10.3 Patient Safety Education at US and Canadian Medical Schools
10.4 Medicine Residents Understanding of the Biostatistics and Results
10.5 Medical Students' Experiences of Moral Distress:
10.6 Evaluating Obstetrical Residency Programs Using Patient Outcomes
10.7 Effect of Communications Training on Medical Student Performance
10.8 Comparison of the instructional efficacy of internet-based CME
11.0 CRITIQUE OF CANCER KAP QUESTIONNAIRES
11.1 Personal and social profile
11.2 Smoking experience
11.3 Knowledge of cancer symptoms, signs, diagnosis, and treatment
11.4 Symptoms and signs of cancer
11.5 Knowledge of cancer risk factors
11.6 knowledge of cancer prevention measures
11.7 opinions about cancer
11.8 Attitudes about cancer
11.9 Health-seeking behavior
12.0 CRITIQUE OF ETHICS KAP QUESTIONNAIRES
12.1 Perceptions of bioethics & health service law --1
12.2 Perceptions of bioethics & health service law --2
12.3 Cultural / religious issues in clinical practice -1
12.4 Cultural / religious issues in clinical practice -2
1.0 INTRODUCTION
1.1 Objectives of the workshop: This workshop will provide some basic principles in questionnaire/survey design and give workshop participants an opportunity for hands-on experience designing a questionnaire. Following participating in this workshop, learners will be able to: (a) Design a blueprint for a survey/questionnaire appropriate to their own application; (b) Construct and edit questions to avoid common problems in wording and framing; (c) Select an appropriate response format from a menu of alternatives; (d) Design the overall format of the survey/questionnaire to facilitate data management and analysis.
1.2 Scope: In order to achieve the objectives above the workshop will cover aspects of research using questions that go beyond the immediate objectives. This will include
2.0 PRINCIPLES OF QUESTIONNAIRE DESIGN
2.1 Content
Utmost care should be taken in preparing the study questionnaire. Correct decisions must be made about what items to include in the questionnaire. This is guided by the hypothesis under study and knowledge of potential confounding factors. A start is made by reviewing questionnaire of previous similar studies. The content of a question may be one of the following: knowledge, attitude, belief, experience, behavior, and attributes.
2.2 Wording of questions
The following should be observed in selecting questionnaire items: clarity, comprehensibility, neutrality and scaling. The question must be worded properly to make sure they are easy to understand. The wording of the questionnaire items should leave no room for ambiguity. The words must be easy. Technical jargon must be avoided. The wording should be neutral, neither positive nor negative. Biased questions are leading, threatening, value-laden, or assumptions. Each question should contain only one concept or item of information; questions should not be double barreled. The responses must be scaled appropriately. Double negatives should be avoided.
2.3 Format and layout
The format and layout of the questionnaire are important and have an impact on response rates. The order of the questions must be logical moving from the superficial to the more detailed. Embarrassing questions should be kept towards the end because they may spoil the whole interview. Closed questions are preferred to open questions. Questions should not be too long. The total number of questions must be appropriate. The order of the questions must be logical moving from the superficial to the more detailed. Embarrassing questions should be kept towards the end because they may spoil the whole interview. Closed questions are preferred to open questions. Questions should not be too long. The total number of questions must be appropriate. The questionnaire should be designed for easy reading. Use of boxes and different colors helps. The font print size must be readable. The logical sequence of questions must be proper. Skip patterns should be worked out carefully and exhaustively.
2.4 Pilot testing
The reliability and validity of the questionnaire should be tested during the pilot study. Pilot testing also has the additional advantage of testing the study procedures and administrative bottle necks.
The following are common problems in questionnaires that can be uncovered during pilot testing: ambiguous questions, questions that are not self-explanatory, two questions in one, use of unfamiliar words, asking for events that are difficult to remember, insufficient number of response categories, overlapping categories, questions that are too long, questions that have too many ideas, questions that require too much detail, leading questions, improper use of rating scales.
2.5 Ethical and confidentiality issues
Before administering a questionnaire the investigator should be aware of some ethical issues. Informed consent must be obtained. The information provided could be sub-poened by a court of law and the investigator can not refuse to release it. In the course of the interview the investigator may get information that requires taking life-saving measures. Taking these measures will however compromise the confidentiality. Such a situation may arise in case of an interviewee who informs the interviewer that he is planning to commit suicide later that day. Such information may have to be conveyed immediately to the authorities concerned.
3.0 METHODS OF QUESTIONNAIRE ADMINISTRATION
3.1 Questionnaire administration by face-to-face interview
In a face-to-face interview, the interviewer reads out questions to the interviewee and completes the questionnaire. The interview may be structured or unstructured. The interviewer should make sure that circumstances of the interview are optimal in terms of place and time. The interviewers should be selected carefully and adequately trained. They should be given an interviewer’s manual to guide them. It is important that interviewers are continuously monitored.
Face to face interview has the following advantages: (a) The interviewer can establish the identity of the respondent. In mailed questionnaire the answers may be from another person other than the intended respondent. (b) There are fewer item non-responses because of the presence of the interviewer who will encourage and may coax the respondent to answer all items. (c) The interviewer can clarify items that the respondent does not understand or is likely to misunderstand. (d) There is flexibility in the sequence of the items. (e) Open-ended questions are possible (e) Items irrelevant to the particular interviewee can be dropped thus saving time.
Face-to-face questionnaire administration also has disadvantages: (a) It costs more in terms of time and money. The interviewer has to travel, search for, and spend time with the respondent. (b) A prior appointment is needed to ensure that the respondent will be available at the place and time of the proposed interview. (c) Personal chemistry may not work well. The interviewee may resent the interviewer on the basis of gender, ethnicity, or any other personal and behavioral characteristic. (d) The presence of the interviewer may influence interviewee responses in a subtle way. The interviewee may try to give responses that he thinks are acceptable to the interviewer on the basis of the interviewer's gender, race, SES, and suggestive questioning. (e) The common errors in face to face interview are omitting a question, too much or too little probing, failure to record information, and cheating by the interviewer.
3.2 Questionnaire administration by telephone
Questionnaire administration by telephone has the following advantages: (a) Considerable savings in time and money. It is possible to conduct a nation-wide survey sitting in one office. (b) Has fewer item non-responses because of the personal contact involved. (c) Skip patterns can be followed to save time. (c) Difficult questions can be explained. (d) Interviewer bias is less than in face to face interview.
The disadvantages of questionnaire administration by telephone are: (a) Selection bias may operate when the study sample includes only those who have telephones and the telephone numbers are listed. The problem of unlisted numbers can be overcome by use of random digit dialing. (b) Selection bias may arise due to the day and time of day that the telephone call is placed. Office workers will be missed in early morning calls. Workers on night shifts will be missed in evening calls. (c) It is not possible to be sure whether the person at the other end of the line is the actual intended respondent. Telephone interview can be improved by use of computers.
Computer-assisted telephone interview can make the process quicker when the interviewer is prompted by the computer. The computer will work out the skip patterns and will alert the interviewer to responses that are inappropriate or contradictory. Telephone interview must also be supervised for optimal results. The supervisor should listen in as the interview is conducted.
3.3 Questionnaire administration by mail
In the method of questionnaire administration by mail, a questionnaire is mailed to the respondent's address. The respondent completes and returns the questionnaire in a pre-addressed and stamped envelope. Questionnaire administration by mail has 2 main advantages: (a) it is the cheapest method of data collection (b) There is no bias due to interviewer involvement. The disadvantages are: (a) low overall response (b) Higher item non-response (c) Delays in returning the questionnaire.
The following measures are undertaken to increase response to mailed questionnaires: (a) sending the questionnaire with a personalized cover letter (b) promising a token of appreciation for return of the questionnaire. (c) Making the questionnaire anonymous by not including any information on the returned questionnaire that can be used to identify a particular individual. (d) Providing a self-addressed and stamped envelop for the response (e) using pre-coded questionnaires so that all the respondent has to do is to select responses (f) follow up by letter for those who delay in returning the questionnaires.
3.4 Computer-administered questionnaire:
The advantages of computer interview are: (a) It frees the interviewer's time. (b) There are no transcription errors because information in entered on-line. (c) No items are missed because the computer will not allow the respondent to move to the next item before answering the previous one. (d) The respondent can give more honest responses when facing an anonymous computer than when faced by a human interviewer. The disadvantage of computer-administered questionnaires is that the respondent does not have the opportunity to vary the order of questions to his convenience.
4.0 PREPARATION FOR DATA COLLECTION:
4.1 Research Protocol: Data collection processes must be clearly defined in a written protocol which is the operational document of the study. The protocol should include the initial version of the questionnaire. This can be updated and improved after the pilot study.
4.2 The objectives of the data collection must be defined clearly. Operational decisions and planning depend on the definition of objectives. It is wrong to collect more data than what is necessary to satisfy the objectives. It is also wrong to collect data just in case it may turn out to be useful.
4.3 The study population is identified. The method of sampling and the size of the sample are determined.
4.4 Staff training is necessary. The training should go beyond telling them what they will do. They must have sufficient understanding of the study that they can detect serious mistakes and deviations.
4.5 A pilot study to test methods and procedures should be carried out. However well a study is planned, things could go wrong once field work starts. A pilot study helps detect and correct such pitfalls.
4.6 A quality control program must be part of the protocol from the beginning.
5.0 CHECK LIST FOR INSTRUMENT DESIGN (from ACP MERC)
5.1 Basic Questions for Design
· What do I need to ask
· Whom should I ask
· How much time and $$$ do I have to spend
· What should the questionnaire look like
5.2 Format
· Make as appealing as possible
· Keep as short as possible, easy to read
· Number all questions and pages
· Include instructions (KISS)
· Make sure respondents know who sent the questionnaire and why
· Tell respondents how data will be used
· Assure anonymity if appropriate
· Assure confidentiality if appropriate
5.3 Content of questions
· Is each question necessary?
· Is each question sharply delineated to elicit the specific, desired response (data)?
· Do the questions cover the decisive features of the needed data?
· Do any questions ask for information which respondents do not possess?
· Are more concrete questions required to obtain and accurate description of respondent behavior?
· Are more general questions required to elicit prevailing attitudes for overall facts?
· Are the questions colored by personal or sponsorship biases (aimed in a certain direction)?
· Does each question present a sufficient number of alternative answers to permit the respondents to express their opinions accurately?
5.4 Wording of questions
· Is each question short and simple and written in unambiguous, understandable, nontechnical language?
· Are any questions misleading because of the absence of important alternative choices, poorly constructed alternatives, improper order, or an inadequate frame of reference?
· Are stereotyped, prestige-carrying, or superlative words and phrases used that bias the response?
· Are questions framed so that they annoy, embarrass, or anger the respondents and cause them to falsify answers?
· Would a more or less personalized wording of the questions better elicit the desired information?
5.5 Sequence of questions
· Do initial questions set the stage for those that follow and aid in the recall of ideas, or do they make subsequent topics inappropriate and/or embarrassing?
· Are the questions grouped, ordered, and located so as to arouse interest, to maintain attention, and to avoid resistance?
5.6 Form of responses
· Should the responses be obtained in a form requiring a check, a word or two, or a free text answer?
· What is the best type of check question to ask - dichotomous, multiple choice, or scale?
· Should a distinction of degree be made when rating items by employing an ordinal or interval/ration scale?
· Are the directions concise and clear, located next to the point of application, and made easy to follow by the inclusion of properly paced blank spaces, columns, or boxes? Are illustrations necessary?
· Is the instrument structured to facilitate the tabulation of data?
5.7 Pretesting the instrument
· Was the questionnaire pretested?
· Was a clear explanation of the purpose of the study and the specific intent of each question given during the pretesting period?
· After redrafting the wording of the proposed instrument, was the reliability of the responses checked?
6.0 SCALES OF QUESTIONNAIRES
6.1 Construction of the agreement scale
The more common agreement scale also consists of a series of attitude statements. Unlike the ordered scale, however, these sentences do not represent gradations of the attitude. These statements are written to be clearly favorable or unfavorable. The agreement scale achieves a wide range of score by having respondents report the intensity of an attitude. This is accomplished by providing gradations within the response alternatives. The respondents are asked to indicate their agreement with each statement on a 5 point Likert scale such as:
Strongly
Agree
Undecided
Disagree
Strongly
Agree
Disagree
Steps for Constructing and Using an Agreement Scale
1. Accumulate a large number of clearly favorable or clearly unfavorable statements about the attitude you wish to measure (usually 50-60). As with the statements for the ordered scale, a good source is the literature or prospective respondents.
2. As a pilot group (30 or more) to respond to these statements. The pilot group should consist of people who are similar to the intended respondent group and likely to express a wide range of attitudes regarding the attitude you wish to measure.
3. Score responses by assigning them from one-to-five points (five for most favorable, one for least favorable). This means responses will be scored differently depending on whether the statement is phrased negatively or positively.
4. Compute a score for each respondent by totaling the points corresponding to responses.
5. Stratify scores from high (top 25%) to low (lowest 25%).
6. Analyze each statement according to how high and low scorers responded. The method for accomplishing this step is called item analysis.
Item analysis
1. Retain those items (approximately 20) which provide good discrimination between high and low scorers.
2. Construct the questionnaire by listing the retained statements in random order.
3. Administer the instrument.
4. Compute the score for each respondent by totaling points for each item response.
Other Likert type rating scales
Strongly disapprove – disapprove – undecided – approve – strongly approve
To a little extent – to some extent – to a great extent – to a very great extent
Very Dissatisfied - dissatisfied - Neither Satisfied nor dissatisfied – Satisfied- Very satisfied
Little or no influence- Some influence-moderate influence-great influence
Very ineffective-ineffective-undecided-effective-very effective
6.2 Steps for constructing and using a semantic differential
1. Determine the attitude objects you wish to investigate.
2. Select appropriate adjective pairs (approximately 10). You may wish to select from the list provided a the end of this section. You may create your own.
3. Write the attitude object work or phrase at the top of the pate and place the adjectives beneath. You may use the same adjective order for each attitude object. Do not list all positive adjectives together, mix them with negative adjectives.
4. Example
How do you feel about each of these subjects. Place an X on one of the seven lines between each word
pair.
Physiology
Bad Good
Friendly Unfriendly
Fair Unfair
Sour Sweet
Biochemistry
Bad Good
Friendly Unfriendly
Fair Unfair
Sour Sweet
5. Instruct the respondents about how and where to mark their ratings. Responses should be made quickly and based upon first impression. Assure respondents that you are interested in their impressions, not studied responses.
6. Compute a person’s score by assigning scores ranging from 7 to most positive and 1 to the most negative. A person’s score for any one attitude object is the average of responses to the attitude pairs.
Additional Adjectives
Negative Positive
boring interesting
unhealthy healthy
uninformative informative
dull lively
confusing clear
weak strong
irrelevant relevant
unfair fair
superficial profound
dirty clean
biased objective
worthless valuable
purposeless purposeful
useless useful
closed open
passive active
tense relaxed
static dynamic
unhappy happy
unfriendly friendly
angry calm
wrong right
cold warm
6.3 The ordered scale
The ordered scale consists of a collection of statements that express a range of opinions about and object of attitude:
l School teaches you skills that help in getting a job
l Most teachers care deeply about their students
l School is interesting
l There are too many rules at school
l School is boring
l School is a good use of my time
Before inclusion on the measurement instrument, these statements are assigned a scale value (weight) along an attitudinal continuum by a panel of experts. The instrument is then constructed with statements placed in random order without indicating their scale values. Respondents are asked to check those statements with which they agree. Only opinion statements are used. Factual statements are not included in this type of measure, since people with differing attitudes could agree to a statement of fact. Note that the respondents are aske to select only htose items with which they agree and to reject all others.
Steps for Constructing and Using an Ordered Scale
1. Accumulate a large number of statements about the attitude object (approximately 100). A good source for these statements might be the literature or interviews with prospective respondents. Make sure that the statements represent a wide range of opinion, including moderate ones.
2. Place each statement on a separate piece of paper.
3. Select a group of experts to judge (30 or more similar to intended respondents) as ask each to sort the statements into 5-7 piles (odd number) with numerical values ranging from 1 = highly unfavorable, through neutral = 4, to 7 = highly favorable. Make sure the
judges understand that they are classifying statements and not indicating their agreement or disagreement with the statements.
4. Throw out statements that have been placed in widely differing piles, as these are ambiguous.
5. For each remaining statement, arrive at a scale value y computing the median or mean position of the assignment by the judges.
6. Select a set of statements (fewer than 25) whose scale values will give you a spread that evenly covers the continuum from highly unfavorable to highly favorable.
7. Construct the questionnaire by listing the statements in random order. Do not indicate scale values on the instrument.
8. Administer the instrument, instructing respondents to indicate those statements with which they agree.
Compute a score for each respondent by finding the mean of the scale values of the statements selected.
7.0 VALIDATING AN ASSESSMENT TOOL
7.1 Validity is not as easy to pin down as reliability. The difficulty lies in finding a suitable criterion of what the test is trying to measure outside of the test itself, against which the test may be checked by correlation or prediction.
7.2 Pinpoint the specific actions or responses that we will accept as evidence of what we want to
measure. That is, what do we mean by “competency,” “correctness,” “climate.”
7.3 Utilize the definition to operationalize the construct which we have defined. How would one recognize proficiency, success, or competency? Validity will be based on whether we accept the items in the survey as valid measures of the attributes contained in our definition of our construct. This may be called content or face validity.
7.4 Analyze performers or learners who are good at the skill we are evaluating and compare to
those who are not. (a) How did we know the good performers from the poor performers? Define the gap. (b) Subjective differences in quality. Take these subjective differences and convert them to objective, measurable criteria.
7.5 Consider defining appropriate outcomes, and evaluating for these.
7.6 Make the assessment methodology as close to reality as possible. Create an evaluative activity in which we may assess skills in a setting very similar to if not the same as the real situation. (a) Then apply to experts and novices to test validity (blindly apply) (b) Did the instrument classify known performers correctly?
7.7 SUMMARY OF PROCESS (five steps)
- Identify the purpose of the measurement. (e.g. behavior on a continuum from excellent to poor)
- Specify the actions that discriminate between excellent and unacceptable performance.
- Determine the real-world behavior against which you will validate the assessment.
- Select the appropriate methodology for assessing the skill.
- Develop or find the assessment instrument.
8.0 ASSESSING RELIABILITY OF AN INSTRUMENT
8.1 Use the same instrument with the same people taking it on two different occasions with no
intervening influences such as training, education, other evaluation.
8.2 Administer two equivalent forms of the instrument to the same population.
8.3 Compare a group’s performance on one half of the test with the performance on the other half
(split-half technique), correlating performance on even numbered and odd numbered items.
9.0 QUALITY CHECKLIST FOR INSTRUMENTS
9.1 INTRODUCTION
- There is a clear statement of the instrument’s purpose
- The respondent is told how information resulting from the instrument will be used
- Those who will see the data are identified
- The respondent is told why s/he was selected to complete the instrument
- The privacy of confidential information is insured
- The anonymity of the respondent is guaranteed (if appropriate)
- Motivators for responding are supplied
- Directions are provided for responding to items which “do not apply”
- The respondent is told if other materials are needed to complete the instrument
- Directions for returning the instrument are adequate (when, where, how)
9.2 ITEM STEMS
- The stem is relevant to the stated purpose of the instrument
- The stem focuses on one center (one key verb)
- The wording of the stem is appropriate to the reading level of respondents
- The possible response i not biased by the owrding of the stem (e.g. giveaway hint for plural)
- “Supply” items identify the appropriate unit of response
- Each stem is independent of other items
- The level of analysis necessary to respond to the stem is appropriate to the capabilities of the respondents
9.3 RESPONSES
- Response categories are unidimensional
- Response categories are not overlapping
- Response categories are exhaustive
- Response categories are relevant to the stems
- “Not applicable” “I don’t know” “No opinion” options are provided where appropriate
- As sufficient amount of space is left for supply responses
- Space is provided for comments where appropriate
- Guidelines are provided for comments
9.4 FORMAT
- Individual items are appropriately spaced
- Items are grouped in a logical order (by content, type, etc.)
- Sufficient space exists for the desired response
- The instrument is easy to read
- The instrument is not of excessive length
- The instrument is pleasing to the eye
9.5 DIRECTIONS
- Directions are given when necessary
- The language used is appropriate to the level of the respondents
- The directions are clear and complete
- An example item is provided if necessary
10.0 DATA COLLECTION (From ACP MERC)
10.1 Collection and presentation of data
· Decide how to order and break down data early in the investigation
· Classification categories should be sufficiently comprehensive and specific
10.2 Pros and Cons
PROS CONS
Will get desired information Little feedback
Focused Inflexible
Appropriate Impersonal
Efficient Bias in design
Quantitative Reliability
Content validity Validation
Planning for analysis difficult
Return rate
10.3 Getting data back
· Send pre-letter telling the purpose of the survey and who you are, why they should answer, who is being surveyed
· Then send short cover letter with the questionnaire
· Offer to send respondents a summary of results
· Explain why personal/demographic questions are necessary
· Keep procedures simple; self addressed-stamped envelopes; simple folding
· Keep questionnaire as short as possible, provide sufficient room for responses
· Phone or mail reminder
10.4 Structured interviews
· Information gathering by talking
· Decide ahead, information needed and questions to ask
· Face to face
· Permits probing of sensitive issues
· Permits rapport and interpretation of nonverbal cues
· Time consuming, expensive, requires special training
· Telephone interview
· Permits some probing
· Miss nonverbal cues
· Less expensive but > paper & pencil
· Not everyone has a telephone (bias)
· Some people are reluctant to give feelings or personal info over the phone
· How to ask questions
· Nominal scale: categories for single dimension (non ordered)
· Ordinal scale: reports data as ranks, reflecting feelings, attitudes
· Interval scale: categorizes data with absolute value
· Open ended: fill in the blank
· Introduction
· Who conducts
· Why conducted
· Why respondents selected
· How much time will take
· Do Guidelines
· Begins with a few non-threatening and easy to answer items
· Use simple and direct language
· Make items as brief as possible
· Emphasize the crucial words in each item
· Leave adequate space fro respondents to make comments
· Group items into coherent categories
· Provide some variety in the type of items used
· Include clear, concise instructions on how to complete the questionnaire
· Make sure there are clear instructions regarding what to do with the completed questionnaire
· Provide incentives as a motivation for promptly completing the questionnaire
· Use professional production methods
· Provide a well-written personal cover letter
· Include other experts and relevant decision-makers in your questionnaire design
· Plan how to analyze and use the data when designing the questionnaire
· Be prepared to handle missing data
· Test your questionnaire on representatives of the target audience
· Number each page and provide identifying info on each page
· Don’t Guidelines
§ Don’t use ambiguous, bureaucratic, technical, or colloquial language
§ Don’t use negatively worded questions unless absolutely necessary
§ Don’t use double-barreled items (one statement, one response)
§ Don’t bias respondents by hinting at a “desired” response
§ Don’t ask questions to which you already know the answers
§ Don’t include any extraneous or unnecessary items
§ Don’t put important items at the end
§ Don’t allow respondents to fall into response sets (HALO)
10.5 Information collectors:
· Criteria for selection
· Appropriate level of education
· Appropriate experience
· Sound and practical judgment (common sense)
· Interpersonal skills
· Ability to follow directions
· Considerations for selection
§ Availability
§ Special hiring requirements (who is the audience, funding rules)
§ Bias (personality and attitudes, may test)
· Basics for training
· Training regarding the program
· The need/purpose of information
· How, what, why they are collecting the info
· Communication skills
· Rules, restrictions, constraints
· Preparation with instruments, time for review, practice
· How data is to be recorded and reported
· Dealing with potential problems
· Provide “Guide to Information Collection”
· Instruments
· Forms
· Directions for administration and collection of data
· Names of people to contact
· Places to visit
· Persons to contact in case of problem
· List of potential problems and suggested solutions
· Observe those providing the information
11.0 PRACTICAL EXERCISES IN QUESTIONNAIRE CONSTRUCTION AND ADMIJISTRATION
Use a face to face questionnaire and a telephone questionnaire to collect the following class data:
11.1 Basic Class data
- Identification number (please use a fictitious one but remember it for future use)
- Socio-demographic: age, gender, district of birth, type of primary school (government or private), number of siblings
- Anthropometric: weight and height
- Health: blood pressure, pulse rate, respiratory rate, use of glasses (yes/no)
- Family history: use of glasses by father (yes/no), mother (yes/no), any sibling(yes/no).
- Opinions and attitudes: color preference (choose only among primary colors), ideal age at marriage, desired number of children, ideal age at retirement, mothers working outside the home, husband doing domestic work,
Read the following abstracts and construct a questionnaire with 5-10 of what you think are the most important items for collecting the information necessary
11.2 Attitudes, experiences, and professional expectations of general surgery residents JAMA 2009; 302(12):130-131.
11.3 Patient Safety Education at US and Canadian Medical Schools Academic Medicine Vol 84 No 12 Dec 2009
11.4 Medicine Residents Understanding of the Biostatistics and Results in the Medical Literature JAMA 2007; 298(9):1010-1022
11.5 Medical Students' Experiences of Moral Distress: Academic Medicine Vol 85 No 1 Jan 2010
10.6 Evaluating Obstetrical Residency Programs Using Patient Outcomes JAMA 2009; 302(12):1277-1283
11.7 Effect of Communications Training on Medical Student Performance JAMA 2003: 290(9):1157-1165
11.8 Comparison of the instructional efficacy of internet-based CME with live interactive CME workshops: a randomized controlled trial JAMA 2006; 295(7):758
12.0 PRACTICAL EXERCISES IN CRITIQUING QUESTIONNAIRES ON KAP CANCER RISK FACTORS AND PREVENTION
Critique the following questionnaires in view of what you have learned about good questionnaires. Suggest ways of improvement.
12.1 Personal and social profile
Complete the following identifying information by circling the correct response or writing the correct answer. Ignore questions or responses that do not apply
4. Age
a. 20-40 b. 41-60 c. 61-80 d. Above 80
5. Gender
a. Male b. Female
6. Region of birth
(a) Europe, Australia , New Zealand & the Americas
(b) Middle-East and North Africa
(c) Sub-Saharan Africa
(d) Central Asia
(e) South Asia
(f) South-east Asia and the Far East
7. Highest Level of Education Attained:
a. School Certificate b. Batchelor c. Masters d. Doctorate
8. University title
- Academic Staff b. Non-academic staff c. Student d. Other e. Assistant Lecturer
9. Marital Status:
a. Married b. Divorced c. Widowed d. Never Married
10. Ethnic Group
………………………
11. Weight in Kg
……………………
12. Height in cm
…………………….
12.2 Smoking experience
Answer to the best of your recollection your possible exposure to the following cancer risk factors
1. Cigarette Smoking Status
a. Never smoked b. Current Smoker c. Stopped last year d. Stopped several years ago
2. If a current smoker, give the number of cigarettes (sticks) smoked per day
a. 1-10 b. 11-20 c. 21-30 d. 31-40 e. Above 40
3. If a current or a former smoker, give the age you started smoking cigarettes
a. 10-20 b. 21-30 c. 31-40 d. More than 40
4. Which of the following viral Infections have you experienced before
a. EBV b. HIV c. HBV d. HCV e. HSV f. HPV
5. Estimate the number of times a week that you consume green vegetables
a. 0 b. 1-3 c. 4-6 d. 7+
6. Estimate the number of times a week that you consume fruits
a. 0 b. 1-3 c. 4-6 d. 7+
7. Estimate the number of times a week that you consume meats (beef, mutton, chicken, pork)
a. 0 b. 1-3 c. 4-6 d. 7+
12.3 Knowledge of cancer symptoms, signs, diagnosis, and treatment
Which of the following types of cancer have you heard of or read about? (please tick yes or no)
Heard | Read | |
Lung | ||
Breast | ||
Cervix | ||
Mouth | ||
Uterus | ||
Cervix | ||
Blood | ||
Intestine | ||
Stomach | ||
Bone | ||
Skin | ||
Eyes | ||
Bladder | ||
Kidney | ||
Brain |
12.4 Symptoms and signs of cancer
What are the early indicators (signs and symptoms) of cancer ? (tick all that you know)
Yes | No | Do not know | |
Non-healing ulcer | |||
Very high fever | |||
Inability to fall asleep | |||
Abnormal secretion or bleeding | |||
Increased appetite | |||
Decreased appetite | |||
Insanity | |||
Pain while passing motion | |||
Pain while passing urine | |||
Swelling or thickening of any part of the body | |||
Severe pain in the body | |||
Frequently passing urine at night | |||
Chronic cough | |||
Chronic diarrhoea (more than 2 weeks) | |||
Bloody cough | |||
Sudden chest pain | |||
Swelling and thickening of body parts | |||
Difficulty in swallowing | |||
Difficulty in passing motion (constipation) for more than 2 weeks | |||
Frequent Bone Fractures or Bone Pains |
12.5 Knowledge of cancer risk factors
For each type of cancer indicate the most likely risk factor (tick all that applies)
Type of exposure/risk Type of Cancer
a. Lung a. Breast c. b. Cervix c. Oral
1. Tobacco: Cigarette | Lung | Breast | Cervix | Oral |
2. Tobacco: chewing | ||||
3. Diet: high lipid content | ||||
4. Diet: Low vegetables & Fruits | ||||
5. Diet: Low in fiber | ||||
6. Infections: Viral | ||||
7. Infections: Bacterial | ||||
8. Early and frequent sexual exposure | ||||
9. Frequent Pregnancy and child-bearing | ||||
10. Alcohol | ||||
11. Occupational Exposure | ||||
12. Environmental exposures | ||||
13. Irradiation | ||||
14. Contaminated Food | ||||
15. Chemicals and Drugs |
12.6 knowledge of cancer prevention measures
Which of the following measures you would take to prevent cancer
Yes No Do not know
1. Stop smoking | Yes | No | Do not know |
2. Take low calorie diet | |||
3. Frequently eating preserved food | |||
4. Taking vegetarian diet | |||
5. Regular exercise | |||
6. Avoid drinking alcohol | |||
7. Increase the intake of meat | |||
8. Include green vegetables and meat in the diet | |||
9. Increase fiber in the diet | |||
| |||
| |||
| |||
| |||
|
12.7 opinions about cancer
Indicate your views or perception about cancer in general by ticking the most appropriate cell
Agree | Disagree | Do not know | |
1. Cancer can be cured if detected early | |||
2. Cancer is a serious problem 3. | |||
4. Cancer is a punishment for sins and can not be cured | |||
5. Some cancers are not death-threatening | |||
6. Only smokers will get cancer 7. | |||
8. Some cancers can be detected early | |||
9. There are no facilities for early detection | |||
10. Cancers can not be cured even if detected early | |||
11. All cancers can be detected early before any signs & symptoms | |||
12. Cancer is very rare disease that affects only those who are unlucky | |||
13. Cancer is hereditary 14. | |||
15. Cancer is contagious 16. |
12.8 Attitudes about cancer
Indicate your attitude about cancer by indicating your agreement or disagreement on the following statements
Agree | Disagree | Not sure | |
1. I prefer to detect cancer early in order to save life | |||
2. All individuals are at risk of getting cancer | |||
3. I prefer traditional treatment If I have breast cancer | |||
4. I will not see a doctor even if I have early signs of cancer | |||
5. I do not like to sit next to someone smoking | |||
6. I am not concerned about what I eat |
12.9 Health-seeking behavior
For each of the following health-seeking behaviors you have
Have been immunised against hepatitis B?
a. Yes b. No
When did you last have a breast examination done by a doctor?
a. One month ago b. 6 months ago c. I year ago e. Never
When did you last have a mammography done?
a. One month ago b. 6 months ago c. I year ago e. Never
How frequently do you do breast self-examination?
a. Once a week b. Once every 6 months c. Once a year e. Never
When did you last have a pap smear?
a. One month ago b. 6 months ago d. I year ago e. Never
Did you have an annual physical examination last year?
a. Yes b. No
Indicate how frequently you take the following foods in a week
0 | 2-4 | 5-9 | 10+ | |
Meat (beef, mutton, pork, chicken) | ||||
Vegetables | ||||
Fruits | ||||
Fish |
13.0 Critique of a questionnaire on Assessment Of Knowledge Of Medical Ethics Among Participants In An Ethics Training Program Held In Nigeria
13.1 Perceptions of bioethics & health service law --1
1. You become aware, by accident, that one of the patients is secretly is HIV-
positive (the laboratory report somehow became stuck to the underside of an
envelope he gave to you). Do you
A. tell his spouse about his condition and warn her to be careful?
B. inform your patient that you learned about his HIV by accident, and ask him if he is seeking treatment?
C. make sure that when you treat him, you wear a mask and gloves, and warn other members of the health team to do the same?
D. carries on as normal, saying nothing about it?
2. You are part of a team providing general health care. In the course of this, blood is taken for tests and there is some left over. A commercial laboratory approaches you for supply of this excess blood so that it can prepare a saleable protein from it. Do you
A. send the blood to this laboratory as requested?
B. agree to send the blood if a fee is distributed among the health team?
C. revise the patients’ consent form to allow them to agree to the donation?
D. negotiate a payment to all the patients who are prepared to consent to donating their excess blood?
3. A number of alternative treatments for your patient are possible. The one which promises the best outcome is also the most expensive and it also carries the most risks. In respect of a patient who is established not to be wealthy, do you
A. explain the costs but omit to emphasize the risks?
B. explain both the risks and the expenses involved?
C. advise a cheaper but possibly less effective treatment?
D. give the patient a written list of treatments and their costs and ask him to make up his own mind?
4. The health team is doing blood grouping, among other tests and you become aware, from the records, that a friend of yours, also a patient, cannot be the biological father of the child he is bringing up. Do you
A. do nothing?
B. warn your friend about the non-paternity?
C. amend the records so that the truth is forever concealed?
D. ask your supervisor what to do?
5. While working in the clinic, it is becoming obvious to you that a number of patients are pretending to be ill, to obtain time off work. You overhear conversations in which they admit to doing second jobs during their medical leave, like driving taxis and selling insurance. Do you
A. warn the doctor concerned?
B. argue with the patients whose conversations you have overheard?
C. contact the police?
D. do nothing?
6. Later in life, as an executive in a big bioengineering company, you are involved in a project which promises to bring relief to hundreds of thousands of sufferers from deafness and blindness. However in your senior position you become aware that the company is only publishing the results of trials which show a favorable outcome; many trials convincingly demonstrate that thousands of patients could be harmed, but they are suppressed. Do you
A. do nothing?
B. state your concern, as a ‘whistleblower’ in an anonymous letter to the
newspapers?
C. ask for a personal interview with the Minister of Health asking for
investigation and complete confidentiality?
D. look for another job pending your full exposure of the malpractice?
7. A member of your team is giving powdered peach stones, free, to the patients, because he believes that this prevents cancer. You have reliable information that the peach stone powder is toxic (in fact it can generate cyanide in the stomach). You have argued with your colleague about this but he is adamant that he is only doing good. Do you
A. continue to argue with your colleague in the hope of persuading him that he is misguided?
B. report your concerns to the team leader?
C. go directly to the Ministry of Health with your concerns?
D. write to the New Straits Times about your concerns?
8. Working within the Ministry of Health, you use your lunch break to design a rather clever diagnostic device, which you believe to be novel. Do you
A. provide its specifications to the Chief Medical Officer through your Head of
department?
B. form a company to exploit it?
C. contact a company and offer to sell it to them?
D. apply for a patent by completing application forms from the Patent Office?
13.2 Perceptions of bioethics & health service law --2
1. As part of the management team in a large city hospital, you are called to a meeting to vote on the response to the following problem. A decision must be taken with at the most four hours. It appears that a patient about to give birth would definitely die, along with her baby, if a caesarian section is not performed. (The pelvis is too narrow and in the any case the baby is in a transverse lie.) For religious reasons the patient has refused the operation. The obstetricians want to perform the caesarean but have come to the hospital management group for their decision. Do you vote for
A. applying to the High Court for permission for the hospital to conduct the
operation?
B. obtaining consent from the patient’s husband then going ahead with the
caesarean?
C. letting events merely take their course, while trying to keep the patient as
comfortable as possible?
D. telephoning the Minister of Health for his opinion?
2. Consider the term “patient autonomy”. Do you take it to mean that
A. each patient has individual characteristics?
B. a patient’s own choice of treatments and possible outcomes must be
respected?
C. patients must be regarded as valuable human beings?
D. the patient must automatically follow the best advice given to him?
3. In normal circumstances patients register at your reception area and are told to wait before being called in to see you. You gently move a patient’s head for a better examination but she says that you twisted her neck and complains to the police. Are you protected by the
A. fact that the police will not take up the case?
B. lawyers that your clinic will hire?
C. legal doctrine of implied consent?
D. the sum of money which will be given to the patient in compensation?
4. If for a certain procedure involving physically manipulating the patient, written consent is necessary, but you fail to obtain this and go ahead, are you legally liable in
A. battery?
B. negligence?
C. recklessness?
D. carelessness?
5. A six-year old boy is brought to your clinic for a simple procedure. However you notice multiple bruises on his back and the backs of his arms. The father who accompanies him appears have identifiable needle marks on his arms. Do you
A. demand from the father an explanation of how the bruises occurred.
B. telephone the police immediately with your suspicions.
C. immediately after the end of the consultation, telephone the Society for the
Protection of Children (assuming one exists in your area).
D. note down the address of the family and watch the house for a few hours in the
evening?
6. Running your own clinic/ dispensary, you are asked for a certain treatment by a fifteen year old boy. However you aware that his parents violently disagree with the treatment, which however in your opinion will increase the boy’s ability to benefit from his schooling. Reasoning with the parents has had no effect. Do you
A. contact several colleagues in the same specialty and ask them what they would
do?
B. seek legal advice?
C. merely proceed with the treatment?
D. ask the boy to go elsewhere?
7. A non-urgent procedure is needed for a six-year old boy. However seemingly the boy cannot sit still long enough for the treatment to proceed. Trying to settle him forcibly with the help of his parents causes him to become hysterical and to start kicking and screaming. You noticed that in the waiting room he was continually climbing over the chairs and running round the table. Do you
A. refuse to waste time with the boy?
B. tell the parents how to get psychiatric help?
C. ask the parents to restrain the boy with leather straps so that you can proceed?
D. inform the parents that the child has attention deficit hyperactivity disorder
(ADHD).
8. A 16-year old girl has been your patient for some time, but has been very difficult. At this session she tells you that she is in love with you and wishes to elope with you. Do you
A. immediately send her away and refuse further treatment altogether?
B. call her parents and ask them to reason with her?
C. tell her not to be silly and ask her to sit still for the consultation?
D. ignore her comments, call in a chaperone and proceed?
13.3 Cultural / religious issues in clinical practice -1
1. A female Muslim patient preparing for a chest X-ray becomes very uncomfortable and adamantly refuses to be touched by one of the only two male radiographers attached to the hospital. Do you
- advise the patient that this is acceptable for the sake of health?
- call your superior, to have him explain the procedure to the patient?
- call for a chaperone?
- have the hospital call a female radiographer from a nearby hospital?
2. The parents of a young Muslim patient in a coma but not brain dead have mentioned to you, that they will seek a court order to have their daughter’s life support removed as specialists have told them the chance of recovery is small and it has been two years since her devastating accident. Another reason is the rising hospital bills. Do you
A. inform the hospital director?
B. seek an injunction to prevent the disconnection?
C. help to secure financial assistance for the patient?
D. counsel the parents on the obligations of a Muslim?
3. Following in vitro testing, your research supervisor has proposed that you test a new drug on cats, after which you will examine its effects on their reproductive organs. Do you
A. agree to do this as the purpose of animal research is to spare humans?
B. counter-propose the use of smaller animals, for example, rats?
C. Concur, as long as the testing will not cause suffering and pain to the animal?
D. decline as you know that the testing is not a necessity (dharuurat)?
4. A Muslim female confides to you that she feels uneasy examining the eye of a patient of the opposite sex. Do you
1 advise her to carry on as usual in a professional manner?
2 advise her to wear gloves?
3 dismiss her feelings as irrelevant?
4 discuss her concerns with your superiors?
5. In training, you witness a colleague perform a procedure which only a medical officer is allowed to do. Although there is no adverse effect on the patient, do you
5 feel obligated to report the incident to your supervisor immediately?
6 keep quiet?
7 advise your colleague on the possible implications of the actions?
8 tell your colleague what you observed and that you would keep the matter secret?
6. A Muslim female patient is worried over the reactions of her husband when he finds out that she has sought treatment without his permission. Do you
9 simply refrain from treating?
10 advise her to obtain permission from her husband?
11 carry on, as a spouse cannot overrule the choice of the patient to seek treatment?
12 ask the patient to come back with her husband so you can explain the necessity of the treatment?
6. As a researcher on ageing, you are offered a position on the research committee of a well-known international company working on an elixir claimed to guarantee a lifespan of 100 years. Do you
13 seize the opportunity to enhance your international standing?
14 turn down the offer as you believe that life and death is in the hands of Allah?
15 accept the offer for the sake of acquiring new knowledge?
16 decline as you fear criticisms from other Muslims?
7. Your sister tells you that she has been taking a natural herbal product from China
17 advise your sister to stop taking the product?
18 investigate whether the product is halal?
19 do nothing?
20 explore the side effects of similar products?
8. In the presence of a bed-ridden but conscious Muslim patient, when it is time to perform the obligatory prayers (solat), do you
21 ask if you could assist the patient to pray?
22 ask your supervisor for guidance?
23 decide that the patient is not fit to perform the solat?
24 do nothing?
13.4 Cultural / religious issues in clinical practice -2
1. When treating or giving advice to a non-Muslim patient, do you
A. continue to practice values according to the teachings of the Qur’an?
B. proceed without any reference to Islamic values?
C. perform your duties according to what you have been taught?
D. see no obligation to inculcate Islamic perspectives?
2. A married couple who rely on your advice propose to travel to India
A. Surrogacy is not permitted in Islam.
B. This is medically justified.
C. This is allowed on the basis of an individual’s reproductive rights as its
impact on society is negligible.
D. All methods of assisted reproduction can be considered provided the
couple is legally married.
3. Your 45-year-old Muslim aunt with 12 children including a young baby consults you regarding permanent sterilization. You would
A. inform her that it is strictly prohibited.
B. tell her that she should opt for other types of family limitation.
C. tell her that such a decision is acceptable with her husband’s consent.
D. tell her that since she is approaching menopause contraception is not
necessary.
4. A devout member of your ethics committee objects to a proposal to issue free needle to heroin addicts. Do you:
A. agree that this is wrong.
B. point out that this will only encourage drug addiction.
C. suggest more stringent prosecution of addicts.
D. point out that the principle of lesser harm may be applied.
5. The Ministry of Health is seeking individual views of euthanasia. In your response to the Ministry, do you point out that:
A. in all forms it is viewed by the Islamic Law as murder.
B. it may allowed for the purpose of easing the burden of a prolonged illness.
C. it may allowed in some circumstances based on individual human rights.
D. it may be ethical in the case of withdrawing life support from a patient in vegetative state.
6. A 40-year-old Muslim lady 12 weeks pregnant insists on having an abortion following detection of Down syndrome fetus. As a member of the ethics committee you would advise:
A. complying with her request.
B. having the prenatal test repeated to verify the result.
C. against the abortion since there is a likelihood that the result is incorrect.
D. telling her that abortion in this circumstances is not acceptable.
7. The ethics committee is reconsidering the matter of organ donation. Should it decide that:
A. organ donation from a non Muslim to a Muslim is not allowed.
B. only the living are allowed to donate.
C. an adult Muslim can sign a statement authorizing removal of his organs in case of death.
D. physicians have the authority to decide on the donation of organs of the
deceased.
8. A 10-year-old boy is diagnosed to have haemophilia A. Porcine Factor VIII is available. Other options are either less effective or very costly. His parents are reluctant to agree to the treatment. As a member of the healthcare team you would:
A. suggest the less effective treatment.
B. advise treating the patient with Porcine VIII since he is bleeding but
conceal this from his parents.
C. put out an appeal to the public for extra funds for the non-porcine
treatment.
D. explain to the parents that the porcine source is acceptable based on the
principle of dharurah.
9. A 20-year-old girl who was previously well was found dead in her girl friend’s apartment. Consent for a post-mortem examination could not be obtained. Do you consider that the best course of action is:
A. to not perform the examination and release the body.
B. to proceed with the examination since it is a medico legal autopsy which does violate the Law (Shari’at).
C. to request the police officer in-charge to obtain the court order for the
examination.
D. to consult the forensic pathologist to obtain permission to perform only the external examination.
10. The parents of a newborn with a congenital defect with a poor chance of survival seek your advice with regard to a very invasive and expensive treatment, for which they can well afford. You would
- help them look for medical institutions locally or abroad where such
treatment is offered.
B. advise them against it as it is a waste of resources.
C. advise them to submit to God’s will and accept death as predestined.
D. tell them that heroic treatment of a patient with a poor prognosis is not an Islamic option.
11. You are chairing a discussion on Medical Ethics at a local conference. One of the participants poses this question to you: “Do medical ethics slow down medical research and progress? Your response is that:
A. To some extent it does, justifiably.
B. There is no justification for it doing so.
C. It legitimately safeguards and regulates medical practice.
D. Progress in medical research is independent of medical ethics.