Home 
Contact 
Profile 
Videos 
A course for staff of the Global Center for Mass Medicine at the Ministry of Health by Dr. Omar Hasan Kasule MB ChB (MUK), MPH (Harvard), DrPH (Harvard) Professor of Epidemiology at King Fahad Medical City Riyadh
8.1 DEFINITION OF HEALTH SYSTEMS RESEARCH
8.1.1 Definition of a Health system: all public and private sector health institutions
8.1.2 Definition of Research: collection, analysis, and interpretation of data to answer a question.
8.1.3 Rationale of health systems research: 1. Understand all disease factors 2. Decisions
8.1.4 Types of health systems research: 1. Basic research 2. Applied research
8.1.5 Impact of health systems research: efficiency and effectiveness of the health system.
8.1.6 Research team health policymakers, health managers, the community, scientists.
8.1.7 Beneficiaries of research: health policymakers, health managers, hospital directors.
8.1.7 Main guidelines of health systems research: 1. focus on priority problems 2. action-oriented 3. multi-disciplinary 4. timely results 5. simple short term research designs, 6. cost-effective, 7. reports in the format needed by end-users.
8.2 Components of a health systems research proposal:
8.2.1 Background, problem statement, literature review.
8.2.2 Objectives, study design, sampling
8.2.3 Data collection, data processing and data analysis
8.2.4 Ethical considerations
8.2.5 Pre-test
8.2.6 Work plan, Budget, Plan for administration and monitoring of research
8.2.7 Plan for presentation and dissemination of results
8.3 DETERMINING RESEARCH PRIORITIES
8.3.1 A health problem exists: 1. if there is gap between the actual and the ideal 2. if reasons for the gap are not clear 3. if potentially more than one solution is possible
8.3.2 Information needed to define a research question: 1. Information on present health status 2. Information to evaluate on-going interventions 3. information to define problems
8.3.3 Criteria used to prioritize among research questions: 1. relevance 2. no duplication 3. urgency 4. political acceptability 5. feasibility of study 6. applicability of results 7. ethical acceptability
8.3.4 Use of the nominal group technique to select priorities: 1. Individuals list their ideas on paper 2. Lists are displayed and are grouped 3. voting and ranking 4. scoring and rediscussion 5. second vote and re-ranking
8.3.5 Advantages of the nominal group technique: 1. separation of discussion from voting 2. voting is anonymous 3. results indicate input from all members 4. voting aggregates individual judgments
8.4 ANALYSIS OF AND STATEMENT OF THE RESEARCH PROBLEM
8.4.1 Steps in problem analysis: 1. obtaining views of managers and health care workers 2. specification of the core problem(s) 3.Triangulation, looking at a problem from various angles.4. listing factors of the problem, the relation between the problem and the factors and the interrelations among factors 5. grouping factors as: socio-cultural, economic, service related, disease related. 6. constructing a problem analysis diagram.
8.4.2 Components of a problem statement: 1. socio-cultural and economic background to the problem 2. health status assessment 3. nature of the problem: how, who, when, where, how 4. solutions tried in the past 5. justification for new research 5. type of information needed 6. definition of crucial terms and concepts
8.5 FORMULATION OF RESEARCH OBJECTIVES
8.5.1 Types of research objectives: 1. general 2. specific.
8.5.2 Advantages of stating research objectives: 1. focussing the study 2. avoiding unnecessary data collection 3. organizing the study in parts/phases.
8.5.3 Guidelines for writing objectives: 1. coherent and in logical sequence 2. in operational terms 3. realistic 4. action verbs.
8.5.4 Research title: 1. must be concise 2. must reflect objectives.
8.6 HYPOTHESES
8.6.1 Types of hypotheses: 1. null hypothesis 2. alternative hypothesis
8.6.2 Advantage of hypothesis-based research: 1. Research based on hypotheses is more powerful than research based only on stating problems. 2. avoiding fishing trip findings
8.7 STUDY DESIGN
8.7.1 Types of studies: 2. observational 2. intervention
8.7.2 Observational studies: 1. descriptive (case studies, surveys) 2. analytic / comparative (cross sectional, case control, follow up).
8.7.3 Intervention studies: 1. experimental 2. quasi experimental (no randomization and no comparison group eg before and after study).
8.7.4 Validity vs precision.
8.7.5 Factors of reliability: 1. study design 2. data collection 3. data analysis.
8.8 SELECTING THE SAMPLE
8.8.1 Populations: 1. study population. 2. Study units.
8.8.2 Representativeness.
8.8.3 Qualitative sampling: 1. extreme case sampling 2. maximum variation sampling 3. homogenous sampling 4. typical case sampling 5. critical case sampling 6. snow ball or chain sampling 7. quota sampling.
8.8.4 Probability sampling: 1. simple random sampling 2. systematic sampling 3. stratified sampling 4. multistage sampling 5. cluster sampling.
8.9 SAMPLE SIZE COMPUTATION
8.9.1 Parameters for sample size computation: n = sample size, s = standard deviation e = standard error r = rate p = proportion, m = mean, u = cut off on the normal distribution corresponding to selected power v = cut-off on the normal distribution corresponding to a 0.05 significance level, .
8.9.2 Sample size computation for one sample: 1. single mean n = s2 / e2 2. single rate n = r / e2. 3. single proportion n = p(100-p) / e2
8.9.3 Sample size computation for two equal size samples: 1. difference between 2 means n= {(s1)2 + (s1)2} / e2. 2. difference between 2 rates n= {(r1)2 + (r1)2} / e2. 3. Difference between 2 proportions {(p1(100- p1) + (p2(100- p2)} / e2
8.9.4 Sample size computation for 2 unequal sample sizes: 1. difference between 2 means n = [{(u + v)2}{(s1)2 + (s1)2}] / [m1 + m2]2. 2. difference between rates n = [{(u + v)2}{(s1)2 + (s1)2}] / [r1 + r2]2 3. difference between 2 proportions n = [(u-v)2 {p1(100 - p1) + p2(100 - p2)}] / [(p1 - p2)2]
8.10 SAMPLING ERRORS
8.10.1 Sampling error: 1. wrong sample 2. non coverage
8.10.2 Assignment error: 1. defective matching in case control studies 2. non randomization for experiments 3. no quasi experimental if randomization not possible
8.10.3 Sampling bias: 1. non response 2. volunteer bias 3. registered patient bias 4. prevalence bias 5. seasonal bias 6. accessibility bias 7. male bias.
8.10.4 Selective drop out or non-response
8.11 THE STUDY QUESTIONNAIRE VARIABLES
8.11.1 Determining variables: 1. factors of the problem are rephrased as variables 2. operational definition of variables.
8.11.2 Classification of variables: 1. background variables eg age and sex 2. real variables vs indicators variables 3. quantitative vs qualitative 4. discrete vs continuous 5. dependent vs independent 6. causative, associative, and confounding.
8.11.3 Preliminary questions: 1. what to do to achieve according to our objectives and variables? 2. is the questionnaire the right technique? 3. what type of questionnaire? 4. do we have enough understanding of the topic to design a questionnaire or should we use qualitative methods? 5. what is the informant’s level of education?.
8.12 OPEN QUESTIONNAIRES
8.12.1 Advantages of completely open questionnaire: 1. can be used for facts, opinions, and sensitive issues 2. enables deep probing 3. can detect issues that arise de novo during research 4. information in respondents words good illustration 5. raw info can be reinterpreted later.
8.12.2 Disadvantages of completely open questionnaire: 1. incomplete recording of relevant issues 2. analysis difficult and time consuming. Disadvantages of open questionnaires can be mitigated by 1. training interviewers 2. list of probe questions 3. pre-test. Partially categorized questionnaires.
8.13 CLOSED QUESTIONNAIRES
8.13.1 Advantages of pre-categorized questionnaires: 1. quick data collection 2. quick analysis.
8.13.2 Disadvantaged of pre-categorized questionnaires: 1. important information may be put in the ‘other’ category 2. pressure on interviewer to force info into a category 3. interviewer may stop after answer to the first category 3. if interviewee hesitates interviewer may fill in leading to bias 4. too many responses in the other category. 5. reasoning behind response not known.
8.14 Steps in designing a questionnaire:
8.14.1 content according to objectives and variables
8.14.2. formulate questions to give information about variables making sure that each question measures one thing at a time. Avoid leading questions. Avoid words with double meaning. Avoid vague words. Avoid emotionally laden words. Sensitive questions should be asked in a socially acceptable way.
8.14.3. Sequence the questions, informant friendly, conversational, start non controversial matters, keep sensitive questions at the end, use simple everyday language, use local idioms.
8.14.4. Formatting the questionnaire: data, time, interviewer name, layout, enough space, boxes
8.14.5. translation and back translation.
8.15 QUESTIONNAIRE PRE-TESTING and VALIDATION
8.15.1 Pre-test and pilot testing: 1. pretesting is a small scale trial of a research component 2. pilot study is a small scale trial of the whole research components. 3. Pretest is done in the workshop or in the field.
8.15.2 Advantages of a pre-test: 1. identify and resolve potential problems so save time and money from mistakes 2.
8.16 WHAT IS INCLUDED IN QUESTIONNAIRE TESTING
8.16.1 Respondents: 1. availability, 2. acceptability of methods and questions, 3. willingness to particip4. 8.16.2 Data collection tools: 1. reliability, 2. time required, 3. logical sequence of questions, 4. clarity of wording, 5. accuracy of translation, 6. open/closed, coding, 7. instructions.
8.16.3 Sampling procedures: 1. clarity of instructions 2. ease of locating respondents.
8.16.4 Research staff: 1. effectiveness of training 2. output per researcher 3. team dynamics 4. logistical support 5. reliability 6. adequacy of supervision.
8.16.5 Data collection and analysis procedures.
8.16.6 Budget
8.17 Validity of a questionnaire:
8.17.1 Validity indicates that the instrument measures what it is supposed to measure.
8.17.2 Validity is described as content validity, face validity, criterion validity, and construct validity. Face validity is based on appearances.
8.17.3 Criterion validity is based on use of criteria either for predictive validity or for concurrent validity. 8.17.4 Predictive validity is a measure of how well a certain criterion predicts the future.
8.17.5 Concurrent validity is an assessment of how well a measure agrees or concurs with other measures. Construct validity is experimental demonstration of how well a measure distinguishes groups of people.
8.17.6 Reliability assesses measurement error. A reliable instrument will have a low measurement error and repeated measurements will show stability.
8.18 DATA COLLECTION 1
8.18.1 Stages in data collection process: 1. permission 2. data collection: logical, sequential, quality 3. data handling.
8.18.2 Data collection methods: 1. available information 2. Written questionnaire 3. face to face Interviewing 4. Focus group discussion 5. Other methods
8.18.3 Data collection tools: 1. questionnaire. 8.1
8.18.4 Data collection plan: 1. what info, 2. how (approach, technique, tools), 3. who will do what?, 4. how long, 5. how many researchers, 6. sequence of work, 7. quality control, 8. weekly schedule.
8.18.6 Advantages of data collection plan: 1. clarity of tasks 2. efficient mobilization of human and material resources 3. minimize errors and delays.
8.19 DATA COLLECTION 2
8.19.1 Ensuring quality: 1. fields manual. 2. Recruit and train research assistants. 3. Pretest questionnaire and procedures. 4. Supervision. 5. Quality control.
8.19.2 Errors in data collection: 1. Invalid data due to wrong indicator 2. invalid data due to wrong measurement.
8.19.3 Unreliable data: 1. variations in the research assistants 2. impact of the research assistant on the respondent 3. instructions that are not standard 4. inter-observer . inter-interviewer variation.
8.20 DATA COLLECTION USING A QUESTIONNAIRE 1
8.20.1 Methods of questionnaire administration: 1. face to face 2. telephone 3. mail 4. internet
8.20.2 Written questionnaire less flexible than face to face interview
8.20.3 Bias in data collection by questionnaire: 1. defective instrument eg closed questionnaire for matters vaguely understood, open questionnaire with no guidelines, vaguely phrased questions, leading questions, questions not in a logical order. 2. measuring / weighing scales not standardized 3. observer bias 4. interviewer impact on respondent 5. information bias.
8.20.4 Ethical issues in data collection: 1. violation of privacy rights 2. concealed observer with no consent 3. breach of confidentiality 4. violate cultural / social taboos.
8.20 DATA COLLECTION USING A QUESTIONNAIRE 2
8.20.1 Interviewing guidelines: 1. Treat respondents as partners. 2. Informed consent. 3. Anonymity for sensitive topics. 4. Promise feedback. 5. Minimise social distance (clothing, sitting arrangement eg more distance for opposite gender), 6. show interest in what the informant says, 7. make informant feel at ease, 8. do nor show disapproval. 9. do not tape record unless the informant is confortable 10. eye contact do not read all the time.
8.20.2 Interviewer tasks: 1. introduce the interview 2. pose questions 3. evaluate answers 4. record answers 5. keep control of the interview 6. summarise the interview 7. note any information after the interview.
8.20.3 Training research assistants: 1. written exercises 2. role play 3. actual interview.
8.21. FOCUS GROUP DISCUSSION
8.21.1 Focus group discussion: is 6-12 persons discussing freely and sponateneously under a facilitator.
8.21.2 Uses of FGD: 1. focus research to enable generation of more specific hypotheses 2. formulate questions for larger research 3. understand and solve problems in interventions 4. develop health education messages 5. explore controversial topics.
8.21.3 Strengths of FGD: 1. iterative 2. cheap 3. can deal with senstitive issues when participants do not know one another.
8.21.4 Limitations of FGD: 1. not qualitative 2. risky as a sole tool of research; best used with other tools.
8.22. FOCUS GROUP DISCUSSION
8.22.1 Conduct of FGD: 1. determine purpose 2. situation analysis 3. recruit participants 4. select participants 5. physical seating arrangements 6. written discussion guide.
8.22.2 Functions of facilitators in FGD: 1. introduce the session 2. encourage discussion and involvement 3. handle sensitive issues 4. build rapport / empathy 4. avoid being an expert 5. control discussion but unobstrusively 6. summarize at end 7. listen to additional comments.
8.22.3 Functions of the recorder: 1. record date, time, participant details 2. record group dynamics 3. record emotional aspects 4. record vocabulary levels. 5. Number of sessions and duration.
8.22.4 Analysis of results: 1.code and subcode 2. comments 3. summarise 4. systematic comparison 5. major differences.
8.23 OTHER METHODS OF DATA COLLECTION
8.23.1 Observation: 1. observer as participant vs non participant. 2. Open vs concealed observation. 3. Observe persons vs objects).
8.24.2 Projective techniques (subject asked to react to visual or verbal stimulus)
8.24.3 Mapping: .
8.24.4 Scaling (respondent asked to rank or organize data).
8.24 DATA HANDLING AND ANALYSIS
8.24.1 Data handling: 1. ID for each data form. 2. Storage. 3. Confidentiality espp for sensitive questions.
8.24.2 Data analysis plan: 1. sort data 2. quality checks (completeness and consistency) 3. data processing (categorize, code, summarize)
8.24.3 Data analysis / quantitative: 1. frequency, 2. cross tabulation
8.24.4 Data analysis / qualitative: 1. use a sample of questionnaires to develop a coding system 2. apply the coding system to all data 3. classify codes (knowledge, attitudes, opinions) 3. compute percentages.
8.25 RESEARCH WORK PLAN
8.25.1 Workplan is a schedule, chart, or graph that summarises different components of a research project and its implementation.
8.25.2 Components of a work plan: 1. tasks 2. where 3. who 4. how 5. when 6. how long
8.25.3 Gantt chart: 1. tasks 2. who 3. duration
8.26 RESEARCH BUDGET
8.26.1 Budgeting starts from the beginning and evolves with the proposal
8.26.2 Budget preparation: 1. budget categories 2. expense per category 3. how to control or reduce the budget
8.26.3 Computation of unit cost and total cost
8.26.4 Computation of person days
8.26.5 Budget justification
8.26.6 Budget presentation
8.27 RESEARCH PROJECT ADMINISTRATION AND MONITORING
8.27.1 Project Leader: roles and functions
8.27.2 Project momnitoring
8.27.3 Field work activities: Brief project managers and community leaders, Assure project resources, Review informal information available, Logistics of data collection, Field work manuals,
Train research assistants, Pre-test, Collect and process data
8.27.4 Field work report: Introduction, Field experiences, Preliminary findings,
Workshop to present result.