Survey Methodology and Questionnaire Design
The Basic Research Process
Quantitative research involves formulating a problem, developing assumptions (hypotheses), and defining variables. The process includes analyzing variable distribution, examining cross-variable relationships to test hypotheses, and conducting a comprehensive analysis using tables and graphs.
Festinger and Katz’s Sample Survey Development
- Establish the problem and define overall objectives.
- Specify data objectives and assumptions.
- Define sample type, size, and composition.
- Design the questionnaire (pilot testing may be necessary).
- Conduct fieldwork.
- Code responses, clean data, and prepare for statistical treatment.
- Tabulate and analyze data to test hypotheses.
- Analyze data and create a final report.
Survey Types
Cross-Sectional Surveys
Cross-sectional surveys capture data at a single point in time, providing a snapshot of a specific moment.
Longitudinal Surveys
Longitudinal surveys track the same sample population over time to observe changes and trends.
Questionnaire Structure and Organization
Organize questions by thematic areas in a logical sequence. Use clear headings (chapeaus) to signal topic changes. Begin with introductory questions to engage respondents and build rapport. Employ the “funnel” approach, starting with general questions and progressing to more specific ones.
Types of Questions
Sociodemographic Questions
These questions gather information on gender, age, marital status, employment, income, etc., to understand how these factors relate to other variables. Place these questions at the end of the questionnaire.
Past Behavior Questions
These questions address past experiences, such as previous occupations.
Mitigating Questions
These unrelated questions help respondents relax and transition between topics.
Consistency Control Questions
These questions verify the accuracy of responses by cross-referencing related information.
Filter Questions
These questions direct respondents to specific question sets based on their answers. Contingency questions provide further details within a specific subset of the sample.
Example:
Q1: Do you use any illegal drugs?
Yes (go to question 2)
No (go to question 5)
Battery of Questions
A battery of questions presents a thematic block with the same response categories for multiple questions.
Survey Question Design
- Questions must be relevant to the research objectives.
- Use clear and correct language, avoiding jargon and abbreviations.
- Ensure questions are meaningful to respondents.
- Avoid double-barreled questions (asking two questions in one sentence).
- Keep questions concise and easy to understand.
- Ensure questions are unambiguous and avoid different interpretations.
- Avoid emotionally charged language.
- Limit the number of response categories for closed questions.
- Response categories should be exhaustive (covering all possibilities) and mutually exclusive.
- Include “don’t know” or “no answer” options, but do not read them aloud.
- Use a natural, conversational style familiar to respondents.
Types of Questions
Open Questions
Open questions allow respondents to express themselves freely. They are useful for exploratory research but can be difficult to code.
Closed Questions
Closed questions provide pre-defined response options, making them easy to code but potentially limiting information.
Dichotomous Responses
Dichotomous responses offer two distinct choices.
List of Responses
Lists can be nominal, ordinal, or interval. Single or multiple responses can be allowed.
Semi-Open/Semi-Closed Questions
These questions offer pre-coded categories and an “other” option for additional input.
Confidence Level
The confidence level represents the probability that the estimated population parameter falls within a specific range. Each researcher sets their desired confidence level.
Sampling Methods
Probability Sampling
Probability sampling involves knowing the probability of each unit’s inclusion in the sample. This allows for error estimation and inferences about the population.
Simple Random Sampling
Individuals are randomly selected from a complete, up-to-date list of the population.
Systematic Sampling
Individuals are selected at a fixed interval (k) from a list.
Stratified Sampling
The population is divided into homogeneous strata, and random samples are drawn from each stratum proportional to its size.
Cluster Sampling
The population is divided into natural clusters (e.g., neighborhoods), and a sample of clusters is selected. A subsample is then drawn from each selected cluster.
Non-Probability Sampling
Non-probability sampling does not guarantee representativeness or allow for precise error estimation.
Accidental Sampling
The sample consists of readily available individuals.
Quota Sampling
Quotas are set for specific characteristics (e.g., gender, age), and sampling continues until the quotas are met.
Sources of Error in Sample Surveys
- Sampling Error: The difference between the sample value and the true population value.
- Coverage Error: Excluding certain population elements due to the selection process.
- Measurement Error: Unreliable responses due to interviewers, respondents, the questionnaire, or the interview situation.
- Non-Response Error: Lack of response from selected individuals (total non-response) or to specific questions (partial non-response).
Non-response can reduce sample size, increase sampling error, and affect representativeness if concentrated in a particular social sector.