Measurement, Sampling, Research Design, and Surveys in Quantitative Research
Unit 6: Chapter 5: Principles of Measurement
Define Measurement and Understand its Role in Quantitative Research
- Measurement involves assigning numbers to data to mark characteristics, linking conceptual ideas to empirical observations. In quantitative research, measurement is used to gather data to answer questions. It serves as both a descriptive and evaluative device.
Distinguish Between Categorical and Continuous Data (Nominal vs. Ordinal, Interval, Ratio)
- Categorical data includes nominal and ordinal levels, while continuous data includes interval and ratio levels. Nominal data describes the presence or absence of a characteristic, while ordinal data ranks elements. Interval data has specific numerical scores with equal intervals but no absolute zero, and ratio data has an absolute zero point.
Identify and Define the 4 Levels of Measurement, Distinguish their Characteristics, and Understand Limitations of Each
- The four levels of measurement are nominal, ordinal, interval, and ratio. Each level has distinct characteristics and limitations.
- Nominal: Involves categories with no inherent order. It simply categorizes data into mutually exclusive groups. Examples include gender, eye color, and marital status. The main limitation of nominal data is that it does not allow for any meaningful mathematical operations.
- Ordinal: Involves ranking or ordering data without precise measurement of the intervals between them. Examples include ranking preferences or opinions. While ordinal data provides a rank order, the intervals between ranks may not be equal, and there is no true zero point.
- Interval: Involves numerical data where the intervals between values are equal, but there is no true zero point. Examples include temperature measured in Celsius or Fahrenheit. Interval data allows for meaningful mathematical operations such as addition and subtraction, but it does not have a true zero, making ratios meaningless.
- Ratio: Involves numerical data with a true zero point, where ratios are meaningful. Examples include height, weight, and income. Ratio data allows for all mathematical operations, including multiplication and division, and meaningful ratios can be calculated.
Reliability and Validity – Define them and Understand the Relationship Between them
- Reliability refers to the consistency and stability of measurement, while validity refers to the accuracy and truthfulness. Reliability is necessary but not sufficient for validity. The reliability coefficient (Cronbach’s alpha) is a numeric value reflecting the reliability of a measure, typically calculated for ordinal and interval-level measures.
- Validity: The extent to which the device/instrument measures what it is supposed to.
- Reliability is necessary, but not sufficient, for validity.
Define the Reliability Coefficient (Chronbach’s Alpha) and Know for Which Levels of Measurement it is Calculated
- The reliability coefficient, such as Cronbach’s alpha, reflects the reliability of a measure. It is typically calculated for ordinal and interval-level measures to assess internal reliability.
- A numeric value between 0 and 1 that reflects the reliability of a measure; 0.70 or above is deemed “acceptable”.
Understand Threats to Validity Related to Data Collection and Sampling
- Threats to validity include issues related to data collection and sampling, such as instrument validity, researcher behavior, maturation, mortality, and attrition. These factors can impact the accuracy and truthfulness of collected data, affecting the validity of study findings.
Unit 7: Chapter 6: Principles and Techniques of Sampling
Identify, Define, and Distinguish Population, Sampling Frame, and Sample, and Understand the Relationships Between them
- Population: All people or units possessing the characteristics in which the researcher is interested.
- Sampling Frame: The set of people or units that have a chance to be sampled.
- Sample: People or units selected for participation in a research study.
- Relationship between them: The sampling frame serves as the basis for selecting the sample from the population. The sample represents a subset of the population, and data collected from the sample are used to make inferences about the entire population.
Understand the Goal of Sampling and Related Concepts, Specifically Generalizability and Representativeness
- Goal of Sampling: The primary goal of sampling is to select a subset of the population (i.e., the sample) in a way that allows the researcher to make accurate inferences about the entire population.
- Generalizability: Refers to the extent to which the findings from a study conducted on a sample can be applied or generalized to the larger population from which the sample was drawn.
- Representativeness: Refers to the degree to which the characteristics of the sample accurately reflect the characteristics of the population.
Understand Probability Sampling and When/Why it is Used
- Probability Sampling: A sampling technique in which every individual or unit in the population has a known and non-zero chance of being selected for the sample.
- When/Why it is used: Probability sampling is typically used when the goal is to obtain a sample that accurately represents the population and allows for the generalization of findings.
Random Selection
- Random Selection: A fundamental aspect of probability sampling, where every member of the population has an equal chance of being selected for the sample.
EPSEM (Equal Probability of Selection Method)
- EPSEM: EPSEM, or Equal Probability of Selection Method, is a principle in probability sampling where every unit in the population has the same probability of being selected for the sample.
Sampling Error
- Sampling Error: Refers to the discrepancy or difference between the characteristics of the sample and the characteristics of the population.
Understand Non-probability Sampling and When/Why it is Used
- Non-probability Sampling: A sampling technique in which individuals or units are selected for the sample based on criteria other than random selection.
- When/Why it is used: Non-probability sampling is often used when probability sampling methods are impractical or impossible to implement.
Understand How Probability and Non-probability Sampling Techniques Impact Representativeness and Generalizability
- Impact on Representativeness and Generalizability: Probability sampling techniques tend to produce samples that are more representative of the population and have higher generalizability. Non-probability sampling techniques may result in samples that are less representative and have lower generalizability.
Considerations in Determining Sample Size
- Sample Size Determination: Determining the appropriate sample size involves considering factors such as the desired level of precision, the variability of the population, the research objectives, and the available resources.
Unit 8: Chapter 7: Research Design
Identify and Define 3 Basic Forms of Research Design and Understand How they Differ on 2 Key Characteristics
- Experiments: In experiments, researchers intentionally change how the independent variable (IV) is presented to participants and then observe the effects on the dependent variable (DV). Random assignment of participants to different conditions is a key characteristic.
- Quasi-experiments: Quasi-experiments occur when the manipulation of the IV happens naturally, rather than being controlled by the researcher. They lack random assignment to conditions.
- Surveys: Surveys involve collecting data from a sample of individuals through self-reported responses to structured questionnaires or interviews. They do not involve manipulation of the IV or random assignment to conditions.
Experimental Design: Definition, Goals, Characteristics
- Experimental design involves recording measurements and observations under defined conditions to establish causation between variables. The primary goal is to determine the effect of the independent variable (IV) on the dependent variable (DV). Characteristics include manipulation of the IV, random assignment to conditions, and control over extraneous variables.
Key Terms: Experimental Stimulus, Manipulation Check
- Experimental stimulus: The event or item to which participant response is measured. Participants in the experimental group are exposed to the experimental stimulus.
- Manipulation check: A procedure to verify if participants interpreted the manipulation of the IV as intended by the researcher. It ensures that differences in the DV between experimental and control groups are due to the manipulation of the IV.
Know the Types and their Strengths and Limitations
- Posttest-only design: Participants are exposed to the experimental condition, and then the DV is measured. Strengths include simplicity and reduced risk of testing effects. Limitations include the inability to establish baseline equivalence between groups.
- Pretest-posttest design: Participants are measured on the DV both before and after exposure to the experimental condition. Strengths include control for initial differences between groups. Limitations include the potential for pretest sensitization.
- Solomon Four design: Combines elements of posttest-only and pretest-posttest designs, including experimental and control groups with and without pretests. Strengths include the ability to assess pretest sensitization effects. Limitations include increased complexity.
- Factorial design: Involves manipulating two or more IVs to examine their combined effects on the DV. Strengths include the ability to test interactions between IVs. Limitations include increased complexity and the potential for interaction effects.
- Longitudinal design: Measures participants on the DV multiple times over an extended period to assess changes over time. Strengths include the ability to examine developmental trends. Limitations include attrition and the time-consuming nature of the study.
Quasi-experimental and Descriptive Designs: Characteristics, Strengths, and Limitations
- Quasi-experimental designs: Involve naturally occurring variations in the IV without random assignment to conditions. Strengths include high external validity. Limitations include lower internal validity due to lack of randomization.
- Descriptive designs: Non-experimental studies that do not involve manipulation of the IV or random assignment to conditions. Strengths include simplicity and applicability to real-world settings. Limitations include the inability to establish causation.
Unit 9: Chapter 8: Surveys & Questionnaires
Surveys, Questionnaires, and Polls – Definitions and Distinctions Between them (Especially Between Surveys and Questionnaires)
- Definitions and distinctions between them: Surveys provide information about attitudes, characteristics, and behaviors of respondents and the populations they represent. Questionnaires are the only method of data collection in surveys, while polls address issues of national or social concern with a statewide or nationwide sample.
Cross-sectional vs. Longitudinal Survey Designs
- Cross-sectional surveys collect data at one point in time and examine relationships between variables. In contrast, longitudinal surveys collect data at several points in time, capturing the process of communication, with each period termed a ‘wave’.
Self-report Surveys: Face-to-face, Phone, Mail, Internet
- Self-report surveys are the most common survey technique and allow for anonymous data collection. They can be conducted face-to-face, over the phone, through mail, or via the Internet.
Advantages and Disadvantages of Each Method
Face-to-Face Surveys:
- Advantages:
- Build rapport and trust between interviewer and respondent.
- Allow for clarification of questions and encouragement of participation.
- Capture nonverbal cues and reactions.
- Disadvantages:
- Expensive due to the need for trained interviewers and travel costs.
- Time-consuming, especially for large samples or lengthy surveys.
- The presence of the interviewer may lead to social desirability bias in responses.
Phone Surveys:
- Advantages:
- Cost-efficient, especially for large samples.
- Allow for random digit dialing and reach a diverse population.
- Quick data collection with large samples.
- Disadvantages:
- Ineffective for lengthy surveys or sensitive topics.
- Resistance and barriers from potential respondents, such as call screening or refusal to participate.
- Increasing use of cell phones reduces reach and effectiveness.
Mail Surveys:
- Advantages:
- Respondents can give thoughtful answers and complete surveys at their convenience.
- Suitable for complex or detailed questions.
- Reach dispersed populations and allow for anonymity.
- Disadvantages:
- Low response rates and delays in returns.
- Costs associated with printing, postage, and mailing.
- Need for accurate mailing addresses and the potential for lost surveys.
Internet Surveys:
- Advantages:
- Quick and inexpensive data collection.
- Respondents can complete surveys at their own pace and on their own time.
- Allow for multimedia integration and branching logic in survey design.
- Disadvantages:
- Challenges in identifying and reaching specific populations or samples.
- Security concerns regarding data privacy and protection.
- Lack of control over the respondent environment may affect response quality.
Each survey method has its unique advantages and disadvantages, and the choice depends on factors such as research objectives, target population, budget, and time constraints.
Response Rate Considerations
- The response rate is calculated as the number of surveys completed or returned compared to those sent out. Factors influencing response rates include the importance of the survey issues, user-friendliness, and how the survey was administered.
Privacy Considerations
- Self-report surveys allow for anonymous data collection, which can encourage honest responses. However, concerns about security exist, especially with internet surveys.
General Guidelines for Writing Survey Items
- Items should be clear and unambiguous, avoid double-barreled questions, ensure respondents are competent to answer, be relevant to respondents, avoid leading questions, and consider social desirability. Use existing measures when possible and provide clear instructions for completing the survey.