Compare users exposed to certain factors (e.g., new dataset release) with similar users who were not. Propensity Score Matching (PSM) is a statistical technique used to estimate causal effects in observational studies where randomization is not feasible. The goals of PSM are to: 1. **Reduce Selection Bias**: Match treated and untreated groups based on similar characteristics to ensure comparability, mimicking the balance achieved in randomized experiments. 2. **Control for Confounding Variables**: Adjust for confounding variables that might affect both the treatment and the outcome. 3. **Estimate Causal Effects**: Infer the causal effect of a treatment (or intervention) by comparing outcomes between treated and matched untreated groups. ### Why use Propensity Score Matching? - **Non-Randomized Data**: In real-world settings, treatments or interventions are often not randomly assigned, leading to confounding. - **Balance Covariates**: PSM creates comparable groups by balancing covariates (background variables) that might bias the results. - **Clear Causal Interpretation**: By reducing confounding, PSM allows for a more accurate estimation of the treatment's effect. - **Transparency**: The method is intuitive and allows for diagnostics to assess the quality of matching. ### How Propensity Score Matching works 1. **Calculate Propensity Scores**: - Propensity scores are the probability of receiving the treatment, given a set of observed covariates. - For example, using logistic regression: $P(Treatment = 1 | Covariates) = \text{logistic}(\beta_0 + \beta_1 X_1 + \beta_2 X_2 + \ldots + \beta_k X_k)$ - Covariates can include age, income, education, etc., that may influence the likelihood of receiving the treatment. 2. **Match Treated and Untreated Units**: - Match each treated individual with one or more untreated individuals with similar propensity scores. - Common matching methods include: - **Nearest Neighbor Matching**: Pair treated individuals with the closest untreated individuals based on propensity scores. - **Caliper Matching**: Only match if the propensity score difference is below a threshold. - **Kernel Matching**: Weight untreated observations based on their distance to treated observations. 3. **Check Balance**: - Evaluate whether covariates are balanced between treated and untreated groups after matching (e.g., through standardized mean differences or visualizations). 4. **Estimate Treatment Effect**: - Compare outcomes (e.g., mean differences) between treated and matched untreated groups to estimate the Average Treatment Effect on the Treated (ATT). ### Example: Evaluating a training program's effect on income **Objective**: Determine whether attending a job training program (treatment) increases participants' income (outcome). **Step 1: Calculate Propensity Scores** - Covariates: Age, education level, prior income, work experience. - Use logistic regression to estimate the probability of attending the program (propensity score) for each individual. **Step 2: Match Participants** - Match program attendees (treated) with non-attendees (untreated) based on similar propensity scores. For example: - Treated individual with a propensity score of 0.75 is matched with an untreated individual with a score of 0.74. **Step 3: Check Balance** - Confirm that covariates (e.g., education level, prior income) are similar between the two groups after matching. **Step 4: Estimate Treatment Effect** - Calculate the difference in average income between the treated and matched untreated groups: - Average income of treated group: $50,000. - Average income of matched untreated group: $45,000. - Estimated causal effect (ATT): $50,000 - $45,000 = **$5,000**. ## Benefits and Limitations of PSM ### Benefits - Reduces bias by controlling for observed confounders. - Works with observational data. - Clear and interpretable results. ### Limitations - Unmeasured Confounders: Cannot adjust for variables that are not observed or included in the model. - Data Loss: May exclude unmatched individuals, reducing sample size. - Propensity Score Overlap: Requires sufficient overlap in propensity scores between treated and untreated groups.