Propensity Score Matching
Propensity score matching estimates a treatment effect from observational data by matching treated and control units on their estimated probability of treatment. This tutorial explains what a propensity score is, how RAISINS performs the matching, how to read the standardized mean difference and interpret the Love plot, and how to run the whole analysis code-free… Read more …
Propensity Score Matching (PSM) is a statistical technique for estimating the effect of a treatment, program, or exposure in observational data, where units were not randomly assigned to treated and control groups. It works by matching each treated unit to one or more control units with a similar estimated probability of receiving treatment (the propensity score), so that the matched groups look comparable on the covariates that went into the score, the way randomization would have made them comparable in an experiment.
This tutorial introduces the basic concepts of propensity score matching, including the propensity score itself, matching methods, calipers, and standardized mean differences. It then provides a step-by-step guide to performing propensity score matching in the RAISINS platform, using a real worked example on farm-level data. You will learn how to prepare your data, select the treatment, covariate, and outcome variables, interpret the matching and balance results, read the Love, jitter, and balance plots, and use the built-in RA-One AI assistant. No programming knowledge is required.
1 What is Propensity Score Matching?
Suppose you want to know whether farmers who adopted organic farming ended up with higher crop yield than those who did not. In a controlled experiment you would randomly assign farmers to “adopt” and “do not adopt”, so the two groups would be similar on everything else - farm size, rainfall, farmer age, education, access to extension services, access to credit - and any yield difference could be attributed to adoption itself.
In practice, adoption is never randomly assigned. Farmers who adopt organic methods may already have larger farms, better rainfall, more extension contact, or easier access to credit than farmers who do not. If you simply compare average yield between adopters and non-adopters, part of that difference reflects these pre-existing differences (confounders), not the effect of adoption itself.
Propensity score matching (PSM) addresses this by estimating, for every unit, a propensity score - the predicted probability of being in the treated group, given its covariates - and then matching treated units to control units with a similar propensity score. If matching succeeds, the matched treated and matched control groups are similar on the covariates that were used to build the score, and any remaining outcome difference is a much more credible estimate of the treatment’s effect.
Propensity score matching finds, for each treated unit, one or more control units with a similar estimated probability of being treated, so that the treated and control groups become comparable on the covariates used to build that probability.
1.1 The propensity score, matching, and the caliper
Three ideas drive every PSM analysis, and RAISINS reports them all separately.
- The propensity score is the predicted probability of receiving treatment, estimated from a logistic regression of the treatment indicator on the chosen covariates. It compresses many covariates into one number that both groups can be compared on.
- The matching method decides how treated and control units are paired once every unit has a propensity score. RAISINS supports nearest-neighbor matching (default - each treated unit is paired with the closest available control), optimal matching, full matching, subclassification, exact matching, and coarsened exact matching.
- The caliper is an optional maximum allowed distance between a treated unit’s propensity score and its matched control’s. A treated unit with no control within the caliper is left unmatched rather than forced into a poor match. A smaller caliper gives closer, more comparable matches, but discards more units and shrinks the usable sample.
\[\hat{e}(X_i) = \Pr(\text{Treatment}_i = 1 \mid X_i) \qquad \text{estimated via logistic regression}\]
Propensity score matching does not remove the influence of unmeasured confounders - only the covariates you actually put into the propensity-score model can be balanced. It also does not turn observational data into a randomized experiment; it is a way to make the comparison more credible, not to guarantee it is unbiased. Always report which covariates went into the model, and be explicit that residual confounding from unmeasured variables remains possible.
Matching also does not, by itself, tell you why two groups differ - only whether they were made comparable on the covariates supplied. The outcome comparison performed afterwards is what estimates the treatment effect.
2 How Good Is the Match? Standardized Mean Differences
A propensity-score model can converge and matching can run without ever checking whether it actually worked. The check is covariate balance: for every covariate, RAISINS compares its mean in the treated group and its mean in the control group, both before matching (on the whole sample) and after matching (on the matched sample only), and reports the difference as a standardized mean difference (SMD) - the mean difference divided by a pooled standard deviation, so covariates measured on very different scales (hectares, millimetres of rainfall, years) become directly comparable.
RAISINS, and the RA-One assistant, apply the same practical rule of thumb throughout:
- |SMD| < 0.1 - the covariate is well balanced between treated and control.
- 0.1 ≤ |SMD| < 0.25 - borderline; worth reporting but usually tolerable.
- |SMD| ≥ 0.25 - meaningfully imbalanced; the covariate still differs enough between groups to bias the outcome comparison.
Balance is judged on the SMD, not on a p-value from a group-mean test. A covariate can have a large mean difference with a non-significant t-test in a small sample and still be poorly balanced - always check the SMD.
3 Getting to the Module
Open the RAISINS home page at www.raisins.live and go to the Social Science / Causal Inference Tools section. Select the Propensity Score Matching module.
No programming knowledge is required. You upload your data, select the treatment, covariate, and outcome variables, choose a matching method, and RAISINS performs the complete analysis and produces publication-ready output.
3.1 Data Format
Your uploaded file (CSV or Excel) needs:
- A single Treatment column with exactly two levels (e.g.
0/1, orAdopted/Not Adopted). This is the exposure whose effect you want to estimate. - One or more covariate columns, qualitative (e.g. education level) or quantitative (e.g. farm size, rainfall), measured before treatment assignment. Never include a variable that is itself a consequence of treatment as a covariate.
- An Outcome (study variable) column, measured after treatment, that you eventually want to compare between the matched groups.
Each row is one observational unit (e.g. one farmer, one plot, one patient).
3.2 Computational Provenance & Reproducibility Record
CPRR (Computational Provenance & Reproducibility Record) provides a transparent and comprehensive record of the module’s computational workflow. It states the R version and the exact version of every package used, names the specific function behind each reported result, lists every default parameter and decision rule applied by the module, and provides fully runnable R code that reproduces each analytical step. Users can execute the code in R to independently reproduce and verify the results. It carries its own DOI.
To cite the platform itself in a paper, thesis, or report, use the RAISINS citation, available in APA, Harvard, and BibTeX formats at www.raisins.live/citation.html. That is the primary reference, and for most manuscripts it is all you need.
The CPRR for the propensity score matching module is at www.raisins.live/module_record/psm.html.
Cite the RAISINS paper as your primary reference for the platform. Add the CPRR as supporting documentation when a journal asks for details of the computing environment, or when you want your methods section to be precise about versions and functions rather than saying “matching was carried out using an online tool.” The CPRR supports the citation and ensures computational reproducibility.
4 Preview Mode and Quick Tour
You can explore the entire module before subscribing by using Preview mode on the Welcome page. It loads a built-in dataset, so you can try every feature, including the analysis and the plots, without uploading your own data.
First-time users are also offered a Quick Tour. This is an interactive, step-by-step walkthrough that highlights each tab and control and explains what it does. You can replay it at any time from the Quick Tour tab in the top navigation.
5 A Worked Example: Organic Farming Adoption
RAISINS ships a built-in dataset, Organic_Adoption, on 300 farmers: whether each adopted organic farming (Adoption, the treatment, coded 0/1), six pre-adoption covariates (farm size in hectares, rainfall in mm, farmer age, education level, extension contact, and credit access), and two post-adoption outcomes (yield in kg/ha and income in INR). Of the 300 farmers, 137 adopted organic farming and 163 did not - the two groups were not randomly assigned, so a naive comparison of yield between them would be confounded by these covariates.
5.1 Step 1 - Fit the Propensity-Score Model
With Adoption as the treatment and the six covariates selected, RAISINS fits a logistic regression and reports a coefficient table. In this worked example, three covariates were significant predictors of adoption at the 5% level: Extension_Contact (p = 0.001), Farm_Size_ha (p = 0.011), and Credit_Access (p = 0.047) - farmers with more extension contact, larger farms, and access to credit were more likely to adopt. The overall model was significant (likelihood-ratio chi-square = 38.50, df = 8, p < 0.001), with a McFadden pseudo R-squared of 0.093, meaning adoption is associated with these covariates but far from perfectly predicted by them - exactly the situation matching is meant to handle.
5.2 Step 2 - Match
Using nearest-neighbor matching with a caliper of 0.2 (standard-deviation units of the propensity score), RAISINS matched each treated farmer to the nearest available control farmer within the caliper. Of the 137 treated (adopting) farmers, 100 were matched and 37 were left unmatched because no sufficiently similar control farmer was available; correspondingly 100 of the 163 control farmers were used as matches and 63 remained unmatched. The final matched analytic sample is 200 farmers (100 treated, 100 control).
Unmatched units are not an error - they are farmers whose covariate profile has no comparable counterpart in the opposite group, so including them would only reintroduce the imbalance matching is trying to remove. A tighter caliper unmatches more units but produces closer, more credible matches; a looser (or no) caliper keeps more units but tolerates less similar pairs. RAISINS reports the matched/unmatched/discarded counts for every run so you can judge this trade-off.
5.3 Step 3 - Check Balance
Before matching, several covariates were meaningfully imbalanced: Extension_Contact had the largest imbalance (SMD = 0.48), followed by education level (the “Primary education” category, SMD = -0.34), Credit_Access (SMD = 0.28), and Farm_Size_ha (SMD = 0.27) - all above the 0.25 threshold. After matching, every covariate’s standardized mean difference fell under 0.15, and most fell under 0.08; the largest remaining imbalance (Extension_Contact, SMD = 0.146) is in the borderline range, and every other covariate is under the well-balanced 0.1 threshold. The mean standardized bias across all covariates dropped accordingly, confirming that matching substantially improved comparability between the adopting and non-adopting groups.
Organic_Adoption dataset.
| Covariate | SMD before | SMD after |
|---|---|---|
| Farm_Size_ha | 0.265 | 0.019 |
| Rainfall_mm | -0.185 | 0.041 |
| Farmer_Age | 0.101 | 0.020 |
| Extension_Contact | 0.483 | 0.146 |
| Credit_Access | 0.279 | 0.000 |
| Education (Primary) | -0.340 | -0.022 |
5.4 Step 4 - Compare the Outcome on the Matched Sample
On the 200 matched farmers, mean yield was 4235 kg/ha among adopters versus 3661 kg/ha among non-adopters, a difference of about 574 kg/ha (t-test on the matched sample, p < 0.001). Because the two groups are now balanced on the covariates that predicted adoption, this comparison is a substantially more credible estimate of the association between organic adoption and yield than the same comparison on the full, unmatched sample would have been - though, as always with observational data, it remains an estimate of association under the covariates measured, not proof of causation from unmeasured factors.
A matched-sample mean comparison is a simple, transparent way to summarise the outcome difference, but it is not the only way to estimate a treatment effect after matching. Doubly-robust estimators (which combine propensity-score weighting with an outcome regression) are more efficient and less sensitive to a misspecified propensity-score model; RAISINS’s built-in AI interpretation panel can produce these once an outcome variable is confirmed. Always report which estimator you used.
6 Reading the Plots
The Plots & Graphs tab offers five plot types, each fully customisable (labels, colours, fonts, themes) and downloadable in PNG, JPEG, TIFF, PDF, or SVG format.
- Before-Match Distribution - overlapping density curves of the estimated propensity score for the treated and control groups, on the full (unmatched) sample. Substantial non-overlap between the two curves is a warning sign: it means some treated units have no comparable control anywhere in the covariate space (poor common support), and matching cannot fix that region no matter which method or caliper you choose.
- After-Match Distribution - the same density curves, but computed only on the matched sample. A well-matched analysis shows the two curves sitting almost on top of each other.
- Love Plot - the signature PSM diagnostic. Each covariate is one row; two points show its standardized mean difference before and after matching, with a reference line (commonly at 0.1) marking the balance threshold. A good match pulls every point from outside the threshold (before) to inside it (after).
- Jitter Plot - every unit’s propensity score plotted as a jittered dot, split by treatment group and by matched/unmatched status. It gives an intuitive, unit-by-unit picture of which farmers were used in the matched sample and which were set aside.
- Balance Plot - density or empirical-CDF overlap of the propensity score (or, per-covariate, of an individual covariate), before versus after matching, for a more granular look than the single-number Love plot provides.
Check the Love plot first: if every covariate’s after-matching point sits left of the 0.1 (or at worst 0.25) reference line, the match is defensible. If several points remain to the right, consider a smaller caliper, a different matching method, or accept that some covariates cannot be balanced with the available data.
7 The RA-One AI Assistant
Once your analysis has run, the RA-One tab (and the floating chat bubble available throughout the Analysis workspace) gives you access to RAISINS’s built-in AI assistant. RA-One can:
- Explain propensity-score matching concepts in plain language - what a propensity score is, what a caliper does, how to interpret a standardized mean difference - without ever writing code for you.
- Interpret your own results: it reads your propensity-score model coefficients, matching summary, and before/after balance tables directly, and will tell you honestly if a value has not been computed rather than inventing one.
- Generate any of the five plots described above directly inside the chat, with the same customisation options as the main Plots & Graphs tab, on request (for example, “show me the Love plot” or “how balanced are my covariates after matching?”).
- Build a ready-to-download CSV data template (Treatment + covariates + Outcome columns) if you describe how many covariates and observations you need.
RA-One always distinguishes what it is told from what it is asked to guess: it will never fabricate a coefficient, a standardized mean difference, or a matched-sample count that is not in your actual results.
8 Reporting Your Results
A typical methods-section description of a RAISINS PSM analysis states: the matching method used (e.g. nearest-neighbor), the caliper (if any), the covariates included in the propensity-score model, the matched sample size (treated and control), and the standardized mean differences before and after matching - usually as a table like the one above. If you go on to compare outcomes, state clearly which estimator you used (a simple matched-sample comparison, or a doubly-robust estimator) and cite the RAISINS software (Hisham et al., 2025; R Core Team, 2024) alongside the MatchIt and cobalt R packages that perform the underlying computation (see the module’s CPRR for exact versions and citations).
Rosenbaum, P. R., & Rubin, D. B. (1983). The Central Role of the Propensity Score in Observational Studies for Causal Effects. Biometrika, 70(1), 41-55.
Ho, D., Imai, K., King, G., & Stuart, E. (2011). MatchIt: Nonparametric Preprocessing for Parametric Causal Inference. Journal of Statistical Software, 42(8), 1-28.
Stuart, E. A. (2010). Matching Methods for Causal Inference: A Review and a Look Forward. Statistical Science, 25(1), 1-21.
Austin, P. C. (2011). An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies. Multivariate Behavioral Research, 46(3), 399-424.
