back to portfolio
case study · survey experiment · qualtrics engineering · ordinal logit from scratch

Framing War — Partisan Identity & Support for Military Intervention

A pre-registered, five-condition survey experiment (N ≈ 2,350) asking whether Americans back military action because of the rationale (humanitarian vs. national security) or because of who is asking (their own party's elites). I engineered the instrument in Qualtrics — randomizer, branch logic, embedded-data pipeline, quality gates — fielded it on a national sample, and analyzed it with a from-scratch ordinal logit. The survey itself ships with this repo as an importable .qsf.

Qualtrics · survey flow · QSF2×2 + control Ordinal & Multinomial LogitNewton–RaphsonPre-registeredInteractive
2,199
respondents in the ordinal support model, after embedded-data quality screens
OR 1.43
Republican · Humanitarian frame — the strongest single mover (p = .003)
p .003
LR test: the effect of a cue depends on the respondent's own party (χ² 23.1, 8 df)

// 01 — the question

Classical realism treats public opinion on war as a sideshow to strategic interest. A large body of political-science research disagrees: in democracies, public sentiment constrains foreign policy, and that sentiment is shaped by partisan identity, elite discourse, and framing. The literature is unsettled on a key point — whether humanitarian or national-security justifications win broader support, and whether that depends on which party's leaders do the framing. I designed an experiment to isolate these forces causally, rather than infer them from observational polling that confounds framing with real-world events.

// 02 — design

Respondents completed a pre-treatment battery (baseline attitudes toward force, diplomacy-vs-coercion preferences, political knowledge, party ID), then were randomly assigned to one of five vignettes about a hypothetical crisis in "Country A." The design crosses the partisan source of the endorsement with the rationale, against a neutral control:

the 2×2 + control

Control (no cue) · Democratic/Humanitarian · Democratic/National-Security · Republican/Humanitarian · Republican/National-Security — 434–451 respondents per cell. Vignettes matched on length and style so only the partisan label and justification varied.

Fig 1. Mean 5-point support by condition (raw data, N = 2,199; cell sizes on bars). Humanitarian frames rise above control; national-security frames track it.

// 03 — the instrument: qualtrics engineering

The experiment lives or dies in the survey build. This one ran inside a multi-study omnibus, which means the assignment machinery had to be airtight: treatment is assigned by a block randomizer over embedded-data setters (not over blocks), so the assignment is written into the response record before any branch fires — the same field the analysis pipeline keys on.

interactive — the actual survey flow, simulated
respondent console
Press run 1 respondent to walk a simulated respondent through the flow — consent gate, quality checks, randomizer draw, vignette, outcomes. Or run 500 to watch the Evenly Present Elements balancing do its job (toggle it off to see what naive randomization does to cell balance).
Flow mirrors the fielded instrument 1:1 (consent → panel embedded data → demographics → pre-treatment → randomizer → arm branches → post → quality checks). Vignette text is the real stimulus. Simulated outcomes are drawn from the fitted interaction model in §06.
ship it: the survey as code

The full instrument is in this repo as framing-war.qsf — a standalone, importable Qualtrics survey file I extracted programmatically from the omnibus export: 5 treatment blocks, the randomizer + branch machinery, embedded-data pipeline, display-logic post probes, and quality gates, with the omnibus's other studies stripped out. Import it via Create project → Import a QSF file and the whole experiment is reproducible end-to-end — instrument, not just analysis.

// 04 — method & modeling

The primary outcome is ordered, so a linear model would invent distances between "somewhat" and "extremely" likely. I used an ordinal proportional-odds logit, which posits a latent support index sliced by ordered cutpoints:

logit Pr(Y ≤ j) = θj − x′β   · one β per covariate, shared across thresholds θ1..4

Rather than call a package, I coded the censored log-likelihood and maximized it with Newton–Raphson and a backtracking line search, parameterizing cutpoints as positive increments so they stay ordered. Two specifications — additive (treatment + party ID) and interaction (treatment × party ID) — adjudicated with a likelihood-ratio test. The secondary outcome uses a multinomial logit over the four concern categories.

# Proportional-odds ordinal logit, fit from scratch (no MASS::polr)
def negll(theta):
    beta, thetas = unpack(theta)        # thetas kept ordered: θj = θj-1 + e^d
    cum = sigmoid(thetas - X @ beta)    # Pr(Y <= j)
    p   = diff_to_category_probs(cum)   # Pr(Y = k)
    return -sum(log(p[arange(n), y-1]))
# maximized by Newton-Raphson + backtracking line search; SEs from the
# observed information. Additive vs. treatment×party compared by an LR test.

// 05 — estimates

Odds ratios from the additive model (reference = control condition, Democratic respondents). Values above one mean higher odds of greater support.

termodds ratio95% CIp
Republican · Humanitarian1.431.13–1.82.003
Democratic · Humanitarian1.200.94–1.52.135
Republican · National-Security1.070.84–1.35.591
Democratic · National-Security1.020.80–1.29.873
Respondent: Independent0.730.60–0.88<.001
Respondent: Republican0.920.77–1.10.368
Ordinal proportional-odds logit, N = 2,199. SEs from observed information.
Fig 2. Odds ratios with 95% CIs (log scale). The single strongest mover is a humanitarian rationale carried by a Republican source; national-security frames sit at the control baseline; Independents are markedly harder to move.

// 06 — the partisan interaction

The averages hide who is moving. Adding the treatment × party interaction improves fit decisively — the effect of a cue depends on the respondent's own identity:

LR χ² = 23.1 on 8 df · p = .003
Fig 3. Predicted P(support likely) under the humanitarian frame, by respondent party and cue source — the co-partisan crossover. Democrats swing up on a Democratic cue; Republicans show the mirror image; Independents stay flat.

Co-partisan appeals are the reliable channel: a Democratic humanitarian cue lifts Democrats from ~38% to ~50% predicted support; the same message from a Republican source moves them barely at all — and vice versa.

// 07 — interactive: predict a respondent

The widget runs the fitted interaction model in your browser. Choose an identity and the cue they hear; the predicted 5-point support distribution recomputes live.

mean support (1–5)
P(support likely)
Δ vs control (this party)
condition shown

// 08 — what drives opposition

A multinomial model of each respondent's single greatest concern shows the mechanism behind the support numbers — and how framing reshuffles what people worry about.

Fig 4. Top concern by condition. Civilian casualties and long-term entanglement dominate; their salience shifts with the frame — humanitarian cues make casualties loom larger.

// 09 — why it matters

Substantively, the result speaks to how democracies are talked into war: legitimacy is manufactured less by the objective stakes than by the alignment of moral framing with partisan identity.

Methodologically, the project runs the full experimental stack end-to-end: instrument engineering in Qualtrics (randomization, branching, embedded-data pipelines, quality gates), pre-registration, and hand-built estimators for ordered and categorical outcomes — the same toolkit that underpins rigorous A/B testing and product experimentation.

headline finding

Moral framing moves people — but mostly through the partisan channel. A humanitarian rationale carried by your own party's elites is the most reliable way to build support for intervention; national-security appeals barely move the needle once partisanship is accounted for.