pumBayes: Estimating Ideal Points from Voting Data in R Using Unfolding Models

The PUM

The PUM [30] is a one-dimensional scaling model closely related to item response models. It can be motivated through the use of random utility functions. To ground our presentation, we describe it using the usual political science terminology. In this context, we typically have I legislators/judges who vote in favor or against J issues. PUM assumes that the (latent) preferences of legislator/judge i, denoted by ${\beta }_i$, belong to a one-dimensional Euclidean latent “policy” space. It is common to refer to ${\beta }_i$ as the “ideal point” of legislator/judge i, as it represents their preferred policy. Similarly, each issue has associated with it three positions in the policy space, ${\psi }_{j,1}$, ${\psi }_{j,2}$ and ${\psi }_{j,3}$ such that either ${\psi }_{j,1}<{\psi }_{j,2}<{\psi }_{j,3}$ or ${\psi }_{j,3}<{\psi }_{j,2}<{\psi }_{j,1}$, where ${\psi }_{j,2}$ is associated with an affirmative vote, while both ${\psi }_{j,1}$ and ${\psi }_{j,3}$ are associated with a negative vote on issue j. Legislators/judges choose among these three options based on (random) utility functions of the form:

where ${ϵ}_{i,j,1}$, ${ϵ}_{i,j,2}$, and ${ϵ}_{i,j,3}$ are independent standard Gaussian shocks. Then, if we let $y_{i,j}\in \{0,1\}$ represent the vote of legislator/judge i on issue j, the probability of an affirmative vote is given by

where ${\alpha }_{j,1}=2({\psi }_{j,2}–{\psi }_{j,1}),{\alpha }_{j,2}=2({\psi }_{j,2}–{\psi }_{j,3}),{\delta }_{j,1}=({\psi }_{j,1}++{\psi }_{j,2})/2$ and ${\delta }_{j,2}=({\psi }_{j,3}+{\psi }_{j,2})/2$.

Note that this formulation includes a two-parameter probit IRT model as a limiting case. Indeed,

Prior distributions

pumBayes uses prior distributions for the issue-specific parameters $({𝜶}_j,{\mathit{\delta }}_j)$ that are independent across $j=1,\dots ,J$ and take the form

where ${𝜶}_j=({\alpha }_{j,1},{\alpha }_{j,2})$, ${\mathit{\delta }}_j=({\delta }_{j,1},{\delta }_{j,2})$, ${\text{N}}_q(\cdot \mid \mathit{\text{a}},\mathit{\text{A}})$ denotes the q-variate normal distribution with mean vector a and variance matrix A and, similarly, ${\text{TN}}_q(\cdot \mid \mathit{\text{a}},\mathit{\text{A}})𝟏(\Omega )$ denotes the q-variate normal distribution with mean vector a and variance matrix A truncated to $\Omega$.

For the static version of the model, pumBayes uses a standard normal distribution as the prior for the ideal point, i.e., ${\beta }_i\sim N(0,1)$ independently for all $i=1,\dots ,I$. For the dynamic version of the model, the ideal points are still assumed to be independent across individuals, but they are linked across time for a given individual using a first-order autoregressive process whose stationary distribution is again a standard normal distribution,

The autocorrelation parameter $\rho$ of this latent autoregressive process is then given a hyperprior (a Gaussian distribution truncated to the (0,1) interval) and its value learned from the data. This choice of prior distributions ensures that the parameters are identifiable to shifts and scalings of the latent policy space. To ensure identifiability to reflections of the policy space, users must identify individual(s) in the data for whom the sign of their ideal point will be fixed to be positive.

Posterior computation

pumBayes relies on an MCMC algorithm to generate samples from the posterior distribution of the model. The algorithm assumes that any missing data are missing completely at random. The construction of the sampler involves two data augmentation tricks. The first augmentation trick is reminiscent of that described in [31] and involves the introduction of vectors of latent variables, ${\mathit{\text{y}}}_{i,j}^{\ast }=(y_{i,j,1}^{\ast },y_{i,j,2}^{\ast },y_{i,j,3}^{\ast })$ for every $i=1,\dots ,I$ and $j=1,\dots ,J$. The definition of these latent variables is tightly linked to the form of the utility functions in (1):

where $e_{i,j,1}$, $e_{i,j,2}$ and $e_{i,j,3}$ are standard normal distributions. These augmentations ensure that most of the full conditional distributions belong to standard families from which direct sampling is possible.

The second augmentation trick breaks down the mixture prior on ${\mathit{\alpha }}_j$ and ${\mathit{\delta }}_j$ into the two fully connected regions associated with their support. In particular, for $j=1,\dots ,J$, we let $z_j=1$ if and only if ${\alpha }_{j,1}>0$ and ${\alpha }_{j,2}<0$, and $z_j=–1$ otherwise. Then, $\mathrm{Pr}(z_{i,j}=1)=\mathrm{Pr}(z_{i,j}=–1)=1/2$, and

This second augmentation is important to ensure that the algorithm can fully explore the posterior distribution of ${\mathit{\alpha }}_j$ and, in particular, that it can move between the quadrant where ${\alpha }_{j,1}>0$ and ${\alpha }_{j,2}<0$, and the quadrant where ${\alpha }_{j,1}<0$ and ${\alpha }_{j,2}>0$. To sample z_j, the algorithm relies on two Metropolis-Hasting steps with different relative frequencies. For the first of these moves, proposed values are generated by flipping the signs of ${\mathit{\alpha }}_j$ and ${\mathit{\delta }}_j$, while for the second move, the proposed values for ${\mathit{\alpha }}_j$ and ${\mathit{\delta }}_j$ are generated from the prior conditioned on the proposed quadrant. The first of these moves is expected to be most effective for items for which ${\mathit{\alpha }}_j$ is close to $𝟎$ (e.g., quasi-unanimous votes in our political science application). The number of $({\mathit{\alpha }}_j,{\mathit{\delta }}_j)$ pairs in this category is typically relatively low, but when the move is useful, the acceptance probability tends to be very high. Hence, it is often appropriate to attempt it relatively infrequently. On the other hand, the second move is most useful when the posterior distribution of $({\mathit{\alpha }}_j,{\mathit{\delta }}_j)$ is clearly bimodal. This move tends to be useful for a larger number of $({\mathit{\alpha }}_j,{\mathit{\delta }}_j)$ pairs, but the proposals also tend to have more moderate acceptance probabilities. Hence, we propose to perform it relatively more frequently. Full details of the algorithm can be found in [30] and [32].

Installing the package

pumBayes is a package built on top of the R software environment for statistical computing and graphics. For instruction on how to install R, see https://www.r-project.org/. pumBayes requires that various additional R packages be installed first. Please see the “Dependencies” section under “Availability” below.

pumBayes is available from the Comprehensive R Archive Network (CRAN) at https://cran.r-project.org/package=pumBayes and from the original GitHub site https://www.github.com/SkylarShiHub/pumBayes. pumBayes can be installed either directly from CRAN using the install.packages function in R:

or from GitHub using the install_github function in the devtools package:

Fitting static PUMs using pumBayes

We start by demonstrating the use of the function sample_pum_static to fit the static version of PUM. sample_pum_static can handle data in the form of either a rollcall object from the pscl package [17], or in the form of a logical matrix where TRUE values correspond to affirmative votes, FALSE values correspond to negative votes, and NAs correspond to any type of missing data. If a matrix object is used as input, the names of the rows are assumed to correspond to the legislators names.

For our demonstration, we use pumBayes to recover legislators’ preferences from roll-call voting data from the 116^th U.S. House of Representatives. The data can be downloaded from https://voteview.com using the pscl package.

h116 is an object of class rollcall. For our purposes, the key component of any rollcall object is votes, which, as the name suggests, contains the outcomes of the votes cast by each legislator who served in the House during this period. These outcomes are typically encoded as 1, 2 or 3 for an affirmative vote (“Yea”, “Paired Yea” and “Announced Yea”), 4, 5, or 6 for a negative vote (“Announced Nay”, “Paired Nay” and “Nay”), and 0, 7, 8, 9 or NA for different types of missing values (“Not Yet a Member”, “Present” and “Not Voting”). The component codes of the rollcall object typically includes an explanation of these codes.

Another useful component of a rollcall object is legis.data, which contains information about the legislators involved with this dataset.

Typically, datasets downloaded from https://voteview.com need to be preprocessed before they can be analyzed. The function preprocess_rollcall in pumBayes can handle transformations of the original rollcall object, including the removal of pre-specified legislators or votes, or the merging of pairs of legislators. In the specific case of h116, we will use this function to perform the following operations on the data:

• The President of the United States at the time (Donald Trump) needs to be removed. The following code identifies the row index in the matrix of votes that corresponds to him (note that there are no other legislators in this House that share the last name TRUMP):

  

• In the analysis of congressional voting records, lopsided (unanimous or quasi-unanimous) votes are often removed before scaling because they provide little or no information about legislators’ preferences. In these cases, we are interested in removing only completely unanimous votes.

  

• Two legislators (Justin Amash, representing MI-3, and Paul Mitchell, representing MI-10) left the Republican Party during this period to become independents. Hence, each of them appears twice in h116. For the purpose of this analysis, we merge their records before and after becoming independents and label the combined voting records for the whole 116^th House as such.

  

• Finally, after all the above filters are applied, we are interested in removing any legislator who missed more than 60% of the remaining votes from the dataset.

  

All of the above parameters are bound into a list that is passed as an argument to the function preprocess_rollcall. This function returns a “clean” version of the original rollcall object:

The final step before being able to fit the model is to select the hyperparameters. The default values used by pumBayes correspond to $𝝁={(–2,10)}^{’}$, $\omega =5$ and $\kappa =\sqrt{10}$. A way to evaluate the suitability of these default values is to visualize the implied prior on the probability of an affirmative outcome obtained from Equation (2). The function tune_hyper allows us to generate a random sample from this implied distribution, and a histogram of these samples provides a convenient visualization.

As Figure 1 shows, the default hyperparameter values imply a prior that places most of its mass close to either 0 or 1. This in turn implies an assumption that, for most votes, the majority of legislators are quite certain about how they will vote. This seems most appropriate in the context of this application. The prior also places slightly more mass close to 1 than it does close to 0. Again, this is appropriate, as the agenda-setting powers of the majority mean that most motions put to a vote in the U.S. House tend to pass, and therefore, receive more affirmative than negative votes.

Figure 1

The function tune_hyper can also be used to explore the implications of alternative prior specifications. For example, a prior that is concentrated around 0.5 can be obtained by setting $𝝁={(–40,0)}^{’}$, $\omega =0.1$ and $\kappa =5$ (please see Figure 2).

Figure 2

We are now ready to invoke the function sample_pum_static to sample from the posterior distribution. We aim at generating a total of 40,000 samples, of which the first 20,000 will be discarded as part of the burn-in period. These samples are obtained by thinning a longer chain every 10 iterations.

Typically, the ideological ranks implied by the ideal points are the key quantities of interest in this kind of analysis. These can be obtained using the function post_rank.

Note that the posterior median rank for Mo Brooks (who represented Alabama’s 5^th district) is 428, placing him among the most conservative Republicans in the House during this period. On the other hand, the posterior median rank for Martha Roby (representing Alabama’s 2^nd district) is 280, placing her as a centrist Republican.

It is illustrative to compare the ideological ranks estimated by PUM with those estimated using traditional approaches such as IDEAL [33], which is implemented in the pscl package:

An effective way to visualize this comparison is a scatterplot of the posterior median ranks estimated by the two models.

As Figure 3 shows, the rankings generated by PUM can differ significantly from those generated by IDEAL, especially for Democratic legislators. The top four discrepancies are for the members of the “Squad”: Representatives Alexandria Ocasio-Cortez (NY), Ilhan Omar (MN), Ayanna Pressley (MA), and Rashida Tlaib (MI). IDEAL positions them as centrists, while PUM places them among the most liberal in the Democratic caucus. Given the Squad’s advocacy for progressive policies, PUM’s ranking seems more accurate [19, 20].

Figure 3

The results look similar if we instead compare against the ranks generated using the wnominate package (see Figure 4):

Figure 4

As we mentioned in the introduction, the differences in the estimates generated by these models are due to the fact that PUM allows for both monotonic and non-monotonic response functions. To illustrate this, we construct estimates of the response functions associated with three separate votes using the function item_char in pumBayes:

As Figure 5 shows, PUM can capture both monotonic and non-monotonic response functions depending on the information contained in the observed data. Vote 5 (clerk session vote number 6) and vote 9 (clerk session vote number 10) are examples of votes with monotonic response functions, while vote 6 (clerk session vote number 7) is an example of a vote with a non-monotonic response function.

Figure 5

Comparisons between PUM and other Bayesian models can be carried out using a blocked version of the Watanabe–Akaike Information Criterion (WAIC) [34, 35, 36]. For a given House and model, the WAIC is given by

where ${\theta }_{i,j}$ represents the probability that legislator i votes “Aye” in issue j. Lower values of WAIC provide evidence in favor of that particular model. Like the Akaike Information Criterion and Bayesian Information Criterion, the WAIC attempts to balance model fit with model complexity. However, unlike these two criteria, WAIC is appropriate for hierarchical settings and is invariant to reparametrizations of the model.

To calculate WAIC values, we first calculate the probability array of voting “Aye” using functions predict_pum and predict_ideal contained in pumBayes. The output of each of these functions is a three-dimensional array, where the dimensions correspond to members, issues, and MCMC iterations. Then the function calc_waic can be used to evaluate the complexity-adjusted fit of both PUM and IDEAL models using Equation (3):

The values clearly indicate that PUM outperforms IDEAL on this particular dataset, as evidenced by its significantly smaller WAIC value.

Fitting dynamic PUMs using pumBayes

We now demonstrate the use of the function sample_pum_dynamic for estimating time-varying preferences. Data is passed into sample_pum_dynamic through two arguments. The first is a logical matrix containing the vote outcomes and in which TRUE values correspond to affirmative votes, FALSE values to negative votes, and NAs correspond to any type of missing data. This is the same as one of the formats accepted by sample_pum_static. The second argument is a vector that indicates the time period to which each vote belongs.

For our illustration, we focus on voting data for the U.S. Supreme Court between 1937 and 2021, which is available for download at https://mqscores.wustl.edu/replication.php. This dataset includes the voting outcomes on non-unanimous decisions, with the outcome being encoded as 1 (TRUE) if the judge voted to reverse the lower court decision, and as 0 (FALSE) if they voted to unhold it. The data is available as part of pumBayes through the object scotus.1937.2021.

The syntax for sample_pum_dynamic is very similar to that for sample_pum_static. Furthermore, note that we use the same hyperparameters for $({\mathit{\alpha }}_j,{\mathit{\delta }}_j)$ as we did in the static case:

The path of the ideal points is often a quantity of interest. For example, we might want to plot their posterior means and associated $95\%$ credible intervals for a few selected Justices.

Figure 6 suggests that Hugo Black’s preferences shifted substantially during his long service in the Court. Early in his tenure, his preferences tended to become more negative (’liberal’) over time. However, starting around 1950, his ideal point shows a consistent trend toward becoming more positive (“conservative”). Antonin Scalia’s preferences also seem to have evolved over time but, in his case, he became first more positive (“conservative”) and then more negative (“liberal”). In contrast to these two Justices, Warren E. Burger’s preferences remained relatively steady during his term as Chief Justice.

Figure 6

A salient feature of PUM is that it can sometimes produce posterior distributions for the ideal points that are bimodal. Hugo Black’s preferences during the late 60s is a good example (see Figure 7).

Figure 7

A key hyperparameter in dynamic PUM models is the autocorrelation coefficient $\rho$ of the underlying autoregressive process. To investigate the performance of the model, we might want to compare the prior and posterior distributions for this parameter. Figure 8 shows that, in this dataset, the posterior distribution of $\rho$ is centered close to the prior, but is much more concentrated. Indeed, its posterior mean is 0.891, and the associated 95% posterior credible interval is (0.863,0.905).

Figure 8

Similar to the static case, it is illustrative to compare the complexity-adjusted fit of PUM against that of the more standard dynamic IRT model proposed by [37], which is implemented by the function MCMCdynamicIRT1d in the package MCMCpack [8]:

Similar to the static case, the functions predict_pum and predict_irt can be used to calculate the predicted voting probabilities across MCMC samples, and calc_waic can be used to evaluate the WAIC score for each term. The difference between the score under the standard model and PUM can then be displayed as a line plot.

As seen in Figure 9, the difference in WAIC is positive on most terms, suggesting that PUM tends to provide a better explanation to the U.S. Supreme Court Justices’ voting patterns than the standard dynamic IRT model.

Figure 9