Statistical Inference in Missing Data by MCMC and Non-MCMC Multiple Imputation Algorithms: Assessing the Effects of Between-Imputation Iterations

Masayoshi Takahashi

doi:10.5334/dsj-2017-037

Figures & Tables

Table 1

Variables and Missing Rates.

Variables	Missing Rates
GDP per capita (purchasing power parity)	0.0%
Freedom House index	15.4%
Central bank discount rate	32.9%
Life expectancy at birth	2.6%
Unemployment rate	10.5%
Distribution of family income: Gini index	37.3%
Public debt	22.4%
Education expenditures	24.6%
Taxes and other revenues	6.1%
Military expenditures	43.0%

[i] Data sources: CIA (2016) and Freedom House (2016).

Table 2

Multiple Regression Analyses on GDP Per Capita.

	Incomplete Data		Multiply-Imputed Data
Variables	Coef.	Std. Err.	Coef.	Std. Err.
Intercept	–7.323	3.953	–11.545*	3.495
Freedom	–0.321*	0.127	–0.362*	0.127
Central Bank	–0.118*	0.041	–0.107	0.049
Life Expectancy	3.922*	0.794	4.908*	0.655
Unemployment	–0.205*	0.087	–0.214*	0.070
Gini	0.114	0.253	–0.018	0.363
Public Debt	–0.198*	0.092	–0.002	0.093
Education	0.035	0.164	–0.488*	0.154
Tax	0.357*	0.174	0.471*	0.151
Military	0.123	0.085	0.299*	0.109
Number of obs.		86		228

[i] Note: *significant at the 5% error level. Coef. stands for coefficient. Std. Err. stands for standard error. Since the distributions of these variables are skewed to the right (log-normal), the variables are log-transformed to normalize the distributions.

Table 3

Relations among DA, EMB, and FCS.

	Joint Modeling	Conditional Modeling
MCMC	DA	FCS
Non-MCMC	EMB

Table 4

Summary of the 20 Studies on Multiple Imputation.

Authors	MI Algorithms	Sample Size	Number of Variables	Number of Imputations	Number of Iterations	Missing Rate
Barnard and Rubin (1999)	DA	10, 20, 30	2	3, 5, 10	Unknown	10%, 20%, 30%
Horton and Lipsitz (2001)	DA, FCS	10000	3	10	200	50%
Schafer and Graham (2002)	DA	50	2	20	Unknown	73%
Donders et al. (2006)	FCS	500	2	10	Unknown	40%
Abe and Iwasaki (2007)	DA	100	4	5	100	20%, 30%
Horton and Kleinman (2007)	DA, EMB, FCS	133774	10	10	5	41%
Stuart et al. (2009)	FCS	9186	400	10	10	18%
Lee and Carlin (2010)	DA, FCS	1000	8	20	10	33%
Leite and Beretvas (2010)	DA	400	10	10	Unknown	10%, 30%, 50%
Hardt, Herke, and Leonhart (2012)	DA, EMB, FCS	50, 100, 200	3, 13, 23, 43, 83	20	Unknown	20%, 50%
Lee and Carlin (2012)	DA	1000	8	20	Unknown	10%, 25%, 50%, 75%, 90%
Cranmer and Gill (2013)	EMB, MHD	500	5	Unknown	NA	20%, 50%, 80%
Cheema (2014)	FCS	10, 20, 50, 100, 200, 500, 1000, 2000, 5000, 10000	4	Unknown	Unknown	1%, 2%, 5%, 10%, 20%
Kropko et al. (2014)	DA, EMB, FCS	1000	8	5	30	25%
Shara et al. (2015)	Unknown	2246	8	Unknown	Unknown	20%, 30%, 40%
Deng et al. (2016)	FCS	100	200, 1000	10	20	40%
von Hippel (2016)	DA	25, 100	2	5	Unknown	50%
Hughes, Sterne, and Tilling (2016)	Unknown	100, 1000	5	50	Unknown	40%, 60%
McNeish (2017)	DA, FCS	20, 50, 100, 250	4	5, 25, 100	Unknown	10%, 20%, 30%, 50%

[i] Note: DA stands for Data Augmentation, EMis for Expectation-Maximization with Importance Sampling, FCS for Fully Conditional Specification, EMB for Expectation-Maximization with Bootstrapping, and MHD for Multiple Hot Deck. Unknown means that information is unavailable. NA means Not-Applicable.

Table 5

Abbreviations and the Missing Data Methods.

Abbreviations	Missing Data Methods
CD	Complete data without missing values
LD	Listwise deletion
EMB	MI by AMELIA II
DA1	MI by NORM2 with no iterations
DA2	MI by NORM2 with 2*EM iterations
FCS1	MI by MICE with no iterations
FCS2	MI by MICE with 2*EM iterations
D-SI	Deterministic SI by `norm.predict` in MICE
S-SI	Stochastic SI by `norm.nob` in MICE

Table 6

Bias and RMSE (Theoretical Data).

2	3	4	5	6	7	8	9	10
CD	Bias	0.001	0.003	0.001	0.002	0.001	0.001	0.001	0.002	0.001
RMSE	0.040	0.047	0.038	0.039	0.058	0.026	0.046	0.039	0.047
LD	Bias	0.032	0.135	0.105	0.104	0.332	0.085	0.129	0.210	0.116
RMSE	0.059	0.153	0.122	0.121	0.349	0.103	0.160	0.228	0.155
EMB	Bias	0.000	0.004	0.002	0.000	0.005	0.001	0.005	0.005	0.002
RMSE	0.046	0.053	0.050	0.051	0.075	0.041	0.069	0.059	0.072
DA1	Bias	0.001	0.002	0.003	0.001	0.001	0.000	0.003	0.003	0.002
RMSE	0.046	0.053	0.050	0.051	0.074	0.041	0.069	0.058	0.072
DA2	Bias	0.002	0.001	0.005	0.002	0.001	0.000	0.001	0.003	0.000
RMSE	0.046	0.053	0.050	0.051	0.074	0.041	0.069	0.058	0.072
FCS1	Bias	0.002	0.001	0.082	0.040	0.090	0.047	0.093	0.027	0.233
RMSE	0.047	0.053	0.097	0.062	0.116	0.065	0.109	0.052	0.239
FCS2	Bias	0.001	0.002	0.004	0.002	0.001	0.000	0.001	0.002	0.001
RMSE	0.046	0.053	0.050	0.051	0.075	0.041	0.069	0.058	0.071
D-SI	Bias	0.186	0.242	0.174	0.093	0.187	0.098	0.231	0.070	0.163
RMSE	0.192	0.248	0.182	0.110	0.207	0.109	0.248	0.099	0.189
S-SI	Bias	0.002	0.000	0.081	0.038	0.090	0.047	0.091	0.029	0.230
RMSE	0.050	0.057	0.102	0.066	0.124	0.076	0.119	0.062	0.241

[i] Note: Biased results are in boldface, i.e., Bias > 0.010.

Table 7

Coverage of the 95% CI (Theoretical Data).

	2	3	4	5	6	7	8	9	10
CD	95.3	94.9	94.2	94.0	96.0	96.0	95.3	94.9	94.6
LD	88.5	47.9	54.6	56.7	10.8	65.1	69.2	32.1	78.1
EMB	95.0	95.1	94.2	95.5	94.9	94.4	94.3	94.1	95.0
DA1	94.6	94.9	93.2	93.1	94.1	91.8	92.9	92.4	92.9
DA2	94.3	95.8	95.1	94.1	94.8	94.3	94.2	93.2	94.9
FCS1	94.2	95.0	75.0	91.6	84.4	95.5	84.5	96.8	6.8
FCS2	94.7	95.6	94.4	93.9	95.4	94.5	94.2	95.0	95.0
D-SI	0.8	0.2	2.2	37.8	22.2	16.9	8.3	51.0	22.5
S-SI	88.9	89.6	47.8	75.0	62.3	64.4	48.9	76.0	3.7

[i] Note: Confidence invalid results are in boldface, i.e., outside of 93.6 and 96.4.

Table 8

Lengths of the 95% CI (Theoretical Data).

	2	3	4	5	6	7	8	9	10
CD	0.157	0.184	0.144	0.148	0.236	0.102	0.184	0.151	0.180
LD	0.189	0.259	0.226	0.235	0.384	0.213	0.358	0.339	0.390
EMB	0.178	0.209	0.196	0.200	0.301	0.160	0.275	0.229	0.281
DA1	0.176	0.207	0.187	0.192	0.293	0.145	0.256	0.208	0.253
DA2	0.177	0.208	0.194	0.198	0.298	0.158	0.271	0.223	0.274
FCS1	0.178	0.209	0.237	0.211	0.324	0.248	0.306	0.223	0.299
FCS2	0.178	0.209	0.197	0.201	0.302	0.161	0.275	0.228	0.281
D-SI	0.143	0.174	0.133	0.149	0.244	0.103	0.205	0.150	0.188
S-SI	0.157	0.184	0.161	0.155	0.238	0.145	0.188	0.149	0.186

Table 9

Computational Time (Theoretical Data).

	2	3	4	5	6	7	8	9	10
EMB	0.46	0.53	0.53	0.59	0.71	0.78	0.97	1.27	1.69
DA2	0.10	0.16	0.29	0.42	0.55	1.09	1.39	2.22	3.63
FCS2	2.47	5.98	14.48	21.33	25.40	54.71	59.14	85.69	133.17

[i] Note: Reported values are the time in seconds to perform multiple imputation, which is averaged over 1,000 simulation runs. The fastest results are in boldface.

Table 10

Bias and RMSE (Realistic Data).

		2	3	4	5	6	7	8	9	10
CD	Bias	0.003	0.002	0.002	0.002	0.001	0.002	0.000	0.002	0.002
RMSE	0.074	0.086	0.068	0.067	0.066	0.065	0.070	0.069	0.075
LD	Bias	0.034	0.047	0.037	0.054	0.082	0.099	0.083	0.072	0.085
RMSE	0.095	0.128	0.104	0.118	0.141	0.154	0.157	0.159	0.188
EMB	Bias	0.001	0.002	0.002	0.005	0.001	0.000	0.000	0.002	0.006
RMSE	0.084	0.113	0.091	0.090	0.089	0.092	0.102	0.099	0.110
DA1	Bias	0.006	0.001	0.003	0.003	0.001	0.001	0.001	0.001	0.002
RMSE	0.084	0.112	0.090	0.089	0.087	0.091	0.100	0.096	0.105
DA2	Bias	0.009	0.000	0.002	0.004	0.002	0.004	0.000	0.001	0.001
RMSE	0.084	0.111	0.089	0.088	0.086	0.090	0.098	0.094	0.102
FCS1	Bias	0.007	0.013	0.006	0.005	0.002	0.008	0.006	0.012	0.000
RMSE	0.084	0.106	0.081	0.081	0.080	0.081	0.086	0.083	0.088
FCS2	Bias	0.007	0.001	0.002	0.002	0.003	0.005	0.002	0.003	0.005
RMSE	0.084	0.112	0.088	0.088	0.086	0.090	0.097	0.093	0.100
D-SI	Bias	0.188	0.075	0.011	0.035	0.037	0.047	0.023	0.034	0.059
RMSE	0.207	0.163	0.115	0.118	0.118	0.123	0.130	0.127	0.151
S-SI	Bias	0.005	0.014	0.007	0.006	0.002	0.006	0.005	0.009	0.006
RMSE	0.089	0.116	0.096	0.095	0.091	0.094	0.100	0.102	0.105

[i] Note: Biased results are in boldface, i.e., Bias > 0.010.

Table 11

Coverage of the 95% CI (Realistic Data).

	2	3	4	5	6	7	8	9	10
CD	94.6	95.3	95.8	94.7	95.2	96.4	94.6	95.3	94.8
LD	92.2	91.6	92.8	91.5	86.8	85.0	89.8	90.0	90.8
EMB	94.3	94.1	94.7	93.9	96.1	94.2	94.0	94.4	94.7
DA1	94.1	92.2	94.4	93.4	95.7	92.2	93.1	92.9	93.1
DA2	94.0	94.0	94.8	94.4	95.9	94.5	93.8	95.0	95.0
FCS1	94.6	94.7	96.3	96.7	97.0	97.0	96.7	96.9	97.7
FCS2	94.7	93.8	95.5	95.7	96.4	94.3	94.8	95.2	96.1
D-SI	32.7	74.5	79.2	77.6	77.7	74.1	75.3	75.1	68.8
S-SI	87.9	83.2	82.3	82.5	84.2	82.1	81.0	80.3	81.2