Imputation of Missing Covariates in Longitudinal Data

class: title-slide-custom
count: false

<div style="font-size: 46pt; font-weight: 700; color: var(--nord4); margin-top: 60px;">
Imputation of Missing Covariates in Longitudinal Data</div>

<div style="font-size: 26pt; color: var(--nord9); font-weight: 700; margin-top:40px;">
MICE and Alternatives
</div>

<div id = "author">
<div style = "font-size: 32pt; font-weight: bolder; color: var(--nord4);"> Nicole Erler</div>
<div style = "color: var(--nord9);">Department of Biostatistics</div>
</div>

<div id="contact">
<i class="fas fa-envelope"></i> n.erler@erasmusmc.nl &emsp;
<a href="https://twitter.com/N_Erler"><i class="fab fa-twitter"></i> N_Erler</a> &emsp;
<a href="https://github.com/NErler"><i class="fab fa-github"></i> NErler</a> &emsp;
<a href="https://nerler.com"><i class="fas fa-globe-americas"></i> https://nerler.com</a>
</div>

---
layout: true

<div class="my-footer"><span>
<a href="https://twitter.com/N_Erler"><i class="fab fa-twitter"></i> N_Erler</a>
&emsp;&emsp;&emsp;&emsp;
<a href="https://github.com/NErler"><i class="fab fa-github"></i> NErler</a> &emsp;&emsp;&emsp;&emsp;
<a href = "https://nerler.com"><i class="fas fa-globe-americas"></i> nerler.com</a>
</span></div>

---

## Why?

.nord0box[
> "Mixed models can handle missing values ."

]

<br>

&#8680; Fit mixed model to complete cases only?

???

You probably heard somewhere that analysis using mixed models is valid as long
as the **missing data mechanism** is MAR.
Which means that if we can **make the assumption** of MAR, a 
complete case analysis  should be fine.

So why then do we need to impute at all, when we can make that assumption?

- - - -

* Only applies to missing values in the **response**!!!
* Missing data mechanism needs to be **ignorable**.

???

This statement only applies to missing values in the response, but not
when there are missing values in covariates.

- - - -

<div style = "width: 50%; margin-top: 50px;">
<strong>In most cases</strong> complete case analysis is
<span style = "color:var(--nord11); font-weight = 300;">
biased
</span> and
<span style = "color:var(--nord11); font-weight = 300;">
inefficient
</span>.
</div>

???

Which means that in most cases **complete case analysis** is actually
**biased and inefficient**

---
class: center, middle, animated, fadeIn

# The Basics

---

## Imputation of Missing Covariates

<div style = "width: 75%; padding: 1.15em 2em; background-color: var(--nord0); margin: auto; display: block;">
Sample from the predictive <strong>distribution</strong> of the <strong>missing
values given</strong> the <strong>observed values</strong>.
</div>

.footnote[
[<i class = "fas fa-book"></i> Rubin (2004)](https://doi.org/10.1198/000313004X6355)
]

???

Let's start at the begining.
In a setting where we have

* missing values in covariates, and 
* the missingness is ignorable

we can impute by **sampling** from the **predictive distr. of the missing values 
given the observed values**.

- - - - -

<br>

`\begin{align*}
p(\color{var(--nord15)}{x_{mis}} \mid \text{everything else}) = p(\color{var(--nord15)}{x_{mis}} \;\mid\; & \text{outcome},\\
& \text{other covariates},\\
&\text{parameters})
\end{align*}`

???

This is the distribution of an incomplete covariate `$x$` conditional on 
everything else,

where: everything else = outcome, other covariates, and parameters.

---

## A Simple Example

.gr-left[

* `$\mathbf y$`: **response** variable
* `$\color{var(--nord15)}{\mathbf x_1}$`: **incomplete** covariate
* `$\mathbf x_2$`, `$\mathbf x_3$`: **complete** covariates

]

???

Let's look at a simple example.

Imagine we have the following data set, with
* a response variable `$y$`
* an incomplete covariate `$x_1$` which is missing for subject `$i$`
* and two completely observed covariates `$x_2$` and `$x_3$`

- - - - -

.gr-right[
**Idea:** "Predict" missing values<br>
* Fit a model to the cases with observed
`$\color{var(--nord15)}{\mathbf x_1}$`, e.g.
`$$\color{var(--nord15)}{\mathbf x_1} = \beta_0 + \beta_1 \mathbf y +
\beta_2 \mathbf x_2 + \beta_3 \mathbf x_3 + \varepsilon$$`
to get estimates `$\boldsymbol{\hat\beta}$` (and `$\hat\sigma$`)
{{content}}
]

???

The idea for imputation of `$x_1$` is similar to predicting the missing values.

We can think of this as fitting a model to the cases where `$x_1$` is observed,
**for example**, this could be a **linear regression model** that has the 
**response `$y$`** and the other covariates as its covariates.

From this model, we can get estimates for the regression coefficients `$\beta$`
and the standard deviation of the residuals `$\sigma$`.

- - - - -

* Calculate the fitted value(s)
`$\color{var(--nord15)}{\mathbf{\hat{x}_1}}$`
for the case(s) with missing `$\color{var(--nord15)}{\mathbf{x_1}}$`

???

And using these parameter estimates we can then calculate the fitted or
predicted value of `$x_1$` for those cases where `$x_1$` is missing.

---

## Imputation of Covariates

.nord0box[
**Important:** We need to take into account uncertainty!
]

???

But we can't just use the fitted value to impute the missing value.

We need to take into account that there is uncertainty.

- - - - -

<br>

Uncertainty about
- the **parameter estimates** `$\boldsymbol{\hat{\beta}}$`
(and `$\hat\sigma$`)
- the **fitted/predicted value**
`$\color{var(--nord15)}{\mathbf{\hat{x}_1}}$`

???

Uncertainty about the
* parameter estimates,
* and about the fitted values

---

## Bayesian Imputation

Parameters and missing values are **random variables**<br>
&#8680; `$\boldsymbol\beta$`, `$\sigma$` and
`$\color{var(--nord15}{\mathbf x_1}$` have probability **distributions**.

???

The most natural way to take this uncertainty into account is by **making the
assumption** that the parameters and the missing values are **random variables**
and **have probability distribution**.

This assumption is made in the Bayesian paradigm.

- - - - -

Sample imputed values for `$\color{var(--nord15)}{x_{i1}}$` from:
`$$p(\color{var(--nord15)}{x_{i1}} \mid \mathbf x_{-i1}, \mathbf x_2, \mathbf x_3, \mathbf y) = \int p(\color{var(--nord15)}{x_{i1}} \mid y_i, x_{i2}, x_{i3}, \boldsymbol\beta, \sigma)\;p(\boldsymbol\beta, \sigma \mid \mathbf y_{-i}, \mathbf x_{-i1}, \mathbf x_{-i2}, \mathbf x_{-i3})\, d\boldsymbol\beta\, d\sigma$$`
???
And under this paradigm, we would then sample the imputed value for `$x_{i1}$`
from this distribution here, where we would have to integrate out the
unknown parameters.

In practice, we usually have **multiple variables** with missing values and 
**multiple parameters**.
So you probably **don't really want to have to solve these integrals**
especially not if you have to do it by hand.

And that's the **reason** why multiple imputation was developed.
That was in the 1970s, where **computers** weren't quite as **strong and widely
available** as they are today, and doing imputation in the **Bayesian 
framework** wasn't really **feasible**.

---

## Multiple Imputation

**"Approximation" of Bayesian imputation:**<br>
&#8680; Use a **small number of samples** of `$\boldsymbol \beta$` (and `$\sigma$`)
<span style = "color: var(--nord3);">instead of the full distributions</span>

???
The idea in multiple imputation is to **approximate Bayesian imputation** by
just working with a **small number of samples** instead of the full
distributions.
- - - -

.nord0box[
* **Sample** a realization of `$\color{var(--nord8}{\hat{\boldsymbol\beta}}$` and `$\color{var(--nord8)}{\hat\sigma}$`<br>

]

???

How does this work?
- first, we sample a realization of the parameters from their posterior distribution

* Calculate expected value of `$\color{var(--nord15)}{\hat x_{i1}}$`,<br>
  <span style = "color: var(--nord3);">e.g.,
  `$\displaystyle \mu_{i1} = \hat\beta_0 + \hat \beta_1 y_i + \hat \beta_2 x_{i2} +   \hat\beta_3 x_{i3}$`</span><br>

???

- Then, using this realizations, we calculate the expected value of the missing
value

* **Sample imputed value** `$\color{var(--nord15)}{\hat x_{i1}}$` from 
  `$p(\color{var(--nord15)}{x_i} \mid \mu_{i1}, \hat\sigma)$`,<br> 
  <span style = "color: var(--nord3);">
  e.g., `$p(\color{var(--nord15)}{x_i} \mid \mu_{i1}, \hat\sigma) = N(\mu_{i1}, \hat\sigma^2)$`</span>

???
And using this expected value we can define the distribution we assume for the 
missing value, and sample an imputed value from that distribution.

---

## Multiple Imputation

Fill `$\hat x_{i1}$` into original data &#8680; **1 imputed dataset**<br>
&#8680; Repeat multiple times to create **multiple imputed datasets**.

???

We can then **fill in** this imputed value into our original data and have
thereby created one completed dataset.

And to create **multiple imputed datasets** we repeat this multiple times.
- - - -

MI has 3 steps:
1. Impute (multiple times)
2. **Analyse** each dataset separately.
3. Combine results: **pooling**

???
Once we have created our imputed datasets, we can continue with the other two
steps of multiple imputation
- analysing each complete dataset separately, and
- combine the results, which we call pooling

- - - 
--

<br>

Pooling must take into account 
* **within imputation** uncertainty
* **between imputation** variation

<div style = "position: relative; left: 550px; bottom: 130px; width: 30%">
.nord0box[
&#8680; "Rubin's Rules"
]
</div>

???

And in the pooling we need to take into account that there is 
- **within** imputation uncertainty, (the **uncertainty about the parameters**
  from each analysis)
- **between** imputation uncertainty (the **results from each analysis differ**
  from each other)

And the **most commonly used method** to do this was developed by
**Donald Rubin** and is therefore called Rubin's Rules.

---

## Imputation Step

**The Tricky Part:**
* We need a (correct) model for 
  `$p(\color{var(--nord15)}{\mathbf x_{mis}} \mid \text{everything else})$`.
  
???
The imputation step is the tricky part of MI. 
We need to specify a correct model for the distribution of the missing part of
the data given everything else.
--
* Multiple variables could have missing values.<br>
???

But in practice, multiple variables may have missing values.
--

<ul class = "arrow-list">
  <li>$\color{var(--nord15)}{\mathbf x_{mis}}$ is multivariate</li>
  </ul>
???
Which means that `$x_{mis}$` here is multivariate, and probably of mixed type
--

<ul class = "arrow-list">
  <li> $p(\color{var(--nord15)}{\mathbf x_{mis}} \mid \text{everything else})$ is multivariate</li>
  </ul>
???
and so the distribution also is multivariate
--

<ul class = "arrow-list">
  <li> usually no closed form</li>
  </ul>

<img src = "gifs/panik.gif", height = 250; style = "position: absolute; right: 40px; bottom: 200px;">
???
and will probably not have a known closed form
- - - - -
--

**Two main approaches:**
* MICE 
  <span style = "color: var(--nord3);">multivariate imputation by chained equations</span><br>
  FCS 
  <span style = "color: var(--nord3);">fully conditional specification</span><br>
  
* Joint Model MI   <span style = "color: var(--nord3);">multiple imputation</span>

???
There are two main approaches to solve this problem:
* MICE, short for **multivariate imputation by chained equations**, also called
  **fully conditional specification**
* Joint model multiple imputation

In the following I will give you a quick overview of what the idea behind these
two approaches is, and then we will move on to multi-level data

---

## MICE / FCS

Uses the **idea of the Gibbs sampler:**
<div style = "width: 60%;">
Split the multivariate problem into a set of univariate problems
</div>

<img src = "graphics/gibbs.png", height = 400; style = "position: absolute; right: 60px; top: 50px;">
  
???
MICE is based on the idea of the Gibbs sampler.
The Gibbs sampler is a Markov Chain Monte Carlo algorithm, so it comes from the Bayesian world, and it allows us to split a multivariate problem into
a set of univariate problems.
When you need to sample from a multivariate distribution, you can instead 
sample from each of univariate full conditional distributions, meaning the
distributions of one of these variable given all the others.
And when you do that in turn, over and over again, the sample that you obtain
is a sample from the multivariate distribution.

<br><br>
  
* Cycle through the incomplete variables.
* Impute each variable using a **full conditional model**<br>
  <span style = "color: var(--nord3);">(e.g., regression model using all other 
  variables)</span>
* Repeat until convergence.

.footnote[
[<i class = "fas fa-book"></i> Van Buuren (2018)](https://stefvanbuuren.name/fimd/)&emsp;
[<i class = "fab fa-r-project"></i> mice](https://amices.org/mice/)
]

---

## MICE / FCS

.gr-left2[

Create random starting imputations.<br>

Impute
* `$\mathbf x_1$` using `$\mathbf y$`, `$\mathbf x_2$`, `$\mathbf x_3$`, ...
* `$\mathbf x_2$` using `$\mathbf y$`, `$\mathbf x_1$`, `$\mathbf x_3$`, ...
* `$\mathbf x_3$` using `$\mathbf y$`, `$\mathbf x_1$`, `$\mathbf x_2$`, ...<br><br>
* `$\mathbf x_1$` using `$\mathbf y$`, `$\mathbf x_2$`, `$\mathbf x_3$`, ...
* `$\mathbf x_2$` using `$\mathbf y$`, `$\mathbf x_1$`, `$\mathbf x_3$`, ...
* `$\mathbf x_3$` using `$\mathbf y$`, `$\mathbf x_1$`, `$\mathbf x_2$`, ...

]

.gr-right2[
<table class="simpletable">
<tr>
<th></th>
<th>$\mathbf y$</th>
<th>$\mathbf x_1$</th>
<th>$\mathbf x_2$</th>
<th>$\mathbf x_3$</th>
</tr>
<tr><td></td><td colspan = "4"; style = "padding: 0px;"><hr /></td><tr>
<td class="rownr"></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
</tr>
<tr>
<td class="rownr">$i$</td>
<td><i class = "fas fa-check"</i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
</tr>
<tr>
<td class="rownr"></td>
<td><i class = "fas fa-check"</i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td><i class = "fas fa-check"</i></td>
</tr>
<tr>
<td class="rownr"></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
</tr>
<tr>
<td class = "rownr"></td>
<td>$\vdots$</td>
<td>$\vdots$</td>
<td>$\vdots$</td>
<td>$\vdots$</td>
</tr>
</table>
]

---

## MICE / FCS

.pull-left[

**The Idea:**

]

.pull-right[
**In Practice:**

]

---

## Joint Model MI

Approximate `$p(\color{var(--nord15)}{\mathbf x_{mis}} \mid \text{everything else})$` with a known multivariate distribution

Usually: **multivariate normal**

&#8680; each variable is assumed to be (latent) normally distributed

<br>

---
class: center, middle, animated, fadeIn

# Multi-level Data

???

Not that we've covered the basics about multiple imputation and its most
commonly used approaches, MICE, and the assumption of a multivariate
normal distribution, let's take a look how things change in multi-level data.

---

## Multi-level Data

.gr-left2[
<img src="figures/trajectories_all.png", height = 400, style = "margin: auto; display: block;">
]

.gr-right2[

<table class="simpletable">
<tr>
<th></th>
<th>$\mathbf y$</th>
<th>$\mathbf x_1$</th>
<th>$\mathbf x_2$</th>
<th>$\mathbf x_3$</th>
</tr>
<tr><td></td><td colspan = "4"; style = "padding: 0px;"><hr /></td><tr>
<td class="rownr"></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
</tr>
<tr class="hlgt-row">
<td class="rownr">$i$</td>
<td><i class = "fas fa-check"</i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
</tr>
<tr class = "hlgt-row">
<td class="rownr">$i$</td>
<td><i class = "fas fa-check"</i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
</tr>
<tr class = "hlgt-row">
<td class="rownr">$i$</td>
<td><i class = "fas fa-check"</i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
</tr>
<tr>
<td class="rownr"></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
</tr>
<tr>
<td class="rownr"></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
</tr>
<tr>
<td class = "rownr"></td>
<td>$\vdots$</td>
<td>$\vdots$</td>
<td>$\vdots$</td>
<td>$\vdots$</td>
</tr>
</table>
]

???

We have multi-level data when we have measured the same variable repeatedly in
the same patient, but also when we have a clustering structure in our data,
for example in a multi-center study. In both cases, observations from the 
same patient, or the same cluster are not independent.

We typically represent this type of data in long format, so that we now have
multiple rows that belong to the same patient.

---
class: animated, fadeIn

## Multi-level Data

.gr-left2[
<img src="figures/trajectories_allb.png", height = 400, style = "margin: auto; display: block;">
]

.gr-right2[
* observations of the same patient / cluster are **correlated**

<br>

* **unbalanced** data
]

???
Besides the fact that now some rows in our data are correlated, we often have
another feature: unbalanced data.
(different number of measurements, taken at different time points)

---

## Multi-level Data

**(Linear) Mixed Model**
`$$y_{ij} = \underset{\text{fixed effects}}{\underbrace{\mathbf x_{ij}^\top\boldsymbol\beta}} + 
\underset{\text{random effects}}{\underbrace{\mathbf z_{ij}^\top\mathbf b_i}} +
\boldsymbol\varepsilon_i$$`

<br>

* **level-1** variables:<br>repeatedly measured / time-varying
* **level-2** variables:<br>baseline / patient (or cluster) specific / time-constant

???

For analysis: &#8680; typically use a mixed model

* takes into account that the repeated measurements for a patient are not independent
* can handle unbalanced data

Things get interesting when we have missing values in a baseline covariate:<br>
&#8680; when imputing we do not only need to take into account that multiple
missing values may belong to the same patient and should therefore be correlated,
**but that they should be identical.**

---

## FCS in Multi-level Data

If `$\color{var(--nord15)}{\mathbf x_1}$` is **level-1**:<br>
`$p(\color{var(--nord15}{x_{i1}(t)} \mid \text{everything else})$` could be
a mixed model, e.g.:
`$$x_{i1}(t) = 
\underset{\color{var(--nord3)}{\text{fixed effects}}}{\color{var(--nord3)}{\underbrace{\color{var(--nord4)}{\theta_0 + \theta_1 y_i(t) + \theta_2 x_{2i}(t) + \theta_3 x_{3i}(t)}}}} + 
\underset{\color{var(--nord3)}{\substack{\text{random}\\\text{effects}}}}{\color{var(--nord3)}{\underbrace{\color{var(--nord4)}{\mathbf u_i \mathbf z_i(t)}}}} +
\varepsilon_i(t)$$`

But what if `$\color{var(--nord15)}{\mathbf x_1}$` is a **level-2** variable?
???
If `$x_1$` is a level-2 variable:

* imputed values for the same subject should be identical
--

* Use a mixed model?&emsp;
--
<i class="fas fa-thumbs-down" style = "color:var(--nord11);"></i>
--

* Use a GLM?&emsp;
--
<i class="fas fa-thumbs-down" style = "color:var(--nord11);"></i>
--

<br>

**<span style="font-size:1.5rem;">&#8680;</span> Imputation in wide format?**

???
&#8680; For incomplete baseline variables imputation in wide format might be better(?)

---

## Imputation in Wide Format?

---
count: false
class: animated, fadeIn

## Imputation in Wide Format?
<img src="figures/wideform1.png", height = 450, style = "margin: auto; display: block;">

---
class: animated, fadeIn

## Imputation in Wide Format?
<img src="figures/wideform2.png", height = 450, style = "margin: auto; display: block;">

---
class: animated, fadeIn

## Imputation in Wide Format?
<img src="figures/wideform3.png", height = 450, style = "margin: auto; display: block;">

???

* In yellow: unnecessary imputations = imputations that we would not need
for the analysis with a mixed model 
* in red: imputations after death

* we would impute all these values for the outcome and each time-varying covariate

---

## Imputation in Wide Format?

* may **not** be **feasible**
* if feasible:
  * may **require summarizing** the data
  * can be (very) **inefficient**

--
<br><br>

**Alternatives?**

---

## Imputation in Wide Format

.pull-left[
<img src="figures/p_first.png" style="width:100%">
]
--
.pull-right[
<img src="figures/p_mean.png" style="width:100%">
]

---

## Imputation in Wide Format

.pull-left[
<img src="figures/p_rd.png" style="width:100%">
]
--
.pull-right[
<img src="figures/p_ri.png" style="width:100%">
]

---
count: false

## Imputation in Wide Format

.pull-left[
<img src="figures/p_rd.png" style="width:100%">
]

.pull-right[
<img src="figures/p_rs.png" style="width:100%">
]

---

## Imputation in Wide Format

.gr-left2[
<img src = "figures/p_ns.png", height = 450>
]

.gr-right2[
<table class="simpletable">
<tr>
<th></th>
<th style = "color: var(--nord10);">$\mathbf b_0$</th>
<th style = "color: var(--nord10);">$\mathbf b_1$</th>
<th style = "color: var(--nord10);">$\mathbf b_2$</th>
<th>$\mathbf x_1$</th>
<th>$\mathbf x_2$</th>
<th>$\mathbf x_3$</th>
</tr>
<tr><td></td><td colspan = "6"; style = "padding: 0px;"><hr /></td><tr>
<td class="rownr"></td>
<td style = "color: var(--nord10);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord10);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord10);"><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
</tr>
<tr>
<td class="rownr"></td>
<td style = "color: var(--nord10);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord10);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord10);"><i class = "fas fa-check"</i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
</tr>
<tr class = "hlgt-row">
<td class="rownr">$i$</td>
<td style = "color: var(--nord10);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord10);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord10);"><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
</tr>
<tr>
<td class="rownr"></td>
<td style = "color: var(--nord10);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord10);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord10);"><i class = "fas fa-check"</i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td><i class = "fas fa-check"</i></td>
</tr>
<tr>
<td class="rownr"></td>
<td style = "color: var(--nord10);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord10);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord10);"><i class = "fas fa-check"</i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td><i class = "fas fa-check"</i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
</tr>
<tr>
<td class="rownr"></td>
<td style = "color: var(--nord10);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord10);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord10);"><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td><i class = "fas fa-check"</i></td>
</tr>
<tr>
<td class = "rownr"></td>
<td style = "color: var(--nord10);">$\vdots$</td>
<td style = "color: var(--nord10);">$\vdots$</td>
<td style = "color: var(--nord10);">$\vdots$</td>
<td>$\vdots$</td>
<td>$\vdots$</td>
<td>$\vdots$</td>
</tr>
</table>

.footnote[
[<i class = "far fa-hand-point-right"></i> Practical (EP16)](https://nerler.github.io/EP16_Multiple_Imputation/practical/07_Imputation_of_Longitudinal_Data.html#imputation_using_mice)
]
]

---

## Imputation in Wide Format

Simple summaries of longitudinal variables:<br>
* may introduce bias

<br>
Summarize longitudinal trajectories using random effects:

<ul class="fa-ul">
<li><span class="fa-li" style = "color:var(--nord14);">
<i class="far fa-smile"></i></span>
more efficient</li>
<li><span class="fa-li" style = "color:var(--nord13);">
<i class="far fa-meh"></i></span>
requires sufficient fit</li>
<li><span class="fa-li" style = "color:var(--nord11);">
<i class="far fa-frown"></i></span>
only for incomplete baseline covariates</li>
</ul>

.footnote[
[<i class = "fas fa-book"></i> Erler (2016)](https://doi.org/10.1002/sim.6944)
]

---

## Imputation of Missing Covariates

Specifying the **correct imputation** model directly is **not straightforward** for

* multi-level settings
--

* GLMs with non-linear associations
* time-to-event outcomes

<br>

"Classic" FCS / MICE specifies imputation models directly.

.nord0box[
**<span style="font-size: 1.5rem;">&#8680;</span>
We need another approach in these settings.**
]

---
class: center, middle, animated, fadeIn

# Joint Model Multiple Imputation

---

## Joint Model Multiple Imputation

**Idea:** approximate 
`$p(\color{var(--nord15)}{\mathbf x_{mis}} \mid \text{everything else})$` as
**multivariate normal**

.pull-left[

<table class="simpletable">
<tr>
<th></th>
<th>$\mathbf y$</th>
<th style = "color: var(--nord15);">$\mathbf x_1$</th>
<th style = "color: var(--nord15);">$\mathbf x_2$</th>
<th style = "color: var(--nord15);">$\mathbf x_3$</th>
<th style = "color: var(--nord7);" colspan="3">$\mathbf X_{comp}$</th>
</tr>
<tr><td></td><td colspan = "7"; style = "padding: 0px;"><hr /></td><tr>
<td class="rownr"></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord7);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord7);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord7);">$\ldots$</td>
</tr>
<tr>
<tr class = "hlgt-row">
<td class="rownr">$i$</td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td style = "color: var(--nord7);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord7);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord7);">$\ldots$</td>
</tr>
<tr class = "hlgt-row">
<td class="rownr">$i$</td>
<td><i class = "fas fa-check"</i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td style = "color: var(--nord7);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord7);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord7);">$\ldots$</td>
</tr>
<tr class = "hlgt-row">
<td class="rownr">$i$</td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td style = "color: var(--nord7);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord7);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord7);">$\ldots$</td>
</tr>
<tr>
<td class="rownr"></td>
<td><i class = "fas fa-check"</i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td><i class = "fas fa-check"</i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td style = "color: var(--nord7);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord7);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord7);">$\ldots$</td>
</tr>
<tr>
<td class="rownr"></td>
<td><i class = "fas fa-check"</i></td>
<td><i class = "fas fa-check"</i></td>
<td style="color: var(--nord15);"><i class = "fas fa-question"></i></td>
<td><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord7);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord7);"><i class = "fas fa-check"</i></td>
<td style = "color: var(--nord7);">$\ldots$</td>
</tr>
<tr>
<td class = "rownr"></td>
<td>$\vdots$</td>
<td>$\vdots$</td>
<td>$\vdots$</td>
<td>$\vdots$</td>
<td style = "color: var(--nord7);">$\vdots$</td>
<td style = "color: var(--nord7);">$\vdots$</td>
<td style = "color: var(--nord7);">$\vdots$</td>
</tr>
</table>

]

.pull-right[

* `$\mathbf y$`: response (complete)
* `$\color{var(--nord15)}{\mathbf x_1}$`: time-varying (incomplete)
* `$\color{var(--nord15)}{\mathbf x_2}$`,
  `$\color{var(--nord15)}{\mathbf x_3}$`: baseline (incomplete)
* `$\color{var(--nord7)}{\mathbf X_{comp}}$`: other covariates (complete)

.footnote[
[<i class = "fas fa-book"></i> Carpenter & Kenward (20103)](https://www.wiley.com/en-us/Multiple+Imputation+and+its+Application+-p-9780470740521)&emsp;
[<i class = "fab fa-r-project"></i> jomo](https://CRAN.R-project.org/package=jomo)
]

]

---
class: animated, fadeIn

## Joint Model Multiple Imputation

.gr-left[

]

.gr-right[
`\begin{align*}
\boldsymbol y &= \color{var(--nord7}{\mathbf X_{comp}^\top} \boldsymbol\theta_y + 
                  \mathbf b_y \mathbf Z_y + \boldsymbol\varepsilon_y\\
\color{var(--nord15)}{x_1} &= \color{var(--nord7}{\mathbf X_{comp}^\top} \boldsymbol\theta_1 +
\mathbf b_1 \mathbf Z_1 + \boldsymbol\varepsilon_1\\
\color{var(--nord15)}{x_2} &= \color{var(--nord7}{\mathbf X_{comp}^\top} \boldsymbol\theta_2 +
\boldsymbol\varepsilon_2\\
\color{var(--nord15)}{x_3} &= \color{var(--nord7}{\mathbf X_{comp}^\top} \boldsymbol\theta_3 +
\boldsymbol\varepsilon_3
\end{align*}`

]

---
class: animated, fadeIn
count: false

## Joint Model Multiple Imputation

.gr-left[

]

.gr-right[
`\begin{align*}
\boldsymbol y &= \color{var(--nord7}{\mathbf X_{comp}^\top} \boldsymbol\theta_y + 
                  \color{var(--nord10}{\mathbf b_y} \mathbf Z_y +
                  \boldsymbol\varepsilon_y\\
\color{var(--nord15)}{x_1} &= \color{var(--nord7}{\mathbf X_{comp}^\top} \boldsymbol\theta_1 +
\color{var(--nord10}{\mathbf b_1} \mathbf Z_1 + \boldsymbol\varepsilon_1\\
\color{var(--nord15)}{x_2} &= \color{var(--nord7}{\mathbf X_{comp}^\top} \boldsymbol\theta_2 +
\color{var(--nord10}{\boldsymbol\varepsilon_2}\\
\color{var(--nord15)}{x_3} &= \color{var(--nord7}{\mathbf X_{comp}^\top} \boldsymbol\theta_3 +
\color{var(--nord10}{\boldsymbol\varepsilon_3}
\end{align*}`

$$
`\begin{pmatrix}
\color{var(--nord10)}{\mathbf b_y}\\
\color{var(--nord10)}{\mathbf b_1}\\
\color{var(--nord10)}{\mathbf \varepsilon_2}\\
\color{var(--nord10)}{\mathbf \varepsilon_3}
\end{pmatrix}`
\sim N(\mathbf 0, \mathbf V)
$$

]

---
class: animated, fadeIn
count: false

## Joint Model Multiple Imputation

.gr-left[

]

.gr-right[
`\begin{align*}
\boldsymbol y &= \color{var(--nord7}{\mathbf X_{comp}^\top} \boldsymbol\theta_y + 
                  \mathbf b_y \mathbf Z_y + \color{var(--nord10)}{\boldsymbol\varepsilon_y}\\
\color{var(--nord15)}{x_1} &= \color{var(--nord7}{\mathbf X_{comp}^\top}
\boldsymbol\theta_1 + \mathbf b_1 \mathbf Z_1 +
\color{var(--nord10)}{\boldsymbol\varepsilon_1}\\
\color{var(--nord15)}{x_2} &= \color{var(--nord7}{\mathbf X_{comp}^\top}
\boldsymbol\theta_2 + \boldsymbol\varepsilon_2\\
\color{var(--nord15)}{x_3} &= \color{var(--nord7}{\mathbf X_{comp}^\top}
\boldsymbol\theta_3 + \boldsymbol\varepsilon_3
\end{align*}`

$$
`\begin{pmatrix}
\mathbf b_y\\
\mathbf b_1\\
\mathbf \varepsilon_2\\
\mathbf \varepsilon_3
\end{pmatrix}`
\sim N(\mathbf 0, \mathbf V)
\qquad
`\begin{pmatrix}
\color{var(--nord10)}{\mathbf \varepsilon_y}\\
\color{var(--nord10)}{\mathbf \varepsilon_1}
\end{pmatrix}`
\sim N(\mathbf 0, \mathbf W)
$$

]

---
class: center, middle, animated, fadeIn

# Bayesian Analysis

---

## Getting the Correct Distribution

.gr-left2[
* We need `$\;p(\color{var(--nord15)}{\mathbf x} \mid \mathbf y, \ldots)$`
* We know `$\;p(\mathbf y \mid \color{var(--nord15)}{\mathbf x}, \ldots)$`
]

.gr-right2[

<div style = "color: var(--nord3);">
<ul>
<li>$\mathbf x$: incomplete covariate</li>
<li>$\mathbf y$: outcome</li>
<li>$\ldots$: everything else </li>
</ul>
</div>
]

???

* We need to sample from the probability distribution of the incomplete variable
`$x$` given the outcome `$y$` and everything else, including other variables and
parameters.

* But we only know the distribution of the outcome `$y$` given the incomplete and
other covariates, because this is our analysis model.

* But Reverend Thomas Bayes comes to the rescue:

**Bayes Theorem:**

???

Bayes Theorem tells us how we can derive the conditional distribution that we
need from the one that we have.

---
count: false
class: animated, fadeIn

## Getting the Correct Distribution

.gr-left2[
* We need `$\;p(\color{var(--nord15)}{\mathbf x} \mid \mathbf y, \ldots)$`
* We know `$\;p(\mathbf y \mid \color{var(--nord15)}{\mathbf x}, \ldots)$`
]

.gr-right2[

<div style = "color: var(--nord3);">
<ul>
<li>$\mathbf x$: incomplete covariate</li>
<li>$\mathbf y$: outcome</li>
<li>$\ldots$: everything else </li>
</ul>
</div>
]

**Bayes Theorem:**

`$$p(\color{var(--nord15)}{\mathbf x} \mid \mathbf y, \ldots) = 
\frac{p(\mathbf y \mid \color{var(--nord15)}{\mathbf x}, \ldots)\;
p(\color{var(--nord15)}{\mathbf x},\ldots)}{p\left(\mathbf y, \ldots\right)}
\propto p(\mathbf y \mid \color{var(--nord15)}{\mathbf x}, \ldots)\;
\underset{\downarrow}{p(\color{var(--nord15)}{\mathbf x},\ldots)}$$`
<div style="font-size: 0.8rem;width: 20%; position: relative; left: 750px; bottom: 15px;">
joint distribution of everything except the outcome
</div>

???

Denominator does not contain our incomplete variable `$x$`

&#8680; we can omit that part and save us some integration

&#8680; represent the cond. distribution of `$x$` given `$y$` as the product of

* the distribution of the analysis model and
* the joint distribution of the incomplete variable `$x$` and everything else.

---

## How to specify `$p(\color{var(--nord15)}{\mathbf x}, \ldots)$`?

For example:
`$$p(\color{var(--nord15)}{\mathbf x_1}, \mathbf x_2, \mathbf x_3, \boldsymbol\theta)$$`

???

&#8680; We need to figure out how to specify this joint distribution of the
covariates and parameters.

Usually:
* covariates of different types
* parameters include regression coefficients but also variance parameters.

&#8680; we can't specify this multivariate distribution directly because it does
not have a closed form.

--
<br>

**From probability theory:**&emsp; `$\bbox[#2E3440, 5pt]{p(A, B) = p(A\mid B)\;p(B)}$`

???

From probability theory we know that we can split the joint distribution of
two random variables.

In the example:
`$$p(\color{var(--nord15)}{\mathbf x_1}, \mathbf x_2, \mathbf x_3, \boldsymbol\theta) = 
p(\color{var(--nord15)}{\mathbf x_1} \mid \mathbf x_2, \mathbf x_3, \boldsymbol\theta)\;
p(\mathbf x_2 \mid \mathbf x_3, \boldsymbol\theta)\;
p(\mathbf x_3 \mid \boldsymbol\theta)\; p(\boldsymbol\theta)$$`

???

&#8680; apply this rule to our example<br>
&#8680; write the multivariate distribution as a sequence of univariate conditional distributions.

Bayesian &#8680; assume that parameters also are random variables &#8680;
distribution

Only univariate distributions &#8680; we can easily choose an appropriate model
type for each of covariate

---

## The Imputation Model

`\begin{eqnarray*}
p(\color{var(--nord15)}{\mathbf x_1} \mid \mathbf y, \ldots) & \propto & 
p(\mathbf y\mid \color{var(--nord15)}{\mathbf x_1}, \mathbf x_2, \mathbf x_3,
\boldsymbol\theta) \;
p(\color{var(--nord15)}{\mathbf x_1}\mid \mathbf x_2, \mathbf x_3, \boldsymbol\theta)\;
p(\mathbf x_2 \mid \mathbf x_3, \boldsymbol\theta)\;
p(\mathbf x_3\mid\boldsymbol\theta)\; p(\boldsymbol\theta)
\end{eqnarray*}`

???

Write out the full specification of the predictive distribution for
our incomplete covariate.

---
count: false

## The Imputation Model

???

&#8680; for this example the full conditional = joint distribution

.pull-left[

**In General:**<br>
Specify the joint distribution
`$$p(\mathbf y, \color{var(--nord15)}{\mathbf x_1}, \mathbf x_2, \mathbf x_3, \boldsymbol\theta)$$`
&#8680; derive the full conditional 
`$$p(\color{var(--nord15)}{\mathbf x_1} \mid \mathbf y, \ldots)$$`

]

???

&#8680; we can think about this approach in a slightly different way:
* start with the joint distribution of everything and 
* drive the full conditional distributions from the joint

Because we usually don't have a closed form for the joint distribution:<br>
specify the joint as the product of conditional distributions.
- - - - -
--

.pull-right[
<img src="graphics/Gibbs.png", height = 380, style = "position: absolute; bottom: 50px;">
]

---

## Sequence of Models

`\begin{eqnarray*}
p(A, B) & = & p(A \mid B)\; p(B)
\end{eqnarray*}`

???

Before: joint distribution of two random variables A and B can be expressed as
the conditional distribution of A given B and the distribution of B.

---
count: false

## Sequence of Models

`\begin{eqnarray*}
p(A, B) &=& p(A \mid B)\; p(B)\\
p(A, B) &=& p(B \mid A)\; p(A)
\end{eqnarray*}`

&#8680; any sequence possible, but some are more **convenient**

???

Of course it is also possible to do it the other way around.

When we represent the joint distribution using a sequence of conditional
distributions we can **choose the order of the sequence how ever we want.**

<br>

`$$p(\mathbf y, \mathbf x, \boldsymbol\theta) = 
\bbox[#2E3440, 5pt]{\underset{\text{analysis model}}{\underbrace{p(\mathbf y \mid \mathbf x,
\color{var(--nord14)}{\boldsymbol\theta_{y\mid x}})}}}\;\;\;
p(\mathbf x \mid \boldsymbol\theta_x)\;\;\;
p(\color{var(--nord14}{\boldsymbol\theta_{y\mid x}}, \boldsymbol\theta_x)
\qquad \color{var(--nord3)}{\text{with } \boldsymbol\theta = (\boldsymbol\theta_{y\mid x}, \boldsymbol\theta_x)}$$`

???

But specifying the sequence so that **one of the factors is the model for the
outcome** has some huge advantages.

* parameters of interest `$\color{var(--nord14)}{\boldsymbol\theta_{y\mid x}}$`
estimated directly
???
1) parameters that we are interested in are estimated directly &#8680;
imputation and the analysis jointly in the same estimation procedure.

* non-linear associations taken into account

???

2) non-linear associations specified in the analysis model are automatically
taken into account (analysis model is a factor in the full conditional 
distribution the imputed values are sampled from)

---

## Sequence of Models

Model for outcome `$\color{var(--nord14)}{\mathbf y}$` "first" in the sequence:

`$$p(\mathbf y, \mathbf x, \boldsymbol\theta) = 
\bbox[#2E3440, 5pt]{p(\color{var(--nord14)}{\mathbf y} \mid \mathbf x,
\boldsymbol\theta_{y\mid x})}\;\;\;
p(\mathbf x \mid \boldsymbol\theta_x)\;\;\;
p(\boldsymbol\theta_{y\mid x}, \boldsymbol\theta_x)
\qquad \color{var(--nord3)}{\text{with } \boldsymbol\theta = (\boldsymbol\theta_{y\mid x}, \boldsymbol\theta_x)}$$`

&#8680; outcome `$\color{var(--nord14)}{\mathbf y}$` not in any linear predictor<br>
&#8680; **no problem** to use **complex outcomes**

???
Additional advantage if we can specify the sequence so that the 
**analysis model is the first factor** in the sequence 
(outcome does not need to enter any of the linear predictors)

&#8680; allows us to specify the joint distribution (and consequently the full
conditional distributions of incomplete covariates) for settings with 
**complex outcomes**

<br>
**For covariates:** (in the multi-level setting)
`$$p(\color{var(--nord15)}{\mathbf x_1}, \mathbf x_2(t), \mathbf x_3) \mid \boldsymbol\theta) = 
p(\mathbf x_2(t) \mid \color{var(--nord15)}{\mathbf x_1}, \mathbf x_3, \boldsymbol\theta)\;\;\;
p(\color{var(--nord15)}{\mathbf x_1} \mid \mathbf x_3, \boldsymbol\theta)\;\;\;
p(\mathbf x_3 \mid \boldsymbol\theta)$$`

???
In multi-level settings:<br>
specify the sequence to our advantage: so that the models for variables on the
lowest level are specified first so that they have the variables of higher
levels in their linear predictor but not vice versa.

---

## Complex Models? Simple!

<div class = "container">
<div class = "box">
<div class = "box-row">
<div class = "box-cell" style = "background: var(--nord0);">joint<br>distribution</div>&nbsp;
<div class = "box-cell">$=$</div> 
<div class = "box-cell" style = "background: var(--nord0);">analysis<br>model</div>&nbsp;
<div class = "box-cell" style = "background: var(--nord0);">covariate<br>models</div>&nbsp;
<div class = "box-cell" style = "background: var(--nord0)">priors</div>
</div>
</div>
</div>

???

It always comes down to 
* analysis model
* covariate models
* priors

<br>
**(Multivariate) joint model for longitudinal and survival data:**

`$$p(\mathbf T, \mathbf D, \mathbf y, \mathbf x, \mathbf b, \boldsymbol\theta) =
\underset{\text{analysis model}}{\underbrace{
\underset{\substack{\text{survival}\\\text{model}}}{\underbrace{
p(\mathbf T, \mathbf D \mid \mathbf b, \mathbf x, \boldsymbol \theta)}}\;\;
\underset{\substack{\text{(multivariate)}\\\text{longitudinal}\\\text{model}}}{\underbrace{
p(\mathbf y \mid \mathbf b, \mathbf x, \boldsymbol\theta)}}
}}\;\;
\underset{\substack{\text{covariate}\\\text{models}}}{\underbrace{
p(\mathbf x \mid \boldsymbol\theta)}}\;\;
\underset{\text{priors}}{\underbrace{p(\boldsymbol\theta)}}$$`

<br>

.footnote[
[<i class = "fas fa-book"></i> Erler et al. (2020)](https://arxiv.org/abs/1907.10867)&emsp;
[<i class = "fab fa-r-project"></i> JointAI](https:nerler.github.io/JointAI/)
]

???

And the covariate model part is the only addition to what we would specify in
the complete data setting.

If there are many covariates specification of the covariate models can be a 
bit tedious.

---

## Complex Models? Simple!

<div class = "box-cell" style = "background: var(--nord0);">joint<br>distribution</div>&nbsp;
<div class = "box-cell">$=$</div>

<div class = "box-cell" style = "background: var(--nord0);">
<div style = "padding-bottom: 10px;"> <strong>User</strong></div>
<div class = "box-cell" style = "background: var(--nord1); border: 2px solid var(--nord8); ">analysis<br>model</div>
</div>

&nbsp;

<div class = "box-cell" style = "background: var(--nord0);">
<div style = "padding-bottom: 10px;"> <strong>JointAI</strong></div>
<div class = "box-row">
<div class = "box-cell" style = "background: var(--nord1);">covariate<br>models</div>&nbsp;
<div class = "box-cell" style = "background: var(--nord1)">priors</div>
</div></div>

</div></div></div>

???

But here JointAI can help.

---
class: center, middle, animated, fadeIn

# JointAI

---

## Model Types

.row[
.three-cols[
**Univariate Models:**

* `lm_imp()`
* `glm_imp()`
* `clm_imp()`
* `mlogit_imp()`
* `betareg_imp()`
* `lognorm_imp()`
]

.three-cols[
**Mixed Models**

* `lme_imp()`
* `glme_imp()`
* `clmm_imp()`
* `mlogitmm_imp()`
* `betamm_imp()`
* `lognormmm_imp()`
]

.three-cols[
**Survival Models**

* `coxph_imp()`
* `survreg_imp()`
* `JM_imp()`
]
]

---

## Specification

Specification **like standard complete data functions**:

```r
library("JointAI")
lm1 <- lm_imp(SBP ~ gender + age + alc * creat, data = NHANES, n.iter = 200)
```

<br>

Full documentation at
[https://nerler.github.io/JointAI/](https://nerler.github.io/JointAI/)

---
class: center, middle, animated, fadeIn

# Summary

---

## Imputation of Missing Covariates
To impute incomplete covariates `$\color{var(--nord15)}{\mathbf x_{mis}}$`
we need `$p(\color{var(--nord15)}{x_{mis}} \mid \text{everything else})$`

&#8680; Difficult to specify directly.

**Strategies:**<br>
* Gibbs idea: split into univariate problems<br>
  &#8680; **MICE/FCS**
--

* Approximate with something known & easy:<br>
  &#8680; **Joint Model MI**
--

* Smart specification of the (correct) joint distribution:<br>
  &#8680; **Bayesian Analysis**

---
name: comparison

## Comparison

.three-cols1[
**MICE / FCS**:
$$
`\begin{align}
p(\color{var(--nord15)}{\mathbf x_1} \mid \mathbf y, \color{var(--nord15)}{\mathbf x_2},
  \color{var(--nord15)}{\mathbf x_3}, \color{var(--nord7}{\mathbf X_{c}}, \theta_1)\\
p(\color{var(--nord15)}{\mathbf x_2} \mid \mathbf y, \color{var(--nord15)}{\mathbf x_1},
  \color{var(--nord15)}{\mathbf x_3}, \color{var(--nord7}{\mathbf X_{c}}, \theta_2)\\
p(\color{var(--nord15)}{\mathbf x_3} \mid \mathbf y, \color{var(--nord15)}{\mathbf x_1},
  \color{var(--nord15)}{\mathbf x_2}, \color{var(--nord7}{\mathbf X_{c}}, \theta_3)\\
\end{align}`
$$
]

.three-cols2[
**Joint Model MI**:
$$
`\begin{align}
p(\mathbf y \mid \color{var(--nord7}{\mathbf X_{c}}, \theta_y)\\
p(\color{var(--nord15)}{\mathbf x_1} \mid \color{var(--nord7}{\mathbf X_{c}}, \theta_1)\\
p(\color{var(--nord15)}{\mathbf x_2} \mid \color{var(--nord7}{\mathbf X_{c}}, \theta_2)\\
p(\color{var(--nord15)}{\mathbf x_3} \mid \color{var(--nord7}{\mathbf X_{c}}, \theta_3)\\
\end{align}`
$$
]

.three-cols3[
**Fully Bayesian**
$$
`\begin{align}
p(\mathbf y \mid \color{var(--nord15)}{\mathbf x_1}, \color{var(--nord15)}{\mathbf x_2},
  \color{var(--nord15)}{\mathbf x_3}, \color{var(--nord7}{\mathbf X_{c}},
  \color{var(--nord14}{\theta_{y\mid x}})\\
p(\color{var(--nord15)}{\mathbf x_1} \mid \color{var(--nord15)}{\mathbf x_2},
  \color{var(--nord15)}{x_3}, \color{var(--nord7}{\mathbf X_{c}}, \theta_1)\\
p(\color{var(--nord15)}{\mathbf x_2} \mid \color{var(--nord15)}{\mathbf x_3},
\color{var(--nord7}{\mathbf X_{c}}, \theta_2)\\
p(\color{var(--nord15)}{\mathbf x_3} \mid \color{var(--nord7}{\mathbf X_{c}}, \theta_3)\\
\end{align}`
$$
]

.three-cols1[
always 3 steps:
* imputation
* analysis
* pooling
]

.three-cols2[
always 3 steps:<br>
* imputation
* analysis
* pooling
]

.three-cols3[
usually 1 step:<br>
simultaneous analysis & imputation

MI possible
]

---

## Comparison

.three-cols[
* potentially incompatible models
* non-linear associations (theoretically) possible
]

.three-cols[
* assumes linear associations
* potentially incompatible with analysis model (extensions available)
]

.three-cols[
* non-linear associations possible
* assures compatibility
]

---

## Comparison

.three-cols[
* not suited for complex outcomes
* (some extensions available)
]

.three-cols[
* works for longitudinal
* (extensions for survival)
]

.three-cols[
* well suited for complex outcomes
]

---

## Comparison

.three-cols[
* usually fast
]

.three-cols[
* efficient sampling
* issues when many incomplete variables<br>(&#8680; large `$\mathbf V$` and `$\mathbf W$`)
]

.three-cols[
* more computationally intensive
]

---

## To Keep in Mind

Every model / method makes **assumptions**:

The **models need to fit** the data!!!

* analysis model(s)
* covariate models
* not straightforward to check

**Ignorable Missingness**
* impossible to check
* extensions to non-ignorable missingness possible

---
class: the-end, center, middle
layout: true
count: false

## Thank you for your attention!

---
count: false

<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.7/MathJax.js?config=TeX-MML-AM_CHTML">
</script>

// All page modals
var modals = document.querySelectorAll('.modal');

// Get the <span> element that closes the modal
var spans = document.getElementsByClassName("close");

// When the user clicks the button, open the modal
for (var i = 0; i < btn.length; i++) {
btn[i].onclick = function(e) {
e.preventDefault();
modal = document.querySelector(e.target.getAttribute("href"));
modal.style.display = "block";
}
}

// When the user clicks on <span> (x), close the modal
for (var i = 0; i < spans.length; i++) {
spans[i].onclick = function() {
for (var index in modals) {
if (typeof modals[index].style !== 'undefined') modals[index].style.display = "none";    
}
}
}

// When the user clicks anywhere outside of the modal, close it
window.onclick = function(event) {
if (event.target.classList.contains('modal')) {
for (var index in modals) {
if (typeof modals[index].style !== 'undefined') modals[index].style.display = "none";    
}
}
}
</script>