<p>My suggestion to you, and this is a very general technique when you are unsure what method to use, is to let <strong>cross-validation</strong> tell you what method works best.</p>

<p>I imagine you have a couple of options:</p>

<ul>
<li>Not include any row with missing observations</li>
<li>Assign some arbitrary weight (in your case those with compound with the assumed $\Omega$ matrix, and if you are using some non-linear regression to estimate $\hat{\Omega}$ then good luck)</li>
<li>Use mean-substitution or some other placeholder strategy</li>
</ul>

<p>But the fundamental problem is that you don't know what factors made some data unavailable, that is you don't know $\text{Probability}(\text{Missing})$ and you suspect that isn't white noise.</p>

<p>The best way then is to let data drive your results. Run all these methods on 80% of the data, check the prediction error on the remaining 20% and shuffle these groups around, take the mean prediction error and select the method that provided the lowest prediction error.</p>

CC BY-SA 3.0
2014-12-10T05:07:24.283Z
I'm not sure cross-validation would help here. As I see it the issue is not getting the best fit by any empirical definition of best (what cross-validation is good for). Instead the issue for me is what one **should** fit to capture an economic concept (factor prices), which cross-validation will not help with.
jayk
CC BY-SA 3.0
2014-12-10T15:08:19.437Z
Cross-validation isn't  there to get the best fit, it is there to choose between models/functional families. Here the regression model is fixed but the model to fix missing data isn't, and I think cross-validation is a very valid way of deciding among them.
CarrKnight
CC BY-SA 3.0
2014-12-10T15:22:25.657Z
But the criterion by which you judge models *is* their fit. Cross-validation is about predictive regressions.  If you were trying to only predict stock prices, you would be right about this. But you're testing an economic model, not doing prediction. I would be willing  to bet that a good testing fit would come from (say) basically excluding stocks with bankruptcies. But this wouldn't necessarily correspond to a good test of the economic model, as bankruptcies may be something we have to take into account when buying stocks. You need a criterion cross-validation does not present to judge that.
jayk
CC BY-SA 3.0
2014-12-10T04:54:42.813Z
2014-12-10T04:54:42.813Z
CarrKnight
<p>Easiest fix: if you're worried about it you should <strong>value weight your results</strong>. This is suggest by, for instance, <a href="http://onlinelibrary.wiley.com/doi/10.1111/j.1540-6261.1995.tb05171.x/abstract" rel="noreferrer">Kothari, Shanken and Sloan (1995)</a>. Firms that are delisted tend to have extremely small market cap, so value weighting gives them very little impact on summary statistics. Delisted returns should also be used, although I'm not sure how much impact they'll have. I've seen the delisted return stuck into the month after a stock ceases to be traded.</p>

<p>In finding $\beta$'s, I tend to see the regression used only on those dates for which the stock return is observed. The correction really comes in value weighting summary statistics afterwards. Whether all this is "correct" or just the practice I've seen is not something I'm sure about.</p>

<p><strong>Edit:</strong> here's a <a href="http://www.jstor.org/stable/2329566" rel="noreferrer">different perspective</a>.</p>

CC BY-SA 3.0
2014-12-13T08:01:43.000Z
Thanks for the excellent links. I'm looking forward to reading them more in depth. Just a question about the comment about using the delisted returns. Are you talking about doing this in the context of a Fama-Macbeth type regression approach? I'm not sure how that fits into the maximum-likelihood time-series approach describe above. I mean, it's not like it a big deal, I'm just curious if that's what you mean.
jmbejara
CC BY-SA 3.0
2014-12-13T15:06:26.243Z
Not particularly. I think the same value-weighting procedure could be used for whatever procedure you want. For MLE it'll just slightly change the calculation of your likelihood. These look like GMM though? In any case value-weighting could still be done.
jayk
CC BY-SA 3.0
2014-12-10T02:53:16.263Z
2014-12-10T03:06:04.410Z
2014-12-10T03:06:04.410Z
jayk
<h1>Question</h1>
<p>How should I deal with missing data when trying to test the CAPM? Specifically, there are some stocks that are newly listed and/or delisted at any time. I don't want to exclude assets for which I don't have complete data because this would create a kind of survivor bias.  I know that CRSP provides delisting returns that should, but how do I manage the missing data in practice? For example, in the unconstrained model, the procedure looks like this:</p>
<p><img src="https://i.sstatic.net/6GpYT.png" alt="enter image description here" /></p>
<p>(More details about the procedure are given below.) Now, if I wanted to take a bunch of random stocks at some point in time and look at them over some time period, what should I do with the values (<span class="math-container">$Z_it$</span>) for these stocks that aren't listed at time <span class="math-container">$t$</span>. Should I use the de-listing returns where appropriate and then fill in zeros everywhere else? But this would do weird things to the beta of the stock. Should I try to constrain the beta (the factor loadings) of the stocks to be zero in all the places where the stock is unlisted? This would require me to change the model (requiring a model that somehow allows for time varying factor loadings). How do people usually handle this problem? Is there an easy way (even if it is slightly more incorrect)?</p>
<h1>Some Detail about the Estimation Procedure</h1>
<p>For concreteness, suppose I wanted to test the CAPM using the time series regression framework outlined in chapter 6 of Campbell, Lo, and MacKinley (The Econometrics of Financial Markets). Some of the assumptions are listed in this image:
<img src="https://i.sstatic.net/37Btm.png" alt="enter image description here" /></p>

CC BY-SA 3.0
2014-12-11T14:54:34.957Z
The problem is that the factors (right hand side Zkt) are sometimes not observed? Or is it that the returns (left hand side elements of Zt) can be observed?
BKay
CC BY-SA 3.0
2014-12-11T16:28:35.703Z
The problem is $Z_t$. Some stocks get newly listed, some delisted, etc. If I only include stocks for which I have data for the full time period (for which they exist for the full period), then I introduce survivor bias.
jmbejara
<econometrics><finance><financial-economics><applied-econometrics>
Dealing with Missing Data when Testing the CAPM
jmbejara
# Dealing with Missing Data when Testing the CAPM

(Asked by: username_0 on 2014-12-05)

# Question

How should I deal with missing data when trying to test the CAPM? Specifically, there are some stocks that are newly listed and/or delisted at any time. I don't want to exclude assets for which I don't have complete data because this would create a kind of survivor bias. I know that CRSP provides delisting returns that should, but how do I manage the missing data in practice? For example, in the unconstrained model, the procedure looks like this:

![enter image description here](https://i.sstatic.net/6GpYT.png)

(More details about the procedure are given below.) Now, if I wanted to take a bunch of random stocks at some point in time and look at them over some time period, what should I do with the values ($Z\_it$) for these stocks that aren't listed at time $t$. Should I use the de-listing returns where appropriate and then fill in zeros everywhere else? But this would do weird things to the beta of the stock. Should I try to constrain the beta (the factor loadings) of the stocks to be zero in all the places where the stock is unlisted? This would require me to change the model (requiring a model that somehow allows for time varying factor loadings). How do people usually handle this problem? Is there an easy way (even if it is slightly more incorrect)?

# Some Detail about the Estimation Procedure

For concreteness, suppose I wanted to test the CAPM using the time series regression framework outlined in chapter 6 of Campbell, Lo, and MacKinley (The Econometrics of Financial Markets). Some of the assumptions are listed in this image:
![enter image description here](https://i.sstatic.net/37Btm.png)

**Tags:** econometrics, finance, financial-economics, applied-econometrics

> **Comments:**
> **username_1**: The problem is that the factors (right hand side Zkt) are sometimes not observed? Or is it that the returns (left hand side elements of Zt) can be observed?  
> **username_0**: The problem is $Z_t$. Some stocks get newly listed, some delisted, etc. If I only include stocks for which I have data for the full time period (for which they exist for the full period), then I introduce survivor bias.  

---

## 2 answers

### Answer by username_3 on 2014-12-10. Score: 5 | Accepted answer

Easiest fix: if you're worried about it you should **value weight your results**. This is suggest by, for instance, [Kothari, Shanken and Sloan (1995)](http://onlinelibrary.wiley.com/doi/10.1111/j.1540-6261.1995.tb05171.x/abstract). Firms that are delisted tend to have extremely small market cap, so value weighting gives them very little impact on summary statistics. Delisted returns should also be used, although I'm not sure how much impact they'll have. I've seen the delisted return stuck into the month after a stock ceases to be traded.

In finding $\beta$'s, I tend to see the regression used only on those dates for which the stock return is observed. The correction really comes in value weighting summary statistics afterwards. Whether all this is "correct" or just the practice I've seen is not something I'm sure about.

**Edit:** here's a [different perspective](http://www.jstor.org/stable/2329566).

> **Comments:**
> **username_0**: Thanks for the excellent links. I'm looking forward to reading them more in depth. Just a question about the comment about using the delisted returns. Are you talking about doing this in the context of a Fama-Macbeth type regression approach? I'm not sure how that fits into the maximum-likelihood time-series approach describe above. I mean, it's not like it a big deal, I'm just curious if that's what you mean.  
> **username_3**: Not particularly. I think the same value-weighting procedure could be used for whatever procedure you want. For MLE it'll just slightly change the calculation of your likelihood. These look like GMM though? In any case value-weighting could still be done.  

---

### Answer by username_2 on 2014-12-10. Score: 1

My suggestion to you, and this is a very general technique when you are unsure what method to use, is to let **cross-validation** tell you what method works best.

I imagine you have a couple of options:

* Not include any row with missing observations
* Assign some arbitrary weight (in your case those with compound with the assumed $\Omega$ matrix, and if you are using some non-linear regression to estimate $\hat{\Omega}$ then good luck)
* Use mean-substitution or some other placeholder strategy

But the fundamental problem is that you don't know what factors made some data unavailable, that is you don't know $\text{Probability}(\text{Missing})$ and you suspect that isn't white noise.

The best way then is to let data drive your results. Run all these methods on 80% of the data, check the prediction error on the remaining 20% and shuffle these groups around, take the mean prediction error and select the method that provided the lowest prediction error.

> **Comments:**
> **username_3**: I'm not sure cross-validation would help here. As I see it the issue is not getting the best fit by any empirical definition of best (what cross-validation is good for). Instead the issue for me is what one **should** fit to capture an economic concept (factor prices), which cross-validation will not help with.  
> **username_2**: Cross-validation isn't  there to get the best fit, it is there to choose between models/functional families. Here the regression model is fixed but the model to fix missing data isn't, and I think cross-validation is a very valid way of deciding among them.  
> **username_3**: But the criterion by which you judge models *is* their fit. Cross-validation is about predictive regressions.  If you were trying to only predict stock prices, you would be right about this. But you're testing an economic model, not doing prediction. I would be willing  to bet that a good testing fit would come from (say) basically excluding stocks with bankruptcies. But this wouldn't necessarily correspond to a good test of the economic model, as bankruptcies may be something we have to take into account when buying stocks. You need a criterion cross-validation does not present to judge that.  

---

