<p><em>This answer closely follows the logic of estimation of translog cost function presented in Section 4.7 of <a href="https://press.princeton.edu/books/hardcover/9780691010182/econometrics" rel="nofollow noreferrer">Fumio Hayashi's &quot;Econometrics&quot;</a>.</em></p>
<p>Define for convenience the CES aggregate price index <span class="math-container">$P:=(\sum_i \alpha_i^\sigma p_i^{1-\sigma})^\frac{1}{1-\sigma}$</span>. Then the Marshallian demand system in log form can be written as a system of linear equations</p>
<p><span class="math-container">$$ \begin{cases} 
\ln x_1 &amp;= \ln \alpha_1^\sigma -\sigma\ln p_1 - (1-\sigma)\ln P + \ln M\\
...\\
\ln x_n &amp;= \ln \alpha_n^\sigma -\sigma\ln p_n - (1-\sigma)\ln P + \ln M.
\end{cases}$$</span></p>
<p>Note that we are dealing with a <strong>multiple regression model</strong>. If we estimate these equations separately, data will necessarily make estimates of <span class="math-container">$\sigma$</span> vary depending on the equation we estimate, therefore, we have to impose the common coefficient restriction from the start.</p>
<p>Further, a regression model necessarily needs error terms:</p>
<p><span class="math-container">$$ \begin{cases} 
\ln x_1 &amp;= \ln \alpha_1^\sigma -\sigma\ln p_1 - (1-\sigma)\ln P + \ln M + \epsilon_1\\
...\\
\ln x_n &amp;= \ln \alpha_n^\sigma -\sigma\ln p_n - (1-\sigma)\ln P + \ln M + \epsilon_n.
\end{cases}$$</span></p>
<p>The error terms <span class="math-container">$\epsilon_i$</span> <strong>must have an economic interpretation</strong> for us to understand the statistical properties of chosen estimators. In other words, the uncertainty expressed in <span class="math-container">$\epsilon_i$</span> must be attributed either to <span class="math-container">$\alpha_i$</span> or to <span class="math-container">$\sigma$</span> (assuming <span class="math-container">$x_i,p_i,M$</span> are free of measurement/observation errors).</p>
<p>The most accessible and common interpretation goes as follows :</p>
<blockquote>
<p>In reality the weights <span class="math-container">$\tilde \alpha_i$</span> are not constant across individuals, but may fluctuate a little around the population average <span class="math-container">$\alpha_i$</span>, with <span class="math-container">$\epsilon_i\sim\mathcal{N}(0,\sigma_\epsilon^2)$</span> being a proxy for this noise:
<span class="math-container">$$ \tilde \alpha_i = \exp(\epsilon_i/\sigma) \alpha_i. $$</span></p>
</blockquote>
<p>Replacing <span class="math-container">$\alpha_i$</span> by <span class="math-container">$\tilde \alpha_i$</span> in the initial Marshallian demand system and taking logarithms gives exactly the stated regression equation system.</p>
<p><em>Remark: relating <span class="math-container">$\epsilon$</span> to the cross-sectional differences in <span class="math-container">$\sigma$</span> makes things more complicated, as in this case we get heteroskedasticity of the error term in the regression (after taking logarithms, the error term is necessarily multiplied by some function of <span class="math-container">$p_i$</span>).</em></p>
<p>Now, another problem is that without knowledge of <span class="math-container">$\alpha_i,\sigma$</span>, <span class="math-container">$P$</span> is not really observed. This problem is solved by subtracting equation <span class="math-container">$j$</span> from equation <span class="math-container">$j$</span>, as suggested in the answer by @Bertrand. We get a smaller system of equations (<span class="math-container">$n(n-1)/2$</span> against <span class="math-container">$n$</span> in the beginning):</p>
<p><span class="math-container">$$ \ln \frac{x_i}{x_j} = \sigma \ln \frac{\alpha_i}{\alpha_j} -\sigma\ln \frac{p_i}{p_j} + \epsilon_i - \epsilon_j,\quad \forall i\in\{1,..n\}, j\in\{1,..n\}\setminus \{i\}$$</span></p>
<p>Now this new multiple regression model has two new restrictions besides the <em>common coefficient</em> inherited from the original one:</p>
<ol>
<li><strong>correlated error terms</strong>, as <span class="math-container">$\mathbb{Cov}(\epsilon_i-\epsilon_j,\epsilon_j-\epsilon_k) = -\mathbb{V}(\epsilon_j)$</span>,</li>
<li><strong>consistency restiction on pair effects</strong>: <span class="math-container">$\ln{\alpha_i/\alpha_j}+\ln{\alpha_j/\alpha_k} = \ln{\alpha_i/\alpha_k},\quad \forall i\neq j\neq k$</span></li>
</ol>
<p>Correlation in the error terms is not a problem for consistency, but it certainly affects your estimate of the covariances and confidence intervals.</p>
<p>Pair effect consistency on the other hand allows us to reduce the number of coefficients to estimate at the model formulation stage.</p>
<p>In vector form our multivariate regression can be represented as</p>
<p><span class="math-container">$$ y_t = Z_t \delta + u_t $$</span></p>
<p>where (for <span class="math-container">$n=3$</span> without loss of generality)</p>
<p><span class="math-container">$$ y_t = \begin{bmatrix} \ln(\frac{x_{1t}}{x_{2t}}) \\ \ln(\frac{x_{1t}}{x_{3t}}) \\ \ln(\frac{x_{2t}}{x_{3t}})\end{bmatrix},\quad 
Z_t = \begin{bmatrix} 1 &amp; 0 &amp; &amp;\ln(\frac{p_{1t}}{p_{2t}}) \\ 0 &amp; 1 &amp;  &amp;  \ln(\frac{p_{1t}}{p_{3t}})\\ -1 &amp; 1 &amp;  &amp; \ln(\frac{p_{2t}}{p_{3t}}) \end{bmatrix}, \quad \delta = \begin{bmatrix} \sigma \ln \frac{\alpha_1}{\alpha_2} \\ \sigma \ln \frac{\alpha_1}{\alpha_3}\\ \sigma \end{bmatrix}, u_t =  \begin{bmatrix}\epsilon_1-\epsilon_2\\ \epsilon_1-\epsilon_3 \\ \epsilon_2-\epsilon_3\end{bmatrix}.$$</span></p>
<p>The above system yeilds easily to the random-effects estimation (treated extensively in Section 4.6 of Hayashi).</p>
<p>The estimate of CES elasticity of substitution <span class="math-container">$\hat \sigma$</span> is obtained directly, whereas <span class="math-container">$\hat\alpha_i$</span> can be pinned down with some simple normalizaton (e.g. <span class="math-container">$\sum_i \alpha_i = 1$</span>).</p>

CC BY-SA 4.0
2020-12-18T18:08:15.130Z
Sure thing, looks nice enough, let me know when you think you are done. Also are you sure the indexation in the matrix offdiagonal $p_jx_j$-terms  are correct? Another thing you may consider, is whether you wanna adress some endogeneity concerns. I mean if people wanna do this in practice is it then a good idea? (not part of the original question I know)
Jesper Hybel
CC BY-SA 4.0
2020-12-19T07:53:34.313Z
Thank you for the answer. I will have to take some time to digest it the panel regression part, since that's not what I normally do on a day-to-day basis.
Herr K.
CC BY-SA 4.0
2020-12-20T14:26:44.137Z
@JesperHybel I opened my old textbooks and revised the approach. The endogeneity is not a problem. Much care is needed to identify the $\alpha$'s and to achieve efficiency.
Konstantin
CC BY-SA 4.0
2020-12-20T15:15:54.413Z
@Konstantin looks nice. Now go and enjoy christmas :)
Jesper Hybel
CC BY-SA 4.0
2020-12-18T17:36:25.390Z
2020-12-23T06:57:26.273Z
2020-12-23T06:57:26.273Z
Konstantin
<p>In this comment I simply show that</p>
<blockquote>
<p>Under certain assumptions the problem is not an estimation problem, there is an exact solution for <span class="math-container">$\sigma$</span> and a solution for the structural errors <span class="math-container">$\alpha_i$</span> up to a normalisation.</p>
</blockquote>
<p>The demand system considered is described by the Marshall demand function</p>
<p><span class="math-container">$$x_k^\star(p,M) = \left(\frac{\alpha_j}{p_j}\right)^{\sigma}\frac{M}{\sum_j p_j^{1-\sigma}\alpha_j^\sigma},$$</span></p>
<p>which implies that the expenditure shares are given as</p>
<p><span class="math-container">$$s_k(p,I) := \frac{p_k x_k^\star(p,I)}{\sum_j p_j x_j^\star(p,I)} = \frac{p_k^{1-\sigma}\alpha_k^\sigma}{\sum_j p_j^{1-\sigma}\alpha_j^\sigma}.$$</span></p>
<p>I then define <span class="math-container">$K(p,\alpha) := \sum_j p_j^{1-\sigma}\alpha_j^\sigma$</span> where <span class="math-container">$p = (p_1,...,p_N)$</span> and <span class="math-container">$\alpha := (\alpha_1,...,\alpha_N)$</span> and log linearize expenditure shares to get</p>
<p><span class="math-container">$$(1) \ \ \ \log s_k = (1-\sigma) p_k + \sigma \log \alpha_k + K(p,\alpha)$$</span></p>
<blockquote>
<p><strong>Single cross section of aggregate data:</strong> Given data on <span class="math-container">$\{s_k,p_k\}_{k=1}^N$</span> this equation can be estimated using OLS under the assumption that <span class="math-container">$\mathbb E[p_k,\ln \alpha_k] =0$</span>.</p>
</blockquote>
<p>Consider now the situation where two time periods are observed <span class="math-container">$t=1,2$</span> and prices are assumed to show some variation while it is assumed <span class="math-container">$\alpha$</span> the comsumer demand parameters are time constant</p>
<p><span class="math-container">$$(1') \ \ \ \log s_{kt} = (1-\sigma) p_{kt} + \sigma \log \alpha_k + K_t(p_t,\alpha),$$</span></p>
<p>where I have added time index to <span class="math-container">$K$</span> because the price-vector is varying across time. It is clear that this is a standard two-way fixed effects regression with no good-time specific error term. Using methods of demeaning know from fixed effects estimation methodology one can find the exact solution for <span class="math-container">$(1-\sigma)$</span>.</p>
<p><strong>First</strong> take averages over goods to get</p>
<p><span class="math-container">$$\frac{1}{N}\sum_k \log s_{kt} = (1-\sigma) \frac{1}{N}\sum_kp_{kt} + \frac{1}{N}\sum_k \sigma \log \alpha_k + K_t(p_t,\alpha),$$</span></p>
<p>substract from equation (1) to get</p>
<p><span class="math-container">$$(2) \ \ \ \ ds_{kt} = (1-\sigma) dp_{kt} + \sigma d\alpha_k,$$</span>
where the <span class="math-container">$d$</span> in from of the variable simply signifies it has been demeaned in log-version so <span class="math-container">$ds_{kt} := \log s_{kt} - \frac{1}{N}\sum_k \log s_{kt}$</span>.</p>
<p><strong>Second</strong> take first differences over time to get</p>
<p><span class="math-container">$$(3) \ \ \ \ ds_{k2} - ds_{k1} = (1-\sigma) [dp_{k2} - dp_{k1}]$$</span></p>
<p>such that</p>
<p><span class="math-container">$$(4) \ \ \forall k: \ \ \ (1-\sigma) = \frac{ds_{k2} - ds_{k1}}{dp_{k2} - dp_{k1}}$$</span></p>
<p>having found <span class="math-container">$\sigma$</span> one can find <span class="math-container">$\alpha_i$</span>'s using equation (2). This means that one has to solve for <span class="math-container">$d\alpha_k = \alpha_k$</span> under normalisation that <span class="math-container">$\frac{1}{N}\sum_k \log \alpha_k = 0$</span>. So the average of the structural errors cannot be recovered.</p>
<blockquote>
<p>Under the assumption that <span class="math-container">$\alpha_k$</span>'s are time invariant one with data <span class="math-container">$\{p_{jt},s_{jt}\}_{j=1,...N, t=1,2}$</span> recover <span class="math-container">$\sigma$</span> using exactly using (4) and the structural errors <span class="math-container">$\alpha_k$</span> up to a single normalization using (2).</p>
</blockquote>
<p>To illustrate the calculations I provide a small simulation:</p>
<pre><code>#set.seed(1)
N &lt;- 50
M &lt;- 100000
sigma &lt;- 4

# Simulate structural errors assumed time-constant
phi &lt;- 2*runif(N)+2
phi &lt;- phi/((prod(phi))^(1/N))

# Simulate prices   
p1 &lt;- 2*runif(N)+2
p2 &lt;- 2*runif(N)+2

s1 &lt;- (p1/phi)^(1-sigma)
s1 &lt;- s1/sum(s1)
E1 &lt;- M*s1

s2 &lt;- (p2/phi)^(1-sigma)
s2 &lt;- s2/sum(s2)
E2 &lt;- M*s2

x1 &lt;- E1/p1
x2 &lt;- E2/p2

library(data.table)
dt &lt;- data.table(good=rep(1:N,2),time=rep(c(1,2),each=N),price=c(p1,p2),x=c(x1,x2))
dt[,E:=sum(price*x),by=time]
dt[,share:=price*x/E]

dt[,ds:=log(share)-mean(log(share)),by=time]
dt[,dp:=log(price)-mean(log(price)),by=time]
1-coef(lm(ds~dp+as.factor(good),data=dt))[2]
1-coef(lm(log(share)~log(price)+as.factor(good)+as.factor(time),data=dt))[2]
sigma
        
    
# Or from a single instead of all N 
temp &lt;- as.matrix(dt[good==2,.(ds,dp)])
1-(temp[2,1] - temp[1,1]) / (temp[2,2]-temp[1,2])
</code></pre>
<p>The calculations in this post are based on Stephen J. Redding and David E. Weinstein paper A UNIFIED APPROACH TO ESTIMATING DEMAND AND WELFARE.</p>

CC BY-SA 4.0
2020-12-20T18:16:14.763Z
There's a thing called overfitting. It will cause your third period observations be soo not in line with obtained parameter values.
Konstantin
CC BY-SA 4.0
2020-12-20T18:25:51.153Z
The point is simply to show that as the model is written up there is an exact solution if you have more than one time period. This has nothing to do with overfitting. A more fitting term would be overidentification.
Jesper Hybel
CC BY-SA 4.0
2020-12-20T18:53:27.687Z
The point of my comment is just to say, that if we have a dataset $\{x^1_t,..,x^n_t,y_t \}_{t=1}^T$ and we want to pick some values for $a,b^1,..b^n$ to fit a linear relation $y_t = a + \sum b^i x^i$ then we can always throw away any $T-n-1$ observations and make a system of linear equations of the rest. But that does not mean that the datapoints you threw away will satisfy the same equation. The values you get will not be robust. Call it what you like, "too many parameters, too little data".
Konstantin
CC BY-SA 4.0
2020-12-20T19:18:36.263Z
Im sorry i don't follow your example is there no $n$ index on your $y$? And then there is $t$ index in the data but none in the fitting model you fit?? I fail to see the analogue. There is no data thrown away any where. And there is no overfitting because you are not supossed to read it as estimation method. I thought that would be clear from the introduction of the post.
Jesper Hybel
CC BY-SA 4.0
2020-12-20T17:59:52.990Z
2020-12-20T17:59:52.990Z
Jesper Hybel
<p>These other answers seem to be citing some methods I haven't quite heard about yet however they all touch upon the idea developed by Czech economist named <a href="https://en.wikipedia.org/wiki/Jan_Kmenta" rel="nofollow noreferrer">Jan Kmenta</a> which has come to be known as the Kmenta approximation (an in depth explanation as well as detailed derivation can be found in the documentation of the R package,<a href="https://cran.r-project.org/web/packages/micEconCES/vignettes/CES.pdf" rel="nofollow noreferrer">MicEconCES</a>.</p>
<p>A general form of a CES utility function is:</p>
<p><span class="math-container">$$u(x_1,...,x_n)=\gamma\left(\sum_i \delta_i x_i^{-\rho}\right)^{-\frac{v}{\rho}}$$</span></p>
<p>To estimate this we have:
<span class="math-container">$$\ln y=\alpha_0+\sum_i\alpha_i \ln x_i+\frac{1}{2}\sum_i\sum_j \beta_{ij}\ln x_i \ln x_j$$</span>
where <span class="math-container">$y$</span> is our index of utility (note that we can recover an organic index of utility through estimating an expenditure system and using our principles of duality).</p>
<p>To obtain the parameters of the CES function we have the following forms:
<span class="math-container">$$\gamma=\exp(\alpha_0)$$</span>
<span class="math-container">$$v=\sum_i\alpha _i$$</span>
<span class="math-container">$$\delta_i=\frac{\alpha_i}{\sum_i \alpha_i}$$</span>
<span class="math-container">$$\rho=\frac{\beta_{ij}}{\alpha_i \alpha_j}\sum_i \alpha_i \ \   \forall i,j$$</span></p>
<p>I'd heavily recommend using the R package for any empirical work you may be doing. Hope this helps.</p>

CC BY-SA 4.0
2020-12-21T00:40:38.637Z
Interesting. Hope you get the time to perhaps expand a little on what kind of data is used. I'm not sure I follow what $y$ is and how it becomes measurable so to say.
Jesper Hybel
CC BY-SA 4.0
2020-12-21T21:33:01.647Z
Interesting, but not an answer to the original post, comment at best. Only useful when the output of the CES is observed, which is only the case for CES production function. It is stated explicitly in the post that the production function estimation is not the goal here.
Konstantin
CC BY-SA 4.0
2020-12-21T23:43:42.690Z
@Konstantin you can obtain y after estimating and expenditure system and inverting it to obtain our index of utility.
EconJohn
CC BY-SA 4.0
2020-12-22T09:05:13.227Z
@EconJohn It looks like estimating the expenditure system is the crucial part here. Moreover, estimating the expenditure system shall already give us the estimates of $\alpha_i,\sigma$. If I understand correctly, you suggest 0) estimating the expenditure system somehow and getting parameter estimates, 1) constructing a CES aggregate based on parameter estimates, and then 2) re-estimating the same parameters through Kmenta approximation based on this CES aggregate. What are the advantages of this approach?
Konstantin
CC BY-SA 4.0
2020-12-21T00:36:07.973Z
2023-11-24T05:06:56.173Z
2023-11-24T05:06:56.173Z
EconJohn
<p>It may be interesting to exploit the homothetic separability of the CES utility function in <span class="math-container">$x$</span>. It implies that
<span class="math-container">$$\frac{x_i}{x_j} = \left( \frac{\alpha_i}{\alpha_j}\frac{p_j}{p_i} \right)^\sigma $$</span>
and after <span class="math-container">$log$</span>-transformation:
<span class="math-container">$$\ln(x_i) - \ln(x_j) = \beta_{ij} + \sigma (\ln(p_j) - \ln(p_i)). $$</span></p>
<p>After adding a random term, this specification could be used to estimate the <span class="math-container">$\beta_{ij} \equiv \sigma (\ln(\alpha_i)-\ln(\alpha_j))$</span> and <span class="math-container">$\sigma$</span> parameters by OLS and in a second step identifying the <span class="math-container">$\alpha_i$</span> by minimum distance:</p>
<p><span class="math-container">$$ \widehat{\alpha} = \arg \min_\alpha \Big(\widehat{\beta}-\widehat{\sigma}(\ln(\alpha)-\ln(P\alpha)) \Big)'\Omega^{-1} \Big( \widehat{\beta}-\widehat{\sigma}(\ln(\alpha)-\ln(P\alpha)) \Big), $$</span>
where <span class="math-container">$P$</span> represents the adequate permutation matrix.</p>
<p>Some references estimating parameters of CES preference parameters include Diewert and Feenstra (2017) or Redding and Weinstein (2020), with a different approach, however, based on the unit expenditure function.</p>
<p>Most empirical contributions reject the validity of homothetic utility functions (like the CES). There are some propositions on how to built nonhomothetic CES production functions that could easily extended to utility functions, see Shimomura (1999) and the references therein. A simple extension of the above specification is:</p>
<p><span class="math-container">$$\ln(x_i) - \ln(x_j) = \beta_{ij} + \sigma (\ln(p_j) - \ln(p_i)) + \gamma_{ij} M/p + \varepsilon, $$</span>
where <span class="math-container">$p$</span> denotes a consumer and time specific aggregate price index (over all commodities). Notation <span class="math-container">$ij$</span> stands for commodities <span class="math-container">$i$</span> and <span class="math-container">$j$</span> and not for the observations (I skipped the <span class="math-container">$n,t$</span> subscripts for clarity).</p>
<p>This relationship is compatible with nonhomothetic preferences. The homothetic case is obtained for <span class="math-container">$\gamma_{ij}=0$</span> which can be tested.</p>
<p>Regarding the computer software... I fully switched to R few years ago, it would &quot;easily&quot; allow to code the minimum distance estimator, and the link between the utility and demand functions.</p>
<p>References:</p>
<p>Diewert, Erwin and Robert Feenstra (2017), “Estimating the Benefits and Costs of New and Disappearing Products,” <em>mimeo</em>, University of California at Davis.</p>
<p>Redding Stephen J and David E Weinstein (2020), &quot;Measuring Aggregate Price Indices with Taste Shocks: Theory and Evidence for CES Preferences,&quot; The Quarterly Journal of Economics, 135, 503-560.</p>
<p>Shimomura, K., 1999, &quot;A simple proof of the Sato proposition on non-homothetic CES functions,&quot; Economic Theory, 14, 501–503.</p>

CC BY-SA 4.0
2020-12-18T23:16:09.530Z
'+1' for reference the Redding paper, insane paper. Been reading it all day.
Jesper Hybel
CC BY-SA 4.0
2020-12-18T23:38:43.287Z
Just a follow up question: How are you planning to estimate $\beta_{ij}$ and $\sigma(\log p_i - \log p_j)$ in a first step OLS? I mean if you dummies for $(i,j)$-combinations they will capture both $\beta_{ij}$ and $\sigma(\log p_i - \log p_j)$ right?
Jesper Hybel
CC BY-SA 4.0
2020-12-19T07:42:02.677Z
@Bertrand: Thank you for the answer and the reference. They are very helpful!
Herr K.
CC BY-SA 4.0
2020-12-19T09:19:53.537Z
@Jesper: I just added clarification on the relationship between $\beta_{ij}$ and the $\alpha_i$ as well as few details on minimum distance. In the best case there are $NT$ observations available for estimating a given $\beta_{ij}$, so OLS is consistent.
Bertrand
CC BY-SA 4.0
2020-12-19T12:10:28.383Z
Arh, ok you are assuming panel-data but leaving out a time index in notation?
Jesper Hybel
CC BY-SA 4.0
2020-12-19T12:16:44.373Z
Also $N$ is undefined so to be honest short of guessing, I find it hard to know what you mean.
Jesper Hybel
CC BY-SA 4.0
2020-12-19T13:18:40.663Z
What about the theoretically implied constraint $\beta_{ij}+\beta_{jk}=\beta_{ik}$? Vanilla OLS will not handle it by magic.
Konstantin
CC BY-SA 4.0
2020-12-19T13:32:14.903Z
@Konstantin: by the adding-up property, vanilla OLS will handle the $\sum_{i,j} \beta_{ij}=0$ constraint as by magic.
Bertrand
CC BY-SA 4.0
2020-12-19T13:40:02.617Z
I've never heard of the "adding up property". Could you elaborate?
Konstantin
CC BY-SA 4.0
2020-12-19T13:50:58.640Z
If consumption and savings add to income, then in relative terms $c/y+s/y=1$. We first regress $c/y$ on a constant $a$ and any function $f(y)$. Second, we regress $s/y$  on a constant $b$ and a function $g(y)$, OLS ensures that $\widehat{a}+\widehat{b}=1$ and $\widehat{f}+\widehat{g}=0$. This point is treated in textbooks like Deaton and Muellbauer or Greene. This is why we can omitt one share when estimating parameters of a translog demand system.
Bertrand
CC BY-SA 4.0
2020-12-19T14:00:07.963Z
Please let us take this to this chat room: https://chat.stackexchange.com/rooms/117447/ces-utility-parameters-estimation
Konstantin
CC BY-SA 4.0
2020-12-18T23:09:13.650Z
2020-12-20T12:11:45.880Z
2020-12-20T12:11:45.880Z
Bertrand
<p>The CES utility function has the form 
<span class="math-container">\begin{equation}
u(x_1,\dots,x_n)=\left[\sum_{i=1}^n\alpha_ix_i^\rho\right]^{1/\rho},
\end{equation}</span>
where <span class="math-container">$\alpha_i$</span> is the consumption share parameter and <span class="math-container">$\sigma=\frac{1}{1-\rho}$</span> is the elasticity of substitution. </p>

<p>I'm interested in estimating the <span class="math-container">$\alpha_i$</span>'s and <span class="math-container">$\sigma$</span>. Since we do not directly observe utility levels, I think we can instead use data to estimate the Marshallian demand functions implied by the utility function: 
<span class="math-container">\begin{equation}
  x_i(p_1,\dots,p_n,M)=\frac{M(\alpha_i/p_i)^\sigma}{\sum_{j=1}^n\alpha_j^\sigma p_j^{1-\sigma}},\quad i=1,\dots,n.
\end{equation}</span>
Clearly, the function is highly nonlinear in the parameters of interest. So a nonlinear estimation procedure would be required. </p>

<p>Say we have data consumption <span class="math-container">$x_i$</span>, prices <span class="math-container">$p_i$</span> and income <span class="math-container">$M$</span>. <strong>How do we go about estimating the parameters <span class="math-container">$\alpha_i$</span>'s and <span class="math-container">$\sigma$</span>?</strong></p>

<p>I can find a lot of papers on estimating the CES <em>production</em> function, but I don't find them helpful since I don't observe the dependent variable (<span class="math-container">$u$</span>). Given that the CES utility is a common functional form used in the literature, I suspect its estimation must be somewhere in the literature. </p>

<p>I managed to find the <a href="https://www.stata.com/manuals13/rnl.pdf" rel="noreferrer">nonlinear least squares estimation in Stata</a>, but I don't know how susceptible such a procedure is to misspecification and how much confidence one should put on the statistics (e.g. t-stat or p-value) produced by the program.</p>

<p>Any pointer to the papers/textbook chapters on the topic, as well as statistical packages that implement the estimation procedure will be greatly appreciated. </p>

<econometrics><reference-request><applied-econometrics><ces-function>
Estimating CES utility (not production) function parameters
Herr K.
# Estimating CES utility (not production) function parameters

(Asked by: username_0 on 2019-07-03)

The CES utility function has the form
\begin{equation}
u(x\_1,\dots,x\_n)=\left[\sum\_{i=1}^n\alpha\_ix\_i^\rho\right]^{1/\rho},
\end{equation}
where $\alpha\_i$ is the consumption share parameter and $\sigma=\frac{1}{1-\rho}$ is the elasticity of substitution.

I'm interested in estimating the $\alpha\_i$'s and $\sigma$. Since we do not directly observe utility levels, I think we can instead use data to estimate the Marshallian demand functions implied by the utility function:
\begin{equation}
x\_i(p\_1,\dots,p\_n,M)=\frac{M(\alpha\_i/p\_i)^\sigma}{\sum\_{j=1}^n\alpha\_j^\sigma p\_j^{1-\sigma}},\quad i=1,\dots,n.
\end{equation}
Clearly, the function is highly nonlinear in the parameters of interest. So a nonlinear estimation procedure would be required.

Say we have data consumption $x\_i$, prices $p\_i$ and income $M$. **How do we go about estimating the parameters $\alpha\_i$'s and $\sigma$?**

I can find a lot of papers on estimating the CES *production* function, but I don't find them helpful since I don't observe the dependent variable ($u$). Given that the CES utility is a common functional form used in the literature, I suspect its estimation must be somewhere in the literature.

I managed to find the [nonlinear least squares estimation in Stata](https://www.stata.com/manuals13/rnl.pdf), but I don't know how susceptible such a procedure is to misspecification and how much confidence one should put on the statistics (e.g. t-stat or p-value) produced by the program.

Any pointer to the papers/textbook chapters on the topic, as well as statistical packages that implement the estimation procedure will be greatly appreciated.

**Tags:** econometrics, reference-request, applied-econometrics, ces-function


---

## 4 answers

### Answer by username_4 on 2020-12-18. Score: 10 | Accepted answer

It may be interesting to exploit the homothetic separability of the CES utility function in $x$. It implies that
$$\frac{x\_i}{x\_j} = \left( \frac{\alpha\_i}{\alpha\_j}\frac{p\_j}{p\_i} \right)^\sigma $$
and after $log$-transformation:
$$\ln(x\_i) - \ln(x\_j) = \beta\_{ij} + \sigma (\ln(p\_j) - \ln(p\_i)). $$

After adding a random term, this specification could be used to estimate the $\beta\_{ij} \equiv \sigma (\ln(\alpha\_i)-\ln(\alpha\_j))$ and $\sigma$ parameters by OLS and in a second step identifying the $\alpha\_i$ by minimum distance:

$$ \widehat{\alpha} = \arg \min\_\alpha \Big(\widehat{\beta}-\widehat{\sigma}(\ln(\alpha)-\ln(P\alpha)) \Big)'\Omega^{-1} \Big( \widehat{\beta}-\widehat{\sigma}(\ln(\alpha)-\ln(P\alpha)) \Big), $$
where $P$ represents the adequate permutation matrix.

Some references estimating parameters of CES preference parameters include Diewert and Feenstra (2017) or Redding and Weinstein (2020), with a different approach, however, based on the unit expenditure function.

Most empirical contributions reject the validity of homothetic utility functions (like the CES). There are some propositions on how to built nonhomothetic CES production functions that could easily extended to utility functions, see Shimomura (1999) and the references therein. A simple extension of the above specification is:

$$\ln(x\_i) - \ln(x\_j) = \beta\_{ij} + \sigma (\ln(p\_j) - \ln(p\_i)) + \gamma\_{ij} M/p + \varepsilon, $$
where $p$ denotes a consumer and time specific aggregate price index (over all commodities). Notation $ij$ stands for commodities $i$ and $j$ and not for the observations (I skipped the $n,t$ subscripts for clarity).

This relationship is compatible with nonhomothetic preferences. The homothetic case is obtained for $\gamma\_{ij}=0$ which can be tested.

Regarding the computer software... I fully switched to R few years ago, it would "easily" allow to code the minimum distance estimator, and the link between the utility and demand functions.

References:

Diewert, Erwin and Robert Feenstra (2017), “Estimating the Benefits and Costs of New and Disappearing Products,” *mimeo*, University of California at Davis.

Redding Stephen J and David E Weinstein (2020), "Measuring Aggregate Price Indices with Taste Shocks: Theory and Evidence for CES Preferences," The Quarterly Journal of Economics, 135, 503-560.

Shimomura, K., 1999, "A simple proof of the Sato proposition on non-homothetic CES functions," Economic Theory, 14, 501–503.

> **Comments:**
> **username_2**: '+1' for reference the Redding paper, insane paper. Been reading it all day.  
> **username_2**: Just a follow up question: How are you planning to estimate $\beta_{ij}$ and $\sigma(\log p_i - \log p_j)$ in a first step OLS? I mean if you dummies for $(i,j)$-combinations they will capture both $\beta_{ij}$ and $\sigma(\log p_i - \log p_j)$ right?  
> **username_0**: @username_4: Thank you for the answer and the reference. They are very helpful!  
> **username_4**: @Jesper: I just added clarification on the relationship between $\beta_{ij}$ and the $\alpha_i$ as well as few details on minimum distance. In the best case there are $NT$ observations available for estimating a given $\beta_{ij}$, so OLS is consistent.  
> **username_2**: Arh, ok you are assuming panel-data but leaving out a time index in notation?  
> **username_2**: Also $N$ is undefined so to be honest short of guessing, I find it hard to know what you mean.  
> **username_1**: What about the theoretically implied constraint $\beta_{ij}+\beta_{jk}=\beta_{ik}$? Vanilla OLS will not handle it by magic.  
> **username_4**: @username_1: by the adding-up property, vanilla OLS will handle the $\sum_{i,j} \beta_{ij}=0$ constraint as by magic.  
> **username_1**: I've never heard of the "adding up property". Could you elaborate?  
> **username_4**: If consumption and savings add to income, then in relative terms $c/y+s/y=1$. We first regress $c/y$ on a constant $a$ and any function $f(y)$. Second, we regress $s/y$  on a constant $b$ and a function $g(y)$, OLS ensures that $\widehat{a}+\widehat{b}=1$ and $\widehat{f}+\widehat{g}=0$. This point is treated in textbooks like Deaton and Muellbauer or Greene. This is why we can omitt one share when estimating parameters of a translog demand system.  
> **username_1**: Please let us take this to this chat room: https://chat.stackexchange.com/rooms/117447/ces-utility-parameters-estimation  

---

### Answer by username_1 on 2020-12-18. Score: 7

*This answer closely follows the logic of estimation of translog cost function presented in Section 4.7 of [Fumio Hayashi's "Econometrics"](https://press.princeton.edu/books/hardcover/9780691010182/econometrics).*

Define for convenience the CES aggregate price index $P:=(\sum\_i \alpha\_i^\sigma p\_i^{1-\sigma})^\frac{1}{1-\sigma}$. Then the Marshallian demand system in log form can be written as a system of linear equations

$$ \begin{cases}
\ln x\_1 &= \ln \alpha\_1^\sigma -\sigma\ln p\_1 - (1-\sigma)\ln P + \ln M\\
...\\
\ln x\_n &= \ln \alpha\_n^\sigma -\sigma\ln p\_n - (1-\sigma)\ln P + \ln M.
\end{cases}$$

Note that we are dealing with a **multiple regression model**. If we estimate these equations separately, data will necessarily make estimates of $\sigma$ vary depending on the equation we estimate, therefore, we have to impose the common coefficient restriction from the start.

Further, a regression model necessarily needs error terms:

$$ \begin{cases}
\ln x\_1 &= \ln \alpha\_1^\sigma -\sigma\ln p\_1 - (1-\sigma)\ln P + \ln M + \epsilon\_1\\
...\\
\ln x\_n &= \ln \alpha\_n^\sigma -\sigma\ln p\_n - (1-\sigma)\ln P + \ln M + \epsilon\_n.
\end{cases}$$

The error terms $\epsilon\_i$ **must have an economic interpretation** for us to understand the statistical properties of chosen estimators. In other words, the uncertainty expressed in $\epsilon\_i$ must be attributed either to $\alpha\_i$ or to $\sigma$ (assuming $x\_i,p\_i,M$ are free of measurement/observation errors).

The most accessible and common interpretation goes as follows :

> In reality the weights $\tilde \alpha\_i$ are not constant across individuals, but may fluctuate a little around the population average $\alpha\_i$, with $\epsilon\_i\sim\mathcal{N}(0,\sigma\_\epsilon^2)$ being a proxy for this noise:
> $$ \tilde \alpha\_i = \exp(\epsilon\_i/\sigma) \alpha\_i. $$

Replacing $\alpha\_i$ by $\tilde \alpha\_i$ in the initial Marshallian demand system and taking logarithms gives exactly the stated regression equation system.

*Remark: relating $\epsilon$ to the cross-sectional differences in $\sigma$ makes things more complicated, as in this case we get heteroskedasticity of the error term in the regression (after taking logarithms, the error term is necessarily multiplied by some function of $p\_i$).*

Now, another problem is that without knowledge of $\alpha\_i,\sigma$, $P$ is not really observed. This problem is solved by subtracting equation $j$ from equation $j$, as suggested in the answer by @username_4. We get a smaller system of equations ($n(n-1)/2$ against $n$ in the beginning):

$$ \ln \frac{x\_i}{x\_j} = \sigma \ln \frac{\alpha\_i}{\alpha\_j} -\sigma\ln \frac{p\_i}{p\_j} + \epsilon\_i - \epsilon\_j,\quad \forall i\in\{1,..n\}, j\in\{1,..n\}\setminus \{i\}$$

Now this new multiple regression model has two new restrictions besides the *common coefficient* inherited from the original one:

1. **correlated error terms**, as $\mathbb{Cov}(\epsilon\_i-\epsilon\_j,\epsilon\_j-\epsilon\_k) = -\mathbb{V}(\epsilon\_j)$,
2. **consistency restiction on pair effects**: $\ln{\alpha\_i/\alpha\_j}+\ln{\alpha\_j/\alpha\_k} = \ln{\alpha\_i/\alpha\_k},\quad \forall i\neq j\neq k$

Correlation in the error terms is not a problem for consistency, but it certainly affects your estimate of the covariances and confidence intervals.

Pair effect consistency on the other hand allows us to reduce the number of coefficients to estimate at the model formulation stage.

In vector form our multivariate regression can be represented as

$$ y\_t = Z\_t \delta + u\_t $$

where (for $n=3$ without loss of generality)

$$ y\_t = \begin{bmatrix} \ln(\frac{x\_{1t}}{x\_{2t}}) \\ \ln(\frac{x\_{1t}}{x\_{3t}}) \\ \ln(\frac{x\_{2t}}{x\_{3t}})\end{bmatrix},\quad
Z\_t = \begin{bmatrix} 1 & 0 & &\ln(\frac{p\_{1t}}{p\_{2t}}) \\ 0 & 1 & & \ln(\frac{p\_{1t}}{p\_{3t}})\\ -1 & 1 & & \ln(\frac{p\_{2t}}{p\_{3t}}) \end{bmatrix}, \quad \delta = \begin{bmatrix} \sigma \ln \frac{\alpha\_1}{\alpha\_2} \\ \sigma \ln \frac{\alpha\_1}{\alpha\_3}\\ \sigma \end{bmatrix}, u\_t = \begin{bmatrix}\epsilon\_1-\epsilon\_2\\ \epsilon\_1-\epsilon\_3 \\ \epsilon\_2-\epsilon\_3\end{bmatrix}.$$

The above system yeilds easily to the random-effects estimation (treated extensively in Section 4.6 of Hayashi).

The estimate of CES elasticity of substitution $\hat \sigma$ is obtained directly, whereas $\hat\alpha\_i$ can be pinned down with some simple normalizaton (e.g. $\sum\_i \alpha\_i = 1$).

> **Comments:**
> **username_2**: Sure thing, looks nice enough, let me know when you think you are done. Also are you sure the indexation in the matrix offdiagonal $p_jx_j$-terms  are correct? Another thing you may consider, is whether you wanna adress some endogeneity concerns. I mean if people wanna do this in practice is it then a good idea? (not part of the original question I know)  
> **username_0**: Thank you for the answer. I will have to take some time to digest it the panel regression part, since that's not what I normally do on a day-to-day basis.  
> **username_1**: @JesperHybel I opened my old textbooks and revised the approach. The endogeneity is not a problem. Much care is needed to identify the $\alpha$'s and to achieve efficiency.  
> **username_2**: @username_1 looks nice. Now go and enjoy christmas :)  

---

### Answer by username_2 on 2020-12-20. Score: 6

In this comment I simply show that

Under certain assumptions the problem is not an estimation problem, there is an exact solution for $\sigma$ and a solution for the structural errors $\alpha_i$ up to a normalisation.

The demand system considered is described by the Marshall demand function
$$x_k^\star(p,M) = \left(\frac{\alpha_j}{p_j}\right)^{\sigma}\frac{M}{\sum_j p_j^{1-\sigma}\alpha_j^\sigma},$$
which implies that the expenditure shares are given as
$$s_k(p,I) := \frac{p_k x_k^\star(p,I)}{\sum_j p_j x_j^\star(p,I)} = \frac{p_k^{1-\sigma}\alpha_k^\sigma}{\sum_j p_j^{1-\sigma}\alpha_j^\sigma}.$$
I then define $K(p,\alpha) := \sum_j p_j^{1-\sigma}\alpha_j^\sigma$ where $p = (p_1,...,p_N)$ and $\alpha := (\alpha_1,...,\alpha_N)$ and log linearize expenditure shares to get
$$(1) \ \ \ \log s_k = (1-\sigma) p_k + \sigma \log \alpha_k + K(p,\alpha)$$

Single cross section of aggregate data: Given data on $\{s_k,p_k\}_{k=1}^N$ this equation can be estimated using OLS under the assumption that $\mathbb E[p_k,\ln \alpha_k] =0$.

Consider now the situation where two time periods are observed $t=1,2$ and prices are assumed to show some variation while it is assumed $\alpha$ the comsumer demand parameters are time constant
$$(1') \ \ \ \log s_{kt} = (1-\sigma) p_{kt} + \sigma \log \alpha_k + K_t(p_t,\alpha),$$
where I have added time index to $K$ because the price-vector is varying across time. It is clear that this is a standard two-way fixed effects regression with no good-time specific error term. Using methods of demeaning know from fixed effects estimation methodology one can find the exact solution for $(1-\sigma)$.
First take averages over goods to get
$$\frac{1}{N}\sum_k \log s_{kt} = (1-\sigma) \frac{1}{N}\sum_kp_{kt} + \frac{1}{N}\sum_k \sigma \log \alpha_k + K_t(p_t,\alpha),$$
substract from equation (1) to get
$$(2) \ \ \ \ ds_{kt} = (1-\sigma) dp_{kt} + \sigma d\alpha_k,$$
where the $d$ in from of the variable simply signifies it has been demeaned in log-version so $ds_{kt} := \log s_{kt} - \frac{1}{N}\sum_k \log s_{kt}$.
Second take first differences over time to get
$$(3) \ \ \ \ ds_{k2} - ds_{k1} = (1-\sigma) [dp_{k2} - dp_{k1}]$$
such that
$$(4) \ \ \forall k: \ \ \ (1-\sigma) = \frac{ds_{k2} - ds_{k1}}{dp_{k2} - dp_{k1}}$$
having found $\sigma$ one can find $\alpha_i$'s using equation (2). This means that one has to solve for $d\alpha_k = \alpha_k$ under normalisation that $\frac{1}{N}\sum_k \log \alpha_k = 0$. So the average of the structural errors cannot be recovered.

Under the assumption that $\alpha_k$'s are time invariant one with data $\{p_{jt},s_{jt}\}_{j=1,...N, t=1,2}$ recover $\sigma$ using exactly using (4) and the structural errors $\alpha_k$ up to a single normalization using (2).

To illustrate the calculations I provide a small simulation:
#set.seed(1)
N <- 50
M <- 100000
sigma <- 4

# Simulate structural errors assumed time-constant
phi <- 2*runif(N)+2
phi <- phi/((prod(phi))^(1/N))

# Simulate prices   
p1 <- 2*runif(N)+2
p2 <- 2*runif(N)+2

s1 <- (p1/phi)^(1-sigma)
s1 <- s1/sum(s1)
E1 <- M*s1

s2 <- (p2/phi)^(1-sigma)
s2 <- s2/sum(s2)
E2 <- M*s2

x1 <- E1/p1
x2 <- E2/p2

library(data.table)
dt <- data.table(good=rep(1:N,2),time=rep(c(1,2),each=N),price=c(p1,p2),x=c(x1,x2))
dt[,E:=sum(price*x),by=time]
dt[,share:=price*x/E]

dt[,ds:=log(share)-mean(log(share)),by=time]
dt[,dp:=log(price)-mean(log(price)),by=time]
1-coef(lm(ds~dp+as.factor(good),data=dt))[2]
1-coef(lm(log(share)~log(price)+as.factor(good)+as.factor(time),data=dt))[2]
sigma
        
    
# Or from a single instead of all N 
temp <- as.matrix(dt[good==2,.(ds,dp)])
1-(temp[2,1] - temp[1,1]) / (temp[2,2]-temp[1,2])

The calculations in this post are based on Stephen J. Redding and David E. Weinstein paper A UNIFIED APPROACH TO ESTIMATING DEMAND AND WELFARE.

> **Comments:**
> **username_1**: There's a thing called overfitting. It will cause your third period observations be soo not in line with obtained parameter values.  
> **username_2**: The point is simply to show that as the model is written up there is an exact solution if you have more than one time period. This has nothing to do with overfitting. A more fitting term would be overidentification.  
> **username_1**: The point of my comment is just to say, that if we have a dataset $\{x^1_t,..,x^n_t,y_t \}_{t=1}^T$ and we want to pick some values for $a,b^1,..b^n$ to fit a linear relation $y_t = a + \sum b^i x^i$ then we can always throw away any $T-n-1$ observations and make a system of linear equations of the rest. But that does not mean that the datapoints you threw away will satisfy the same equation. The values you get will not be robust. Call it what you like, "too many parameters, too little data".  
> **username_2**: Im sorry i don't follow your example is there no $n$ index on your $y$? And then there is $t$ index in the data but none in the fitting model you fit?? I fail to see the analogue. There is no data thrown away any where. And there is no overfitting because you are not supossed to read it as estimation method. I thought that would be clear from the introduction of the post.  

---

### Answer by username_3 on 2020-12-21. Score: 5

These other answers seem to be citing some methods I haven't quite heard about yet however they all touch upon the idea developed by Czech economist named [Jan Kmenta](https://en.wikipedia.org/wiki/Jan_Kmenta) which has come to be known as the Kmenta approximation (an in depth explanation as well as detailed derivation can be found in the documentation of the R package,[MicEconCES](https://cran.r-project.org/web/packages/micEconCES/vignettes/CES.pdf).

A general form of a CES utility function is:

$$u(x\_1,...,x\_n)=\gamma\left(\sum\_i \delta\_i x\_i^{-\rho}\right)^{-\frac{v}{\rho}}$$

To estimate this we have:
$$\ln y=\alpha\_0+\sum\_i\alpha\_i \ln x\_i+\frac{1}{2}\sum\_i\sum\_j \beta\_{ij}\ln x\_i \ln x\_j$$
where $y$ is our index of utility (note that we can recover an organic index of utility through estimating an expenditure system and using our principles of duality).

To obtain the parameters of the CES function we have the following forms:
$$\gamma=\exp(\alpha\_0)$$
$$v=\sum\_i\alpha \_i$$
$$\delta\_i=\frac{\alpha\_i}{\sum\_i \alpha\_i}$$
$$\rho=\frac{\beta\_{ij}}{\alpha\_i \alpha\_j}\sum\_i \alpha\_i \ \ \forall i,j$$

I'd heavily recommend using the R package for any empirical work you may be doing. Hope this helps.

> **Comments:**
> **username_2**: Interesting. Hope you get the time to perhaps expand a little on what kind of data is used. I'm not sure I follow what $y$ is and how it becomes measurable so to say.  
> **username_1**: Interesting, but not an answer to the original post, comment at best. Only useful when the output of the CES is observed, which is only the case for CES production function. It is stated explicitly in the post that the production function estimation is not the goal here.  
> **username_3**: @username_1 you can obtain y after estimating and expenditure system and inverting it to obtain our index of utility.  
> **username_1**: @username_3 It looks like estimating the expenditure system is the crucial part here. Moreover, estimating the expenditure system shall already give us the estimates of $\alpha_i,\sigma$. If I understand correctly, you suggest 0) estimating the expenditure system somehow and getting parameter estimates, 1) constructing a CES aggregate based on parameter estimates, and then 2) re-estimating the same parameters through Kmenta approximation based on this CES aggregate. What are the advantages of this approach?  

---

