<p>There is no such problem.  If it's hard to sample, it's hard to integrate.</p>

<p>Here is a sketch of the reason why.  Represent every solution $x$ by a $n$-bit string $x_1,\dots,x_n$.  If you can integrate over the set of all solutions, here is an algorithm to sample from all solutions:</p>

<ol>
<li><p>Count the number of solutions; call it $N_0$.  (As you say, this can be done efficiently, as it is just an integral.)  Count the number of solutions $x$ such that $x_1=1$; call it $N'_0$.  (This is an integral of the function $f(x)$ over all solutions, where $f(x_1,\dots,x_n) = x_1$, so it can be computed efficiently.)  With probability $N'_0/N_0$, set $y_1 = 1$, else set $y_1 = 0$.</p></li>
<li><p>Count the number of solutions $x$ such that $x_1=y_1$; call it $N_1$.  (This is another integral.)  Count the number of solutions $x$ such that $x_1=y_1$ and $x_2=1$; call it $N'_1$.  With probability $N'_1/N_1$, set $y_2 = 1$, else set $y_2 = 0$.</p></li>
<li><p>Count the number $N_2$ of solutions $x$ such that $x_1=y_1$ and $x_2=y_2$, and the number $N'_2$ of solutions $x$ such that $x_1=y_1$ and $x_2=y_2$ and $x_3=1$.  With probability $N'_2/N_2$, set $y_3=1$, else set $y_3=0$.</p></li>
<li><p>And so on.  When this procedure finishes, output $y_1,y_2,\dots,y_n$.  This will be a valid solution, sampled uniformly at random from the set of all solutions.</p></li>
</ol>

<p>It's easy to see that if you have a way to compute the exact integral, you'll obtain a way to sample exactly uniformly at random.  I expect it will follow that if you can approximate the integral, you can sample approximately uniformly at random; I'll let you compute the parameters and relation between the two approximations.</p>

<hr>

<p>You can also implement this algorithm using expectations.  Let $X$ be a r.v. that is uniformly distributed on the set of all solutions.  Set $y_1=1$ with probability $\mathbb{E}[X_1]$; set $y_2=1$ with probability $\mathbb{E}[\mathbf{1}_{X_1 = y_1} \cdot X_2] / \mathbb{E}[\mathbf{1}_{X_1 = y_1}]$; and so on.  Here I use $\mathbf{1}_{e(X)}$ to represent a function that is 1 if $e(X)$ holds, and 0 otherwise.  Why does this work?  Well, $\Pr[X_1=1] = \mathbb{E}[X_1]$, and</p>

<p>$$\Pr[X_2=1|X_1=y_1] = {\Pr[X_1 = y_1 \land X_2=1] \over \Pr[X_1=y_1]}
= {\mathbb{E}[\mathbf{1}_{X_1 = y_1} \cdot X_2] \over \mathbb{E}[\mathbf{1}_{X_1 = y_1}]}.$$</p>

CC BY-SA 4.0
2018-09-17T03:09:22.830Z
Got it, thanks! The thing I was missing is that you don't need self reducibility to reduce sampling to counting if you can integrate over all functions (your construction is essentially the reduction JVV86 for self reducible problems). What about computing expectations against the uniform distribution - that is, 1) and 2')? I don't think the same idea will work for that.
Elle Najt
CC BY-SA 4.0
2018-09-17T03:40:08.323Z
@Lorenzo, see edited answer, at the bottom.
D.W.
CC BY-SA 4.0
2018-09-17T03:03:43.767Z
2018-09-17T05:32:42.843Z
2018-09-17T05:32:42.843Z
D.W.
<p>I'm curious if there is a problem (e.g. something like perfect matchings on a given graph, number of solutions to a boolean equation, etc. for precise frameowork see <a href="https://pdfs.semanticscholar.org/50a6/261904e0c56a4430e5cd98c6e7f71f0ed1b4.pdf" rel="noreferrer">JVV86</a>) such that:</p>

<p>1) It is hard to sample approximately uniformly at random from the set of all solutions</p>

<p>2) It is easy to approximately integrate any (polynomially computable) function over the solutions.</p>

<p>I would also be happy with replacing 2) by the weaker:</p>

<p>2') It is easy to approximate the expectation of any polynomially computable function over the uniform distribution on the solutions.</p>

<p>My thoughts:</p>

<ul>
<li><p>If 2) holds, then by integrating the constant function $1$, one can approximately count the number of solutions to the problem. </p></li>
<li><p>If 2) holds but 1) does not, then the problem cannot be self-reducible, as otherwise by reductions in <a href="https://pdfs.semanticscholar.org/50a6/261904e0c56a4430e5cd98c6e7f71f0ed1b4.pdf" rel="noreferrer">JVV86</a> there would be an efficient algorithm for approximate uniform sampling by using the observation of the previous bullet.</p></li>
<li><p>A reasonable candidate technique for accomplishing 2) without 1) is importance sampling.</p></li>
<li><p>I know of situations where sampling is easy and counting is hard (so integration is hard); the case of uniformly sampling solutions to a DNF equation is explained in JVV86 section 4.</p></li>
<li><p>A specific candidate problem I would be interested in is the problem of sampling directed simple cycles from a digraph. (It is shown in section 5. of JVV86 that this problem is hard.) However, I know that counting the number of directed simple cycles is NP-hard, by a similar argument to that presented in JVV, and therefore so is integrating over them. However, perhaps 2') can be solved in this case?</p></li>
</ul>

<p>-- </p>

<p>Any references that seem relevant would be very appreciated, even if they are not a direct answer to my question.</p>

<counting-complexity><randomness><probabilistic-computation><probabilistic-complexity>
Sets of solutions which it is hard to uniformly sample from, but easy to integrate functions over? (Or compute expectations over?)
Elle Najt
# Sets of solutions which it is hard to uniformly sample from, but easy to integrate functions over? (Or compute expectations over?)

(Asked by: username_0 on 2018-09-17)

I'm curious if there is a problem (e.g. something like perfect matchings on a given graph, number of solutions to a boolean equation, etc. for precise frameowork see [JVV86](https://pdfs.semanticscholar.org/50a6/261904e0c56a4430e5cd98c6e7f71f0ed1b4.pdf)) such that:

1) It is hard to sample approximately uniformly at random from the set of all solutions

2) It is easy to approximately integrate any (polynomially computable) function over the solutions.

I would also be happy with replacing 2) by the weaker:

2') It is easy to approximate the expectation of any polynomially computable function over the uniform distribution on the solutions.

My thoughts:

* If 2) holds, then by integrating the constant function $1$, one can approximately count the number of solutions to the problem.
* If 2) holds but 1) does not, then the problem cannot be self-reducible, as otherwise by reductions in [JVV86](https://pdfs.semanticscholar.org/50a6/261904e0c56a4430e5cd98c6e7f71f0ed1b4.pdf) there would be an efficient algorithm for approximate uniform sampling by using the observation of the previous bullet.
* A reasonable candidate technique for accomplishing 2) without 1) is importance sampling.
* I know of situations where sampling is easy and counting is hard (so integration is hard); the case of uniformly sampling solutions to a DNF equation is explained in JVV86 section 4.
* A specific candidate problem I would be interested in is the problem of sampling directed simple cycles from a digraph. (It is shown in section 5. of JVV86 that this problem is hard.) However, I know that counting the number of directed simple cycles is NP-hard, by a similar argument to that presented in JVV, and therefore so is integrating over them. However, perhaps 2') can be solved in this case?

--

Any references that seem relevant would be very appreciated, even if they are not a direct answer to my question.

**Tags:** counting-complexity, randomness, probabilistic-computation, probabilistic-complexity

---

## 1 answers

### Answer by username_1 on 2018-09-17. Score: 6 | Accepted answer

There is no such problem. If it's hard to sample, it's hard to integrate.

Here is a sketch of the reason why. Represent every solution $x$ by a $n$-bit string $x\_1,\dots,x\_n$. If you can integrate over the set of all solutions, here is an algorithm to sample from all solutions:

1. Count the number of solutions; call it $N\_0$. (As you say, this can be done efficiently, as it is just an integral.) Count the number of solutions $x$ such that $x\_1=1$; call it $N'\_0$. (This is an integral of the function $f(x)$ over all solutions, where $f(x\_1,\dots,x\_n) = x\_1$, so it can be computed efficiently.) With probability $N'\_0/N\_0$, set $y\_1 = 1$, else set $y\_1 = 0$.
2. Count the number of solutions $x$ such that $x\_1=y\_1$; call it $N\_1$. (This is another integral.) Count the number of solutions $x$ such that $x\_1=y\_1$ and $x\_2=1$; call it $N'\_1$. With probability $N'\_1/N\_1$, set $y\_2 = 1$, else set $y\_2 = 0$.
3. Count the number $N\_2$ of solutions $x$ such that $x\_1=y\_1$ and $x\_2=y\_2$, and the number $N'\_2$ of solutions $x$ such that $x\_1=y\_1$ and $x\_2=y\_2$ and $x\_3=1$. With probability $N'\_2/N\_2$, set $y\_3=1$, else set $y\_3=0$.
4. And so on. When this procedure finishes, output $y\_1,y\_2,\dots,y\_n$. This will be a valid solution, sampled uniformly at random from the set of all solutions.

It's easy to see that if you have a way to compute the exact integral, you'll obtain a way to sample exactly uniformly at random. I expect it will follow that if you can approximate the integral, you can sample approximately uniformly at random; I'll let you compute the parameters and relation between the two approximations.

---

You can also implement this algorithm using expectations. Let $X$ be a r.v. that is uniformly distributed on the set of all solutions. Set $y\_1=1$ with probability $\mathbb{E}[X\_1]$; set $y\_2=1$ with probability $\mathbb{E}[\mathbf{1}\_{X\_1 = y\_1} \cdot X\_2] / \mathbb{E}[\mathbf{1}\_{X\_1 = y\_1}]$; and so on. Here I use $\mathbf{1}\_{e(X)}$ to represent a function that is 1 if $e(X)$ holds, and 0 otherwise. Why does this work? Well, $\Pr[X\_1=1] = \mathbb{E}[X\_1]$, and

$$\Pr[X\_2=1|X\_1=y\_1] = {\Pr[X\_1 = y\_1 \land X\_2=1] \over \Pr[X\_1=y\_1]}
= {\mathbb{E}[\mathbf{1}\_{X\_1 = y\_1} \cdot X\_2] \over \mathbb{E}[\mathbf{1}\_{X\_1 = y\_1}]}.$$

> **Comments:**
> **username_0**: Got it, thanks! The thing I was missing is that you don't need self reducibility to reduce sampling to counting if you can integrate over all functions (your construction is essentially the reduction JVV86 for self reducible problems). What about computing expectations against the uniform distribution - that is, 1) and 2')? I don't think the same idea will work for that.  
> **username_1**: @Lorenzo, see edited answer, at the bottom.  

---

