<p>The generalized question about exact counting of flats of a matroid has an answer.</p>
<p>Let <span class="math-container">$G$</span> be a graph, and let <span class="math-container">$G^{\circ}$</span> be the same graph with a loop at every vertex. Then, <a href="https://en.wikipedia.org/wiki/Bicircular_matroid#Flats" rel="nofollow noreferrer">the flats of the</a> bicircular matroid of <span class="math-container">$G^{\circ}$</span> are exactly the forests of <span class="math-container">$G$</span>; see <a href="https://academic.oup.com/qjmath/article-abstract/33/4/493/1498307" rel="nofollow noreferrer">Bicircular geometry and the lattice of forests of a graph</a>.</p>
<p>It is known, on the other hand, that counting the number of spanning forests in a graph is a <span class="math-container">$\#$</span>P-hard problem. See for instance <a href="https://www.worldscientific.com/doi/10.1142/S0129054109006437" rel="nofollow noreferrer">Fast exponential-time algorithms for the forest counting and the Tutte polynomial computation in graph classes</a>.</p>
<p><strong>Putting these together shows that exact counting of flats is in general <span class="math-container">$\#$</span>P-hard.</strong></p>
<p><strong>Remark:</strong> The approach that led to finding this example was to search through examples of matroids on wikipedia, and reading the section on flats until I saw one that I knew to be related to NP-hard problems.</p>
<p><strong>Remark:</strong> Additionally, it is known that the forest counting problem remains hard when parametrized by the number of edges. See <a href="https://arxiv.org/pdf/1611.01823.pdf" rel="nofollow noreferrer">here</a>. (Although that paper points out that the problem is fixed parameter tractable for plane graphs, because of this algorithm: <a href="https://arxiv.org/abs/cs/9911003" rel="nofollow noreferrer">https://arxiv.org/abs/cs/9911003</a> .)</p>
<p><strong>Sampling or approximate counting:</strong> I'm not sure about sampling or approximate counting flats from a matroid.</p>
<p>There appears to be a long line of work on the problem of approximately counting forests: see <a href="https://link.springer.com/article/10.1007/BF01608795" rel="nofollow noreferrer">Forests, colorings and acyclic orientations of the square lattice</a> or <a href="http://www.math.cmu.edu/%7Eaf1p/Teaching/MCC17/Papers/trees.pdf" rel="nofollow noreferrer">A Randomised Approximation Algorithm for
Counting the Number of Forests in Dense Graphs</a>. Perhaps another matroid would provide a more tractable example of hardness of approximation / sampling. I will update this answer if I learn about one.</p>

CC BY-SA 4.0
2020-12-12T13:28:06.077Z
2020-12-20T04:40:53.583Z
2020-12-20T04:40:53.583Z
Elle Najt
<p><strong>Question:</strong> Is it possible to uniformly sample in polynomial time from the set of all connected partitions of a graph? Or is there a JVV type argument that proves this to be NP-hard?</p>
<p>To clarify: By a connected partition of <span class="math-container">$G = (V,E)$</span> I mean a partition of the nodes of <span class="math-container">$V$</span> into blocks <span class="math-container">$A_1, \ldots, A_m$</span>, where each <span class="math-container">$A_i$</span> induces a connected subgraph. There are no constraints on the number of blocks. In different language, I am asking about uniformly sampling from the flats of the graphic matroid of <span class="math-container">$G$</span>.</p>
<p>I thought about this for a while, but couldn't figure it out. Maybe someone knows the answer or can suggest something to try? The question of counting the number of connected partitions is closely related and also interesting to me.</p>
<p>Here are some related things I know:</p>
<ul>
<li><p>If you fix the number of blocks of the partition, then this is NP-hard to sample. For example, in the <span class="math-container">$2$</span> block case, this follows by reducing to uniformly sampling simple cycle in the plane dual graph. (Both the NP-hard optimization problem and the probability concentration gadget do not carry over meaningfully to the any number of blocks case.)</p>
</li>
<li><p>Enumeration is possible in total linear time. For instance this follows from this: <a href="https://arxiv.org/pdf/1107.4301.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1107.4301.pdf</a></p>
</li>
<li><p>The idea of the previously linked paper also lets you sample low rank flats somewhat efficiently. That is, by picking weights <span class="math-container">$2^k$</span> so that each flat has a unique min basis, and then using that a rank <span class="math-container">$r$</span> flat in the graphic matroid of a plane graph will have <span class="math-container">$O(6^{2r})$</span> basis ( <a href="https://arxiv.org/abs/0912.0712" rel="nofollow noreferrer">https://arxiv.org/abs/0912.0712</a> ) to obtain a bound on the rejection rate obtained by sampling a nearly uniform rank r independent set (using basis-exchange walk) and then checking if it is the min-weight basis of its span.</p>
</li>
<li><p>There is natural family of Glauber dynamics type Markov chains to try: Any connected partition can be identified with the corresponding flat in the graphic matroid (i.e. the edges with both endpoints in the same block), and one can introduce a Metropolis-Hasting's score function that measures deviation of a subset of edges from being a flat. For instance, for any set of edges, <span class="math-container">$J \subset E$</span>, and some fixed <span class="math-container">$\lambda &gt; 0$</span>, the score can be <span class="math-container">$\lambda^{ loops(G/J)}$</span>, where <span class="math-container">$loops$</span> measures the number of self loops, that is, edges in <span class="math-container">$E \setminus J$</span> that go between the same connected components of <span class="math-container">$G[J]$</span>. For <span class="math-container">$\lambda = 1$</span> this mixes rapidly, but the stationary distribution is not concentrated on the flats.</p>
</li>
<li><p>For <span class="math-container">$\lambda$</span> very small the stationary distribution in the chain of the previous bullet is concentrated on the flats. However, there are examples where the mixing slows down when decreasing <span class="math-container">$\lambda$</span> (for instance, a <span class="math-container">$d$</span>-dipole graph will have a hard time leaving the flat containing all edges). I couldn't figure out cases like the grid graph.</p>
</li>
<li><p>Describing connected partitions as flats as in the Markov chain two bullets above leads to another total linear time enumeration algorithm; you can tell whether it is possible to extend a set of deleted and a set of included edges to a flat by checking whether there are any contradictions. Checking whether there are any such contradictions is straightforward to do efficiently, and so one can use backtracking to enumerate.</p>
</li>
<li><p>Somewhat surprisingly, for small grid graphs, say <span class="math-container">$8 \times 8$</span>, rejection sampling for flats from subsets of edges is reasonably efficient. This can be used to test the Markov chain in the previous bullet for grid graphs, and it seems to work ok. Empirically, a typical connected partition of a grid graph appears to consist of many small blocks.</p>
</li>
<li><p>Another Markov chain uses spanning trees to parametrize partitions. Versions of this have been written about in a few places: <a href="https://arxiv.org/abs/1908.07176" rel="nofollow noreferrer">https://arxiv.org/abs/1908.07176</a> (see the thesis of Tien Vo on &quot;Large-Scale Statistical Inference for Graph-Associated data&quot; for details on the Markov chain mentioned in that paper), <a href="https://arxiv.org/abs/1808.00050" rel="nofollow noreferrer">https://arxiv.org/abs/1808.00050</a>, <a href="https://arxiv.org/abs/1911.05725" rel="nofollow noreferrer">https://arxiv.org/abs/1911.05725</a>.</p>
</li>
<li><p>The complete graph case is already interesting and non-trivial: <a href="https://www.sciencedirect.com/science/article/pii/0097316583900092" rel="nofollow noreferrer">https://www.sciencedirect.com/science/article/pii/0097316583900092</a></p>
</li>
<li><p>For purposes of counting or sampling connected partitions on smaller graphs, ZDDs are a natural structure to use. The python package Graphillion should be able to handle this -- I spoke with one of the developers recently and they apparently have been working on adding connected partitions: <a href="https://github.com/takemaru/graphillion/issues/43" rel="nofollow noreferrer">https://github.com/takemaru/graphillion/issues/43</a></p>
</li>
</ul>

CC BY-SA 4.0
2020-04-08T09:37:54.460Z
I believe sampling/counting flats is still open. A related open problem is the log-concavity of the number of flats parametrised by sizes, or the so-called Whitney number of the second kind. See e.g. Section 4.10 of https://arxiv.org/pdf/1705.07960.pdf
Heng Guo
CC BY-SA 4.0
2020-04-08T09:41:08.710Z
(cont.) Usually an easy to sample structure form a log-concave sequence when parameterised by sizes. (If there were two exponentially large peaks, local Markov chains are always slow.) This is the case for independent sets for matroids thanks to a sequence of great work by Huh and others. Flats are conjectured to be log-concave but we still don't know.
Heng Guo
CC BY-SA 4.0
2020-04-14T17:12:17.310Z
@HengGuo Just to clarify -- you think it's open that counting flats is #P complete, not even when restricted to the case of graphic matroids? (E.g. in the context where the Matroids are representable over some field, or the family of matroids is given by a polytime constructable circuit realizing the independence oracle.) Thanks for the interesting comments.
Elle Najt
CC BY-SA 4.0
2020-04-15T09:58:17.253Z
I think approximate counting / sampling is open. I cannot recall exact counting being #P hard, but perhaps that's not too hard to show? (Just my guess and I haven't tried.)
Heng Guo
<graph-algorithms><np-hardness><counting-complexity><randomized-algorithms><partition-problem>
Uniformly sampling or counting connected graph partitions with any number of blocks
Elle Najt
# Uniformly sampling or counting connected graph partitions with any number of blocks

(Asked by: username_0 on 2020-04-08)

**Question:** Is it possible to uniformly sample in polynomial time from the set of all connected partitions of a graph? Or is there a JVV type argument that proves this to be NP-hard?

To clarify: By a connected partition of $G = (V,E)$ I mean a partition of the nodes of $V$ into blocks $A\_1, \ldots, A\_m$, where each $A\_i$ induces a connected subgraph. There are no constraints on the number of blocks. In different language, I am asking about uniformly sampling from the flats of the graphic matroid of $G$.

I thought about this for a while, but couldn't figure it out. Maybe someone knows the answer or can suggest something to try? The question of counting the number of connected partitions is closely related and also interesting to me.

Here are some related things I know:

* If you fix the number of blocks of the partition, then this is NP-hard to sample. For example, in the $2$ block case, this follows by reducing to uniformly sampling simple cycle in the plane dual graph. (Both the NP-hard optimization problem and the probability concentration gadget do not carry over meaningfully to the any number of blocks case.)
* Enumeration is possible in total linear time. For instance this follows from this: <https://arxiv.org/pdf/1107.4301.pdf>
* The idea of the previously linked paper also lets you sample low rank flats somewhat efficiently. That is, by picking weights $2^k$ so that each flat has a unique min basis, and then using that a rank $r$ flat in the graphic matroid of a plane graph will have $O(6^{2r})$ basis ( <https://arxiv.org/abs/0912.0712> ) to obtain a bound on the rejection rate obtained by sampling a nearly uniform rank r independent set (using basis-exchange walk) and then checking if it is the min-weight basis of its span.
* There is natural family of Glauber dynamics type Markov chains to try: Any connected partition can be identified with the corresponding flat in the graphic matroid (i.e. the edges with both endpoints in the same block), and one can introduce a Metropolis-Hasting's score function that measures deviation of a subset of edges from being a flat. For instance, for any set of edges, $J \subset E$, and some fixed $\lambda > 0$, the score can be $\lambda^{ loops(G/J)}$, where $loops$ measures the number of self loops, that is, edges in $E \setminus J$ that go between the same connected components of $G[J]$. For $\lambda = 1$ this mixes rapidly, but the stationary distribution is not concentrated on the flats.
* For $\lambda$ very small the stationary distribution in the chain of the previous bullet is concentrated on the flats. However, there are examples where the mixing slows down when decreasing $\lambda$ (for instance, a $d$-dipole graph will have a hard time leaving the flat containing all edges). I couldn't figure out cases like the grid graph.
* Describing connected partitions as flats as in the Markov chain two bullets above leads to another total linear time enumeration algorithm; you can tell whether it is possible to extend a set of deleted and a set of included edges to a flat by checking whether there are any contradictions. Checking whether there are any such contradictions is straightforward to do efficiently, and so one can use backtracking to enumerate.
* Somewhat surprisingly, for small grid graphs, say $8 \times 8$, rejection sampling for flats from subsets of edges is reasonably efficient. This can be used to test the Markov chain in the previous bullet for grid graphs, and it seems to work ok. Empirically, a typical connected partition of a grid graph appears to consist of many small blocks.
* Another Markov chain uses spanning trees to parametrize partitions. Versions of this have been written about in a few places: <https://arxiv.org/abs/1908.07176> (see the thesis of Tien Vo on "Large-Scale Statistical Inference for Graph-Associated data" for details on the Markov chain mentioned in that paper), <https://arxiv.org/abs/1808.00050>, <https://arxiv.org/abs/1911.05725>.
* The complete graph case is already interesting and non-trivial: <https://www.sciencedirect.com/science/article/pii/0097316583900092>
* For purposes of counting or sampling connected partitions on smaller graphs, ZDDs are a natural structure to use. The python package Graphillion should be able to handle this -- I spoke with one of the developers recently and they apparently have been working on adding connected partitions: <https://github.com/takemaru/graphillion/issues/43>

**Tags:** graph-algorithms, np-hardness, counting-complexity, randomized-algorithms, partition-problem
> **Comments:**
> **username_1**: I believe sampling/counting flats is still open. A related open problem is the log-concavity of the number of flats parametrised by sizes, or the so-called Whitney number of the second kind. See e.g. Section 4.10 of https://arxiv.org/pdf/1705.07960.pdf  
> **username_1**: (cont.) Usually an easy to sample structure form a log-concave sequence when parameterised by sizes. (If there were two exponentially large peaks, local Markov chains are always slow.) This is the case for independent sets for matroids thanks to a sequence of great work by Huh and others. Flats are conjectured to be log-concave but we still don't know.  
> **username_0**: @HengGuo Just to clarify -- you think it's open that counting flats is #P complete, not even when restricted to the case of graphic matroids? (E.g. in the context where the Matroids are representable over some field, or the family of matroids is given by a polytime constructable circuit realizing the independence oracle.) Thanks for the interesting comments.  
> **username_1**: I think approximate counting / sampling is open. I cannot recall exact counting being #P hard, but perhaps that's not too hard to show? (Just my guess and I haven't tried.)  

---

## 1 answers

### Answer by username_0 on 2020-12-12. Score: 1

The generalized question about exact counting of flats of a matroid has an answer.

Let $G$ be a graph, and let $G^{\circ}$ be the same graph with a loop at every vertex. Then, [the flats of the](https://en.wikipedia.org/wiki/Bicircular_matroid#Flats) bicircular matroid of $G^{\circ}$ are exactly the forests of $G$; see [Bicircular geometry and the lattice of forests of a graph](https://academic.oup.com/qjmath/article-abstract/33/4/493/1498307).

It is known, on the other hand, that counting the number of spanning forests in a graph is a $\#$P-hard problem. See for instance [Fast exponential-time algorithms for the forest counting and the Tutte polynomial computation in graph classes](https://www.worldscientific.com/doi/10.1142/S0129054109006437).

**Putting these together shows that exact counting of flats is in general $\#$P-hard.**

**Remark:** The approach that led to finding this example was to search through examples of matroids on wikipedia, and reading the section on flats until I saw one that I knew to be related to NP-hard problems.

**Remark:** Additionally, it is known that the forest counting problem remains hard when parametrized by the number of edges. See [here](https://arxiv.org/pdf/1611.01823.pdf). (Although that paper points out that the problem is fixed parameter tractable for plane graphs, because of this algorithm: <https://arxiv.org/abs/cs/9911003> .)

**Sampling or approximate counting:** I'm not sure about sampling or approximate counting flats from a matroid.

There appears to be a long line of work on the problem of approximately counting forests: see [Forests, colorings and acyclic orientations of the square lattice](https://link.springer.com/article/10.1007/BF01608795) or [A Randomised Approximation Algorithm for
Counting the Number of Forests in Dense Graphs](http://www.math.cmu.edu/%7Eaf1p/Teaching/MCC17/Papers/trees.pdf). Perhaps another matroid would provide a more tractable example of hardness of approximation / sampling. I will update this answer if I learn about one.


---

