<p>Define a language $L$ to be <em>nicely-ordered</em> if $L \subseteq a^*b^*c^*d^*\cdots$, i.e., in every word of $L$, the letters appear solely in lexicographic order.</p>

<p><em>Conjecture:</em> the optimum is always obtained by some nicely-ordered language, i.e., if $G$ is the smallest such grammar that generates $S$, then there exists a grammar $G'$ of the same size that also generates $S$ and such that $L(G')$ is nicely-ordered. </p>

<p>If this conjecture is true, then your problem reduces to <a href="https://cs.stackexchange.com/questions/53279/is-there-a-known-method-for-constructing-a-grammar-given-a-finite-set-of-finite">a kind of grammar induction</a>.  In particular, we are given a finite language, and the goal is to find the minimal context-free grammar that generates that language.</p>

<p>There are heuristics for this problem.  For instance, see the <a href="https://en.wikipedia.org/wiki/Sequitur_algorithm" rel="nofollow noreferrer">Sequitur</a>, <a href="https://en.wikipedia.org/wiki/Lempel%E2%80%93Ziv%E2%80%93Welch" rel="nofollow noreferrer">Lempel-Ziv-Welch</a>, and <a href="https://en.wikipedia.org/wiki/Byte_pair_encoding" rel="nofollow noreferrer">byte pair encoding</a> algorithms, as well as <a href="https://cs.stackexchange.com/a/44585/755">others</a>.</p>

<p>In the general case (given an arbitrary finite language, find the smallest context-free language that generates it), the problem is NP-hard; I don't know if it remains NP-hard when restricted to nicely-ordered finite languages.  You could take a look at the standard proof of NP-hardness and see if it can be adjusted to apply to nicely-ordered languages.  See also <a href="https://cstheory.stackexchange.com/q/5014/5038">Lower bounds on the size of CFGs for specific finite languages</a> for other results in this space.</p>

<hr>

<p>So, back to the conjecture.  One angle of attack on the problem might to try to prove or disprove this conjecture.  For instance, you might try writing a program to search for counterexamples (generate small random sets, look for the optimal grammar, check whether the conjecture holds, and repeat millions of times).  If the conjecture is true, it might make your problem much easier to solve.  If it's false, it might shed light on what makes the problem challenging.</p>

CC BY-SA 3.0
2016-09-02T22:15:10.593Z
Thanks for the references. I think your conjecture is not right. For example, if $\Sigma=\{a_1,\dots,a_n,b_1,\dots,b_k,c_1,\dots,c_n,d_1,\dots,d_k\}$ with the letter ordering being as written above. Also, let $S=\{a_i b_1b_2\dots b_kc_id_1d_2\dots d_k~|~1 \leq i \leq n\}$, i.e., $S$ has $n$ words of size $2k+2$. Since $b_1\dots b_kd_1 \dots d_k$ happens everywhere, introduce new non-terminal $B$ to represent them (despite not being next to each other in the ordering). If $L$ was to be nicely-ordered, we would need two new nonterminals and had to use both in all rules, i.e., extra $O(n)$ size.
Shahab
CC BY-SA 3.0
2016-09-02T22:17:22.347Z
So, now, the question becomes if some ordering always exists that satisfies the nicely-ordered requirement. My guess would be that it's not the case either but I might be wrong ....
Shahab
CC BY-SA 3.0
2016-09-03T00:05:18.383Z
@Shahab, very nice!  I am persuaded.
D.W.
CC BY-SA 3.0
2016-09-01T18:52:33.910Z
2016-09-01T18:52:33.910Z
2017-04-13T12:48:30.830Z
D.W.
<p>Because the order of your symbols does not matter, you can always move the nonterminals to the very right of the productions. If there is no more than one in any rule, the resulting grammar is regular. If there are several, I think they can easily be coded into a single one.</p>

<p>Thus you can look for the minimal regular grammar, and this problem is quite well-studied.</p>

CC BY-SA 3.0
2016-09-02T21:41:17.553Z
Can you please also give some citation?
Shahab
CC BY-SA 3.0
2016-09-03T00:06:58.633Z
*"If there are several, I think they can easily be coded into a single one."* - Can you give a justification or proof of this statement?  It doesn't look right to me.  Suppose $AB$ appears at the end of one rule, $BC$ appears at the end of another rule, $AC$ appears at the end of a third rule, and $A$ appears at the end of a fourth rule.  Now what?  How are we going to code them so each rule ends with only a single terminal, without increasing the size of the grammar?  (If you try to replace $AB$, $BC$, $AC$, $A$ with $X$, $Y$, $Z$, $A$, but now the rules for $X,Y,Z$ might be much larger.)
D.W.
CC BY-SA 3.0
2016-09-03T20:16:28.483Z
Yes, @D.W. the grammar will become larger. Even more than what you suggest, because you have to do a "leftmost" derivation (at least if you want to preserve the original order of terminals, which is not so relevant here) in the compressed symbols, so X might need many steps until it descends down to a single original non-terminal. But if you then minimize this larger grammar, the result will be minimal. Of course, one "set language" corresponds to many string languages, and it is not clear which one will have the smallest minimal grammar.
Peter Leupold
CC BY-SA 3.0
2016-09-03T20:27:31.307Z
The flaw in your reasoning is that you assume that the minimal grammar for this language will be regular.  But that's not correct. There might be a context-free grammar that's smaller than any regular grammar for the language.  Consequently, finding some regular grammar for the language and then looking for the minimal regular grammar for that language does not necessarily yield an optimum solution to the original problem; there might be a context-free grammar that's smaller still.
D.W.
CC BY-SA 3.0
2016-09-03T20:37:30.087Z
Alright - minimality for regular grammars is measured in the number of nonterminals, while the question here asks for number of occurences of any kind of symbol in the rules. For this, by the way, it is completely irrelevant how many new non-terminals are introduced. Anyway, this is not a flaw in my reasoning, I just suggested to look at the problem in this way. Andmy intuition is that normally if not always the minimalities will coincide.
Peter Leupold
CC BY-SA 3.0
2016-09-02T11:48:48.940Z
2016-09-02T11:48:48.940Z
Peter Leupold
<p>Are there any results known about the size of smallest context free grammar that generates a set of sets?</p>

<p>That is, I am given an alphabet $\Sigma$ as well as a set $S \subseteq \mathbb{P}(\Sigma)$ and I want to find the smallest context free grammar $G$ whose language $L$ has the property that $A \in S \Leftrightarrow (\exists x \in L~\forall l \in \Sigma~[l \in x \Leftrightarrow l \in A])$.</p>

<p>For instance, if $\Sigma =\{a,b,c,d\}$ and $S=\{\{d,a,c,b\},\{b,c,a\},\{a,b\},\{a\},\{\}\}$, I can have following grammar $G$ of size 14 which corresponds to the given $S$:
$$
\begin{array}{l}
G \rightarrow ``dacb"\\
G \rightarrow ``bca"\\
G \rightarrow ``ab"\\
G \rightarrow ``a"\\
G \rightarrow \lambda
\end{array}
$$</p>

<p>However, I can also use the fact that $S$ is a set and have the following smaller grammar $G'$ of size 13 which corresponds to representing $S$ as $``\{\{a,b,c,d\},\{a,b,c\},\{a,b\},\{a\}\}"$
$$
\begin{array}{l}
G' \rightarrow aA\\
A \rightarrow bB\\
A \rightarrow \lambda\\
B \rightarrow cd\\
B \rightarrow c\\
B \rightarrow \lambda
\end{array}
$$</p>

<p>I am interested in the following questions:</p>

<ol>
<li><p>Has this problem been studied somewhere? In this regard, do we know the complexity of this problem? If it happens to be computationally complex, are there any approximation algorithms? What about good heuristics?</p></li>
<li><p>Has problems similar to this been studied before? For example, instead of using context free grammars, use set theoretic operations such as union and intersection or maybe use some other type of grammar (maybe more restricted than CFGs?).</p></li>
</ol>

<p>I am interested in all results that use the underlying assumption of wanting to represent a set (i.e., order is irrelevant). I know that, in the compression literature, there are works on optimal arithmetical encodings of sets. However, I cannot see any direct translation from those results to this domain where we are interested in finding a short grammar.</p>

CC BY-SA 3.0
2016-09-01T12:57:04.740Z
@Aryeh Remember that $A \in S$ "if and only if" the language has a sentence $x$ containing exactly the same set of letters. So, if $L$ generates $\Sigma^*$, it can only represent $\mathbb{P}(\Sigma)$ because that is the only set that contains every possible subset of $\Sigma$.
Shahab
CC BY-SA 3.0
2016-09-01T15:11:51.183Z
Perhaps this [paper](http://www.cs.toronto.edu/~yuvalf/CFG-LB.pdf) helps.
Yuval Filmus
<cc.complexity-theory><approximation-algorithms><grammars><context-free><heuristics>
Finding smallest context free grammar that generates a set of sets
Shahab
# Finding smallest context free grammar that generates a set of sets

(Asked by: username_0 on 2016-08-30)

Are there any results known about the size of smallest context free grammar that generates a set of sets?

That is, I am given an alphabet $\Sigma$ as well as a set $S \subseteq \mathbb{P}(\Sigma)$ and I want to find the smallest context free grammar $G$ whose language $L$ has the property that $A \in S \Leftrightarrow (\exists x \in L~\forall l \in \Sigma~[l \in x \Leftrightarrow l \in A])$.

For instance, if $\Sigma =\{a,b,c,d\}$ and $S=\{\{d,a,c,b\},\{b,c,a\},\{a,b\},\{a\},\{\}\}$, I can have following grammar $G$ of size 14 which corresponds to the given $S$:
$$
\begin{array}{l}
G \rightarrow ``dacb"\\
G \rightarrow ``bca"\\
G \rightarrow ``ab"\\
G \rightarrow ``a"\\
G \rightarrow \lambda
\end{array}
$$

However, I can also use the fact that $S$ is a set and have the following smaller grammar $G'$ of size 13 which corresponds to representing $S$ as $``\{\{a,b,c,d\},\{a,b,c\},\{a,b\},\{a\}\}"$
$$
\begin{array}{l}
G' \rightarrow aA\\
A \rightarrow bB\\
A \rightarrow \lambda\\
B \rightarrow cd\\
B \rightarrow c\\
B \rightarrow \lambda
\end{array}
$$

I am interested in the following questions:

1. Has this problem been studied somewhere? In this regard, do we know the complexity of this problem? If it happens to be computationally complex, are there any approximation algorithms? What about good heuristics?
2. Has problems similar to this been studied before? For example, instead of using context free grammars, use set theoretic operations such as union and intersection or maybe use some other type of grammar (maybe more restricted than CFGs?).

I am interested in all results that use the underlying assumption of wanting to represent a set (i.e., order is irrelevant). I know that, in the compression literature, there are works on optimal arithmetical encodings of sets. However, I cannot see any direct translation from those results to this domain where we are interested in finding a short grammar.

**Tags:** cc.complexity-theory, approximation-algorithms, grammars, context-free, heuristics

> **Comments:**
> **username_0**: @Aryeh Remember that $A \in S$ "if and only if" the language has a sentence $x$ containing exactly the same set of letters. So, if $L$ generates $\Sigma^*$, it can only represent $\mathbb{P}(\Sigma)$ because that is the only set that contains every possible subset of $\Sigma$.  
> **username_1**: Perhaps this [paper](http://www.cs.toronto.edu/~yuvalf/CFG-LB.pdf) helps.  

---

## 2 answers

### Answer by username_2 on 2016-09-01. Score: 3

Define a language $L$ to be *nicely-ordered* if $L \subseteq a^\*b^\*c^\*d^\*\cdots$, i.e., in every word of $L$, the letters appear solely in lexicographic order.

*Conjecture:* the optimum is always obtained by some nicely-ordered language, i.e., if $G$ is the smallest such grammar that generates $S$, then there exists a grammar $G'$ of the same size that also generates $S$ and such that $L(G')$ is nicely-ordered.

If this conjecture is true, then your problem reduces to [a kind of grammar induction](https://cs.stackexchange.com/questions/53279/is-there-a-known-method-for-constructing-a-grammar-given-a-finite-set-of-finite). In particular, we are given a finite language, and the goal is to find the minimal context-free grammar that generates that language.

There are heuristics for this problem. For instance, see the [Sequitur](https://en.wikipedia.org/wiki/Sequitur_algorithm), [Lempel-Ziv-Welch](https://en.wikipedia.org/wiki/Lempel%E2%80%93Ziv%E2%80%93Welch), and [byte pair encoding](https://en.wikipedia.org/wiki/Byte_pair_encoding) algorithms, as well as [others](https://cs.stackexchange.com/a/44585/755).

In the general case (given an arbitrary finite language, find the smallest context-free language that generates it), the problem is NP-hard; I don't know if it remains NP-hard when restricted to nicely-ordered finite languages. You could take a look at the standard proof of NP-hardness and see if it can be adjusted to apply to nicely-ordered languages. See also [Lower bounds on the size of CFGs for specific finite languages](https://cstheory.stackexchange.com/q/5014/5038) for other results in this space.

---

So, back to the conjecture. One angle of attack on the problem might to try to prove or disprove this conjecture. For instance, you might try writing a program to search for counterexamples (generate small random sets, look for the optimal grammar, check whether the conjecture holds, and repeat millions of times). If the conjecture is true, it might make your problem much easier to solve. If it's false, it might shed light on what makes the problem challenging.

> **Comments:**
> **username_0**: Thanks for the references. I think your conjecture is not right. For example, if $\Sigma=\{a_1,\dots,a_n,b_1,\dots,b_k,c_1,\dots,c_n,d_1,\dots,d_k\}$ with the letter ordering being as written above. Also, let $S=\{a_i b_1b_2\dots b_kc_id_1d_2\dots d_k~|~1 \leq i \leq n\}$, i.e., $S$ has $n$ words of size $2k+2$. Since $b_1\dots b_kd_1 \dots d_k$ happens everywhere, introduce new non-terminal $B$ to represent them (despite not being next to each other in the ordering). If $L$ was to be nicely-ordered, we would need two new nonterminals and had to use both in all rules, i.e., extra $O(n)$ size.  
> **username_0**: So, now, the question becomes if some ordering always exists that satisfies the nicely-ordered requirement. My guess would be that it's not the case either but I might be wrong ....  
> **username_2**: @username_0, very nice!  I am persuaded.  

---

### Answer by username_3 on 2016-09-02. Score: 1

Because the order of your symbols does not matter, you can always move the nonterminals to the very right of the productions. If there is no more than one in any rule, the resulting grammar is regular. If there are several, I think they can easily be coded into a single one.

Thus you can look for the minimal regular grammar, and this problem is quite well-studied.

> **Comments:**
> **username_0**: Can you please also give some citation?  
> **username_2**: *"If there are several, I think they can easily be coded into a single one."* - Can you give a justification or proof of this statement?  It doesn't look right to me.  Suppose $AB$ appears at the end of one rule, $BC$ appears at the end of another rule, $AC$ appears at the end of a third rule, and $A$ appears at the end of a fourth rule.  Now what?  How are we going to code them so each rule ends with only a single terminal, without increasing the size of the grammar?  (If you try to replace $AB$, $BC$, $AC$, $A$ with $X$, $Y$, $Z$, $A$, but now the rules for $X,Y,Z$ might be much larger.)  
> **username_3**: Yes, @username_2 the grammar will become larger. Even more than what you suggest, because you have to do a "leftmost" derivation (at least if you want to preserve the original order of terminals, which is not so relevant here) in the compressed symbols, so X might need many steps until it descends down to a single original non-terminal. But if you then minimize this larger grammar, the result will be minimal. Of course, one "set language" corresponds to many string languages, and it is not clear which one will have the smallest minimal grammar.  
> **username_2**: The flaw in your reasoning is that you assume that the minimal grammar for this language will be regular.  But that's not correct. There might be a context-free grammar that's smaller than any regular grammar for the language.  Consequently, finding some regular grammar for the language and then looking for the minimal regular grammar for that language does not necessarily yield an optimum solution to the original problem; there might be a context-free grammar that's smaller still.  
> **username_3**: Alright - minimality for regular grammars is measured in the number of nonterminals, while the question here asks for number of occurences of any kind of symbol in the rules. For this, by the way, it is completely irrelevant how many new non-terminals are introduced. Anyway, this is not a flaw in my reasoning, I just suggested to look at the problem in this way. Andmy intuition is that normally if not always the minimalities will coincide.  

---

