<p>What you need for a Markov chain to converge to an equal distribution over all possible sequences of balls is that it is reversible: the probability of moving from sequence $i$ to sequence $j$ is the same as moving in the opposite direction. I thus propose that you use the following moves (with some fixed probability distribution to choose which type of move to make) to perform a Markov chain on all possible sequences. In the following, a "run" is a maximum-length consecutive subsequence of balls of the same colour. This Markov chain relies on there being at least three colours.</p>

<ol>
<li><p>Pick two runs at random. If you can exchange them and still have a legal sequence, do so.</p></li>
<li><p>Pick two adjacent runs. If you can exchange them and still have a legal sequence, do so.</p></li>
<li><p>Pick two runs of the same colour. Redistribute the balls in them randomly among legal possibilities (so if the maximum number of balls in a single run was 3, and you had 5 balls total in the two chosen runs, the first one is equally likely to get 2 or 3 balls; if there were 3 balls total, the first is equally likely to get 1 or 2; if there were 4 balls total, 1, 2, and 3 are all equally likely).</p></li>
<li><p>Pick some colour $C_i$ at random. Consider the sequence $S&#39;$ of balls with all balls of colour $C_i$ removed. Now, pick at random two points in $S&#39;$ where adjacent balls of different colours touch. </p>

<p>a. If there are two runs of colour $C_i$ at these two points in the original sequence $S$, and neither is maximum length, move a ball from one to the other, with each direction chosen with probability &frac12;. </p>

<p>b. If there are two runs of colour $C_i$ at these two points in the original sequence $S$, but one is maximum length and the other is not, move a ball from the maximum length run to the shorter one with probability &frac12;.</p>

<p>c. If there is just one run of colour $C_i$ at one of these two points in $S$, with probability &frac12; move one ball from the run to the other point.  </p>

<p>d. If there is no run of colour $C_i$ at either of these points, or if there are runs of maximum length at both of these points, do nothing.</p></li>
</ol>

<p>If my analysis is right, this is a reversible Markov chain that eventually converges to a uniform distribution of legal sequences of coloured balls, so if you run this chain for long enough, you will get very close to this uniform distribution.</p>

<p>How can you tell when this has converged? I would suggest watching the entropy of this sequence, and stop when it stops increasing. How do you compute the entropy? There are two main terms in the entropy calculation: the distribution of run lengths, and the sequence of colors each run has. For the distribution of run lengths, assume that there are $n_{i,k}$ runs of colour $i$ with length $k$. The contribution of these to the entropy is
$$ \sum_i \ \log_2 \ { \sum_k n_{i,k} \choose n_{i,1}\ n_{i,2}\ \ldots \ n_{i,r} }, $$ 
where $r$ is the maximum permissible length of a run. Now, let us consider the contribution of the colour sequence to the entropy. Suppose there are $m_{i,j}$ places where a run of colour $i$ is immediately followed by one of colour $j$ (so $m_{i,i}=0$). The contribution of this to the entropy is 
$$ \sum_i \ \log_2\  { \sum_j m_{i,j} \choose  m_{i,1}\ m_{i,2}\ \ldots \ m_{i,c} }, $$ 
where $c$ is the number of colours.  </p>

<p>(In the interest of accuracy, let me note that we are leaving out a number of contributions to the entropy, including the colour of the first ball, but these are lower order terms which should be safe to neglect.)</p>

<p>UPDATE:</p>

<p>There should be ways of speeding this up. I believe that for steps c and d, you can use analysis to perform both these steps over all runs of one color at once. For steps a and b, this is equivalent to the question of finding a random sequence of colored balls with the constraint that no two balls of the same color touch. There should be some good way of doing the mixing for this problem. Then you just have to alternate a/b steps with c/d steps, where each step mixes over those two moves completely. I think this should converge quite fast, although I don't have any rigorous analysis for this Markov chain.</p>

CC BY-SA 3.0
2011-10-24T21:37:16.187Z
2011-10-27T14:18:43.737Z
2011-10-27T14:18:43.737Z
Peter Shor 
<p>As you said, it is not possible to ensure that every permutation is equally likely <strong>and</strong> ensure that the colors are evenly distributed, because one of the permutations has all the reds in a row.</p>

<p>A very elegant, but certainly not obvious, method to ensure the colors are evenly distributed is to leverage a low discrepancy sequence.</p>

<p>Suppose you have $N=400$ balls, numbered from $1$ to $N$, and a  seed value, $s$. </p>

<p>Ensure that all balls of the same color are consecutively numbered. 
That is, in your case, let the first 100 balls be red, the next 40 be yellow, the next 50 green, etc.</p>

<p>Then, allocate the $k^{\textrm{th}}$ ball the value, $x_k$ such that:
$$ x_k = (s+ k \phi) \; (\textrm{mod}\; 1),$$ 
where</p>

<ul>
<li>$\phi = \frac{1+\sqrt{5}}{2} = 1.61803399...$, the golden ratio </li>
<li>the $(\textrm{mod} \; 1)$ operator that takes the fractional part of argument</li>
<li>$s$ is any constant 'seed' value that you wish.</li>
</ul>

<p>That is, each of the $N$ balls will be allocated a value of $x_k$ which will always between 0 and 1.</p>

<p>Now simply order the balls, in ascending order according to their $x_k$ value.</p>

<p>For example, using the seed value of $s=0$, the balls will be ordered as follows:
$$ \{B,R,K,G,R,P,Y,K,B,R,P,G,K,R,B,Y,K,B,R,P,Y,K,B,R,P,G,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,P,Y,K,B,R,P,G,K,B,R,P,G,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,B,Y,K,B,R,P,Y,K,B,R,P,G,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,P,Y,K,B,R,P,G,K,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,B,Y,K,G,R,P,Y,K,B,R,P,G,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,B,Y,K,B,R,P,Y,K,B,R,P,G,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,B,Y,K,B,R,P,Y,K,B,R,P,G,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,P,Y,K,B,R,P,G,K,B,R,P,G,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,B,Y,K,B,R,P,Y,K,B,R,P,G,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,P,Y,K,B,R,P,G,K \} $$
(where "$B$"= Blue, and "$K$" = Black).</p>

<p>Finally, if you wish to take a different sample, simply select a different seed value, $s$.</p>

<p>Python code for this allocating the $x_k$ is as follows:</p>

<pre><code>n=400

phi = (1+pow(5,0.5))/2
x = np.zeros(n)                 
s = np.random.uniform(0,1)
for i in range(n):
    x = (s + phi*(i+1)) %1

print (s)
print (x)
</code></pre>

CC BY-SA 4.0
2018-07-14T03:32:31.667Z
2018-07-24T10:14:54.343Z
2018-07-24T10:14:54.343Z
Martin Roberts
<p>I have 400 balls, in which 100 are red, 40 are yellow, 50 are green, 60 are blue, 70 are purple, 80 are black. (balls of the same colour are identical)</p>

<p>i need an efficient shuffling algorithm, so that after shuffling, balls are in a list, and</p>

<p>Any consecutive 3 balls are not of the same colour. e.g, i can not have "red, red, red, yellow...."</p>

<p>And, all permutation are "equally" likely to occur. (well, if the trade-off of efficiency vs. unbiasedness is good enough, i don't mind more efficiency than unbiasedness).</p>

<p>i tried to adapt Fisher-Yates-Knuth, but the outcome is not ideal.</p>

<p>Why Fisher-Yates not good enough? As FY adopts Monte Carlo inverse transformation. And the output distribution treats the same colour balls differently, i.e. it would generate biased result for my needs.</p>

<p>And, the Naive thinking would be to filter out / backtracking all bad permutations from the whole space. When the restriction is very strong, say, if we have only 300 balls and 100 of which are red, then there will be too many back tracking/failures before getting an appropriate permutation.</p>

<p>So, ultimately, I would wish to be able to iterate through all good permutations. However, because the number of valid permutations is too large, i can only randomly sample some of them. I wanna the statistical feature of the "some" of them resemble the population as much as possible.</p>

CC BY-SA 3.0
2011-10-23T09:53:33.703Z
Have you tried to adapt the answers from [the other question](http://cstheory.stackexchange.com/questions/8677/how-to-shuffle-cards-with-restrictions) you asked? Both questions look very similar :).
Gopi
CC BY-SA 3.0
2011-10-23T10:06:50.243Z
@Gopi: yes, and i hope the answers for either question would bring inspiration to the other.
colinfang
CC BY-SA 3.0
2011-10-23T10:22:07.307Z
The simplest idea that comes to my mind is to start choosing randomly one ball from some color, where each color will be chosen with a probability based on the number of balls left with that color, with the restriction that if the last 2 balls had the same color, you cannot choose it at the current iteration. The efficiency shouldn't be bad and I can't see any bias in it (which doesn't mean that there is none; maybe I miss something).
George
CC BY-SA 3.0
2011-10-23T13:17:31.027Z
@George B.: we went through why this process does have bias on the [other related question.](http://cstheory.stackexchange.com/questions/8677/how-to-shuffle-cards-with-restrictions) As David Eppstein explains in his answer to that question, there's a dynamic programming algorithm that takes $\theta(n^k)$ time, where $k$ is the number of colors. Something more efficient would be nice--even $\theta(n^{k/2})$.
Peter Shor 
CC BY-SA 3.0
2011-10-23T14:26:11.487Z
Oh, I didn't check the other question. You are totally right, it seems that it does introduce bias, even if its not obvious at first sight. I can't think of some better deterministic approach. I'm thinking that a MCMC approach could possibly help: start with a satisfying assignment (should be trivial to construct) and start swapping elements (if the constraints are satisfied). I don't know if the MC is irreducible (does it depend on the constraints?) and if the stationary distribution is uniform (intuitively it seems so, even though the mixing time may be high, depending on the constraints).
George
CC BY-SA 3.0
2011-10-23T14:26:24.653Z
However, even if this approach (or something similar) works, the cost may be higher than the method David Eppstein suggested in the other question. (There may be some flaw in my thoughts, I just want to point to this approach)
George
CC BY-SA 3.0
2011-10-24T12:40:14.460Z
@GeorgeB. Even if David Eppstein's approach is cheaper, I'd be interested in how to solve this problem with a MCMC approach.
Peter Shor 
CC BY-SA 3.0
2011-10-24T14:26:46.857Z
@PeterShor I'm also interested in this. Once I have some time I will think about it (I'm pretty busy at the moment).
George
CC BY-SA 3.0
2011-10-24T16:10:08.327Z
@PeterShor: Me too. Though I didn't do very well in MCMC in my degree, i will try to repick that subject.
colinfang
<ds.algorithms><randomized-algorithms><permutations>
How to shuffle colour balls?
colinfang
# How to shuffle colour balls?

(Asked by: username_0 on 2011-10-23)

I have 400 balls, in which 100 are red, 40 are yellow, 50 are green, 60 are blue, 70 are purple, 80 are black. (balls of the same colour are identical)

i need an efficient shuffling algorithm, so that after shuffling, balls are in a list, and

Any consecutive 3 balls are not of the same colour. e.g, i can not have "red, red, red, yellow...."

And, all permutation are "equally" likely to occur. (well, if the trade-off of efficiency vs. unbiasedness is good enough, i don't mind more efficiency than unbiasedness).

i tried to adapt Fisher-Yates-Knuth, but the outcome is not ideal.

Why Fisher-Yates not good enough? As FY adopts Monte Carlo inverse transformation. And the output distribution treats the same colour balls differently, i.e. it would generate biased result for my needs.

And, the Naive thinking would be to filter out / backtracking all bad permutations from the whole space. When the restriction is very strong, say, if we have only 300 balls and 100 of which are red, then there will be too many back tracking/failures before getting an appropriate permutation.

So, ultimately, I would wish to be able to iterate through all good permutations. However, because the number of valid permutations is too large, i can only randomly sample some of them. I wanna the statistical feature of the "some" of them resemble the population as much as possible.

**Tags:** ds.algorithms, randomized-algorithms, permutations

> **Comments:**
> **username_1**: Have you tried to adapt the answers from [the other question](http://cstheory.stackexchange.com/questions/8677/how-to-shuffle-cards-with-restrictions) you asked? Both questions look very similar :).  
> **username_0**: @username_1: yes, and i hope the answers for either question would bring inspiration to the other.  
> **username_2**: The simplest idea that comes to my mind is to start choosing randomly one ball from some color, where each color will be chosen with a probability based on the number of balls left with that color, with the restriction that if the last 2 balls had the same color, you cannot choose it at the current iteration. The efficiency shouldn't be bad and I can't see any bias in it (which doesn't mean that there is none; maybe I miss something).  
> **username_3**: @username_2 B.: we went through why this process does have bias on the [other related question.](http://cstheory.stackexchange.com/questions/8677/how-to-shuffle-cards-with-restrictions) As David Eppstein explains in his answer to that question, there's a dynamic programming algorithm that takes $\theta(n^k)$ time, where $k$ is the number of colors. Something more efficient would be nice--even $\theta(n^{k/2})$.  
> **username_2**: Oh, I didn't check the other question. You are totally right, it seems that it does introduce bias, even if its not obvious at first sight. I can't think of some better deterministic approach. I'm thinking that a MCMC approach could possibly help: start with a satisfying assignment (should be trivial to construct) and start swapping elements (if the constraints are satisfied). I don't know if the MC is irreducible (does it depend on the constraints?) and if the stationary distribution is uniform (intuitively it seems so, even though the mixing time may be high, depending on the constraints).  
> **username_2**: However, even if this approach (or something similar) works, the cost may be higher than the method David Eppstein suggested in the other question. (There may be some flaw in my thoughts, I just want to point to this approach)  
> **username_3**: @username_2B. Even if David Eppstein's approach is cheaper, I'd be interested in how to solve this problem with a MCMC approach.  
> **username_2**: @PeterShor I'm also interested in this. Once I have some time I will think about it (I'm pretty busy at the moment).  
> **username_0**: @PeterShor: Me too. Though I didn't do very well in MCMC in my degree, i will try to repick that subject.  

---

## 2 answers

### Answer by username_3 on 2011-10-24. Score: 7

What you need for a Markov chain to converge to an equal distribution over all possible sequences of balls is that it is reversible: the probability of moving from sequence $i$ to sequence $j$ is the same as moving in the opposite direction. I thus propose that you use the following moves (with some fixed probability distribution to choose which type of move to make) to perform a Markov chain on all possible sequences. In the following, a "run" is a maximum-length consecutive subsequence of balls of the same colour. This Markov chain relies on there being at least three colours.

1. Pick two runs at random. If you can exchange them and still have a legal sequence, do so.
2. Pick two adjacent runs. If you can exchange them and still have a legal sequence, do so.
3. Pick two runs of the same colour. Redistribute the balls in them randomly among legal possibilities (so if the maximum number of balls in a single run was 3, and you had 5 balls total in the two chosen runs, the first one is equally likely to get 2 or 3 balls; if there were 3 balls total, the first is equally likely to get 1 or 2; if there were 4 balls total, 1, 2, and 3 are all equally likely).
4. Pick some colour $C\_i$ at random. Consider the sequence $S'$ of balls with all balls of colour $C\_i$ removed. Now, pick at random two points in $S'$ where adjacent balls of different colours touch.

   a. If there are two runs of colour $C\_i$ at these two points in the original sequence $S$, and neither is maximum length, move a ball from one to the other, with each direction chosen with probability ½.

   b. If there are two runs of colour $C\_i$ at these two points in the original sequence $S$, but one is maximum length and the other is not, move a ball from the maximum length run to the shorter one with probability ½.

   c. If there is just one run of colour $C\_i$ at one of these two points in $S$, with probability ½ move one ball from the run to the other point.

   d. If there is no run of colour $C\_i$ at either of these points, or if there are runs of maximum length at both of these points, do nothing.

If my analysis is right, this is a reversible Markov chain that eventually converges to a uniform distribution of legal sequences of coloured balls, so if you run this chain for long enough, you will get very close to this uniform distribution.

How can you tell when this has converged? I would suggest watching the entropy of this sequence, and stop when it stops increasing. How do you compute the entropy? There are two main terms in the entropy calculation: the distribution of run lengths, and the sequence of colors each run has. For the distribution of run lengths, assume that there are $n\_{i,k}$ runs of colour $i$ with length $k$. The contribution of these to the entropy is
$$ \sum\_i \ \log\_2 \ { \sum\_k n\_{i,k} \choose n\_{i,1}\ n\_{i,2}\ \ldots \ n\_{i,r} }, $$
where $r$ is the maximum permissible length of a run. Now, let us consider the contribution of the colour sequence to the entropy. Suppose there are $m\_{i,j}$ places where a run of colour $i$ is immediately followed by one of colour $j$ (so $m\_{i,i}=0$). The contribution of this to the entropy is
$$ \sum\_i \ \log\_2\ { \sum\_j m\_{i,j} \choose m\_{i,1}\ m\_{i,2}\ \ldots \ m\_{i,c} }, $$
where $c$ is the number of colours.

(In the interest of accuracy, let me note that we are leaving out a number of contributions to the entropy, including the colour of the first ball, but these are lower order terms which should be safe to neglect.)

UPDATE:

There should be ways of speeding this up. I believe that for steps c and d, you can use analysis to perform both these steps over all runs of one color at once. For steps a and b, this is equivalent to the question of finding a random sequence of colored balls with the constraint that no two balls of the same color touch. There should be some good way of doing the mixing for this problem. Then you just have to alternate a/b steps with c/d steps, where each step mixes over those two moves completely. I think this should converge quite fast, although I don't have any rigorous analysis for this Markov chain.


---

### Answer by username_4 on 2018-07-14. Score: 0

As you said, it is not possible to ensure that every permutation is equally likely and ensure that the colors are evenly distributed, because one of the permutations has all the reds in a row.
A very elegant, but certainly not obvious, method to ensure the colors are evenly distributed is to leverage a low discrepancy sequence.
Suppose you have $N=400$ balls, numbered from $1$ to $N$, and a  seed value, $s$. 
Ensure that all balls of the same color are consecutively numbered. 
That is, in your case, let the first 100 balls be red, the next 40 be yellow, the next 50 green, etc.
Then, allocate the $k^{\textrm{th}}$ ball the value, $x_k$ such that:
$$ x_k = (s+ k \phi) \; (\textrm{mod}\; 1),$$ 
where

$\phi = \frac{1+\sqrt{5}}{2} = 1.61803399...$, the golden ratio 
the $(\textrm{mod} \; 1)$ operator that takes the fractional part of argument
$s$ is any constant 'seed' value that you wish.

That is, each of the $N$ balls will be allocated a value of $x_k$ which will always between 0 and 1.
Now simply order the balls, in ascending order according to their $x_k$ value.
For example, using the seed value of $s=0$, the balls will be ordered as follows:
$$ \{B,R,K,G,R,P,Y,K,B,R,P,G,K,R,B,Y,K,B,R,P,Y,K,B,R,P,G,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,P,Y,K,B,R,P,G,K,B,R,P,G,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,B,Y,K,B,R,P,Y,K,B,R,P,G,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,P,Y,K,B,R,P,G,K,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,B,Y,K,G,R,P,Y,K,B,R,P,G,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,B,Y,K,B,R,P,Y,K,B,R,P,G,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,B,Y,K,B,R,P,Y,K,B,R,P,G,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,P,Y,K,B,R,P,G,K,B,R,P,G,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,B,Y,K,B,R,P,Y,K,B,R,P,G,R,P,Y,K,B,R,P,G,K,R,B,R,K,G,R,P,Y,K,B,R,P,G,K,R,P,Y,K,B,R,P,G,K \} $$
(where "$B$"= Blue, and "$K$" = Black).
Finally, if you wish to take a different sample, simply select a different seed value, $s$.
Python code for this allocating the $x_k$ is as follows:
n=400

phi = (1+pow(5,0.5))/2
x = np.zeros(n)                 
s = np.random.uniform(0,1)
for i in range(n):
    x = (s + phi*(i+1)) %1

print (s)
print (x)


---

