<p>(This is probably a comment but I cannot comment)</p>
<p>This seems to be 1-dimensional <span class="math-container">$k$</span>-means clustering for <span class="math-container">$k=2$</span> and <span class="math-container">$k=4$</span>. Here is a reference that gives a <span class="math-container">$O(kn + n \log n)$</span> algorithm for 1-d <span class="math-container">$k$</span>-means clustering: Grønlund, Larsen, Mathiasen, Nielsen, Schneider, Song, <a href="https://arxiv.org/abs/1701.07204" rel="noreferrer">Fast Exact k-Means, k-Medians and Bregman Divergence Clustering in 1D</a>. It seems similar to the algorithm given in the comments.</p>

CC BY-SA 4.0
2020-12-18T10:49:16.667Z
2020-12-18T11:51:07.493Z
2020-12-18T11:51:07.493Z
xmq
<p>Let <span class="math-container">$d\in\mathbb N$</span> and denote <span class="math-container">$V=\bigcup_{a,b\in\mathbb R}\{a,b\}^d$</span>, the set of all vectors with two distinct values.</p>
<p>Given a vector <span class="math-container">$x\in \mathbb R^d$</span>, I want to compute some <span class="math-container">$v^*\in V$</span> such that <span class="math-container">$\left\lVert x-v^*\right\rVert_2\le \left\lVert x-v\right\rVert_2$</span> for any <span class="math-container">$v\in V$</span>.</p>
<blockquote>
<p><strong>What would be the fastest algorithm to compute <span class="math-container">$v^*$</span>?</strong></p>
</blockquote>
<hr />
<p>It seems possible to show that the problem is equivalent to finding a threshold <span class="math-container">$T\in\mathbb R$</span> such that <span class="math-container">$v^*_i=\begin{cases}a &amp; \mbox{if $x_i&lt;T$}\\b &amp; \mbox{otherwise}\end{cases}$</span>, where <span class="math-container">$a$</span> is the mean of all entries smaller than <span class="math-container">$T$</span> and <span class="math-container">$b$</span> is the mean of all entries larger than or equal to <span class="math-container">$T$</span>.</p>
<p>This suggests a <span class="math-container">$O(d^2)$</span> time algorithm. Can we find the optimal <span class="math-container">$T$</span> faster?</p>
<hr />
<p>I am also interested in an extension of the problem where we look at <span class="math-container">$V_4=\bigcup_{a,b,c,z\in\mathbb R}\{a,b,c,z\}^d$</span> instead of <span class="math-container">$V$</span>. How fast can we find <span class="math-container">$v^*$</span> in this setting?</p>

CC BY-SA 4.0
2020-12-17T20:28:27.850Z
Assuming the unit-cost RAM model, the algorithm you outline in the question does not require time $O(d^2)$, it can be implemented using $O(d\log d)$ operations ($O(d\log d)$ comparisons and $O(d)$ additions and multiplications). Sort the vector, and the for each $i\le d$, try $a={}$ the mean of the first $i$ entries, and $b$ the mean of the remaining entries. Now, the mean $\bar v$ of $v_1,\dots,v_i$ and the squared distance $\sum_{j\le i}(\bar v-v_j)^2$ can be computed using $O(1)$ operations if you know $\sum_{j\le i}v_j$ and $\sum_{j\le i}v_j^2$, and these two quantities can be updated ...
Emil Jeřábek
CC BY-SA 4.0
2020-12-17T20:30:07.580Z
... with $O(1)$ operations while adding or removing one element from the list.
Emil Jeřábek
CC BY-SA 4.0
2020-12-17T20:45:52.350Z
@EmilJeřábek - thanks, that's a cool improvement. Any thoughts on whether a linear algorithm is possible here?
M A
CC BY-SA 4.0
2020-12-18T07:37:12.087Z
@SarielHar-Peled But the hypercube is not fixed in advance, this is what you need to optimize. So what is “the nearest neighbour in the first coordinate”? And how do you make sure each coordinate separately gives the same values of $a$ and $b$? If $a$ and $b$ is fixed, the problem is indeed trivial: you just put $v_i^*=a$ if $x_i<(a+b)/2$, and $v_i^*=b$ otherwise.
Emil Jeřábek
CC BY-SA 4.0
2020-12-19T16:59:58.177Z
Yeh. My brain is dead.
Sariel Har-Peled
<ds.algorithms><cg.comp-geom>
Minimizing $L_2$ norm of a vector with two distinct entries
M A
# Minimizing $L_2$ norm of a vector with two distinct entries

(Asked by: username_0 on 2020-12-17)

Let $d\in\mathbb N$ and denote $V=\bigcup\_{a,b\in\mathbb R}\{a,b\}^d$, the set of all vectors with two distinct values.

Given a vector $x\in \mathbb R^d$, I want to compute some $v^\*\in V$ such that $\left\lVert x-v^\*\right\rVert\_2\le \left\lVert x-v\right\rVert\_2$ for any $v\in V$.

> **What would be the fastest algorithm to compute $v^\*$?**

---

It seems possible to show that the problem is equivalent to finding a threshold $T\in\mathbb R$ such that $v^\*\_i=\begin{cases}a & \mbox{if $x\_i, where $a$ is the mean of all entries smaller than $T$ and $b$ is the mean of all entries larger than or equal to $T$.

This suggests a $O(d^2)$ time algorithm. Can we find the optimal $T$ faster?

---

I am also interested in an extension of the problem where we look at $V\_4=\bigcup\_{a,b,c,z\in\mathbb R}\{a,b,c,z\}^d$ instead of $V$. How fast can we find $v^\*$ in this setting?

**Tags:** ds.algorithms, cg.comp-geom

> **Comments:**
> **username_1**: Assuming the unit-cost RAM model, the algorithm you outline in the question does not require time $O(d^2)$, it can be implemented using $O(d\log d)$ operations ($O(d\log d)$ comparisons and $O(d)$ additions and multiplications). Sort the vector, and the for each $i\le d$, try $a={}$ the mean of the first $i$ entries, and $b$ the mean of the remaining entries. Now, the mean $\bar v$ of $v_1,\dots,v_i$ and the squared distance $\sum_{j\le i}(\bar v-v_j)^2$ can be computed using $O(1)$ operations if you know $\sum_{j\le i}v_j$ and $\sum_{j\le i}v_j^2$, and these two quantities can be updated ...  
> **username_1**: ... with $O(1)$ operations while adding or removing one element from the list.  
> **username_0**: @EmilJeřábek - thanks, that's a cool improvement. Any thoughts on whether a linear algorithm is possible here?  
> **username_1**: @SarielHar-Peled But the hypercube is not fixed in advance, this is what you need to optimize. So what is “the nearest neighbour in the first coordinate”? And how do you make sure each coordinate separately gives the same values of $a$ and $b$? If $a$ and $b$ is fixed, the problem is indeed trivial: you just put $v_i^*=a$ if $x_i<(a+b)/2$, and $v_i^*=b$ otherwise.  
> **username_2**: Yeh. My brain is dead.  

---

## 1 answers

### Answer by username_3 on 2020-12-18. Score: 6 | Accepted answer

(This is probably a comment but I cannot comment)

This seems to be 1-dimensional $k$-means clustering for $k=2$ and $k=4$. Here is a reference that gives a $O(kn + n \log n)$ algorithm for 1-d $k$-means clustering: Grønlund, Larsen, Mathiasen, Nielsen, Schneider, Song, [Fast Exact k-Means, k-Medians and Bregman Divergence Clustering in 1D](https://arxiv.org/abs/1701.07204). It seems similar to the algorithm given in the comments.


---

