<p>While continuity in a strong sense is probably too much to ask for, locality sensitive hashing (LSH) achieves a very similar goal: given a set of items $x_1, \ldots, x_n$ which belong to some metric space with metric $d$, an LSH family of hash functions (parametrized by $D, c, p, q$) maps $x_i$ and $x_j$ into the same bucket with probability $\geq p$ if $d(x_i, x_j) \leq D$ and maps $x_i$ and $x_j$ into different buckets with probablity $\geq q$ if $d(x_i, x_j) &gt; cD$. Of course there is tension in the requirements, so one looks for tradeoffs. Look at the <a href="http://en.wikipedia.org/wiki/Locality-sensitive_hashing" rel="nofollow">wiki page</a> and Alex Andoni's <a href="http://www.mit.edu/~andoni/LSH/" rel="nofollow">webpage</a>, especially the CACM survey.</p>

<p>For context, LSH is a tool to solve the approximate nearest neighbor problem in high dimensional metric spaces. There exist solutions for various metrics, like hamming distance and $\ell_p$ norms, see the CACM survey. The approximate nearest neighbor problem is an important problem in computational geometry, and the high-dimensional version comes up in a variety of applications, for example in computer vision and information retrieval. </p>

<p>The way LSH is used to solve approximate near neighbor search is a fairly natural two-level hashing scheme. As you note, there is tension between the "continuity" property and bounding the number of collisions. For that reason, usually the gap between $p$ and $q$ (as I defined them above) is pretty small. To overcome this we construct a new hash function by choosing several functions from an LSH family and taking their concatenation. The resulting universe is too big, so it is reduced using traditional hashing. Then we build several hash tables using these hash functions. A query is answered by hashing the query point and inspecting the other points in the same bucket(s) to determine if they are close enough (this is the linear search part). Parameters can be chosen so that a query is answered in sublinear time.</p>

CC BY-SA 3.0
2012-08-05T17:27:16.903Z
As far as I know, LSH is a recent application most people have never heard of. Furthermore, LSH is not a type of hashing function, but rather an algorithm that computes indices using standard uniform hash functions. It is an outgrowth of Broder's original [MinHash](http://dx.doi.org/10.1109%2FSEQUEN.1997.666900), itself the spiritual successor of [Flajolet and Martin's observation](http://algo.inria.fr/flajolet/Publications/FlMa85.pdf) that cardinality estimation schemes based on projection are very good at estimating unions of stream (see "Applications to Distributed Computing" p.204).
Jérémie
CC BY-SA 3.0
2012-08-05T18:39:09.217Z
@Jérémie Of course LSH is a type of hash function. **By definition** a family of hash functions is locality sensitive if collision probability is bounded from above and below as described in my answer (see CACM survey section 2.3). If this is not a type of hash function, then pairwise independence isn't either! You are talking about a particular construction of an LSH family, and not all constructions fit your description, for example the constructions based on $p$-stable distributions from Datar et al., SCG 2004
Sasho Nikolov
CC BY-SA 3.0
2012-08-05T18:46:10.257Z
I find your claims that LSH is obscure puzzling too, but that's not a constructive discussion. In any case, notice that the application is near-neighbor search in high dimension, and that's what I am saying is far from unheard of (it's one of the most prominent problems in computational geometry). But, since this comment upsets you, and I admit I was too curt, I will edit that part of my answer. I'd appreciate it if you explain the downvote, or at least what you understand by type of hashing function.
Sasho Nikolov
CC BY-SA 3.0
2012-08-05T19:56:59.987Z
I think that in the vast context of hash functions and their applications, LSH is a fascinating but relatively minor topic. For the main part of their history, and for the bulk of the problems that they solve, we ask that hash function separate data -- in other words I believe that "continuity" is by far the exception not the rule. In particular, luis.espinal seems to be focused on the application of hash functions to hash tables, and I feel like this is not something you have factored in when responding (although everything you say is perfectly true).
Jérémie
CC BY-SA 3.0
2012-08-05T19:59:18.410Z
Another way of seeing it is: you are answering "Is continuity possible?" when the question was "Why the heck would we want continuity in hash functions for hash tables?" But perhaps I am wrong? :-/
Jérémie
CC BY-SA 3.0
2012-08-06T03:07:33.563Z
@Jérémie i believe i am answering "why would you want continuity"; a hash table to me is underspecified as a data structure. nevertheless, the common way to use LSH in a data structure is a sort of hash table. i have added more context.
Sasho Nikolov
CC BY-SA 3.0
2012-08-06T04:01:49.793Z
Thank you both. Jeremy, yep, my original question was pretty much like that, why the heck have continuity in a hash function for a hash table :)
luis.espinal
CC BY-SA 3.0
2012-08-05T07:58:50.043Z
2012-08-06T03:10:02.080Z
2012-08-06T03:10:02.080Z
Sasho Nikolov
<p>Reading available literature (yep, including wikipedia), I see that hash functions should have (<a href="http://en.wikipedia.org/wiki/Hash_function#Continuity" rel="nofollow">continuity</a>) and map values that differ very little to similar/same hash codes, in particular for (<a href="http://en.wikipedia.org/wiki/Hash_function#Continuity" rel="nofollow">hash tables that use linear search</a>) ... if that makes any sense. Supposedly, this would help linear search benefits from locality of reference.</p>

<p>OTH, a hash function should be <a href="http://en.wikipedia.org/wiki/Hash_function#Uniformity" rel="nofollow">uniform</a>, in the sense that hash values are uniformly distributed. Unless I'm misunderstanding something, it looks to me as if these two properties are in conflict. </p>

<p>Let's say we have a sequence $S$ of input values $(s_1, ..., s_n)$ such that the delta between any two value pairs is small and similar across the set. </p>

<p>That is</p>

<p>$$\forall s_i, s_j \in S\,\,|\,\varepsilon_{min} &lt; \delta(s_i, s_j) &lt; \varepsilon_{max}$$ </p>

<p>and with the differences between any two values ($\delta(s_i,s_j)$) being uniformly distributed. In such a case the values in the sequence have a uniform distribution. </p>

<p>Let's suppose we aim to store these values in a hash table with open addressing (linear probing) and a hash function $h$. Obviously, we assume that the hash table is large enough to accommodate all input values (and to be below an initial load factor.) </p>

<p>If the hash function $h$ exhibits continuity, then it will map all the values in $S$ very close to one another (if not, to the same hash code) ergo increasing collisions. That is, it would exhibit <em>clustering</em>.</p>

<p>Wouldn't that then, deteriorate the search performance (via liner probing) as a result of the resulting collisions. If continuity is desirable when using linear probing, then how does one cope with the collisions when a large part of the values are close to one another?</p>

<h2>--- EDIT ---</h2>

<p>Sorry, I left some things out. $\delta(s_i,s_j)$ represents the number of symbols by which the strings $s_i$ and $s_j$ differ (or the number of changes that need to occur to convert $s_i$ into $s_j$.)</p>

<p>For a better example, let $\Sigma = {\sigma_1, \sigma_2, ... , \sigma_n}$ be an alphabet, and $L(i, n)$ be a language over $\Sigma$ such that</p>

<p>$$
L(i,n)\,\triangleq \{s| (s=\sigma_i^*\sigma_k\sigma_i)\wedge (|s| = n) \wedge (\{\sigma_i,\sigma_k\} \subset \Sigma)\}
$$</p>

<p>So, if we were to consider input strings, all of length $n$ such as 
$$aaa..., baaa..., abaa..., ....aaab, caaa..., acaa$$</p>

<p>that only differ on one character, the value and position of which is uniformly distributed among the input strings, then a hash function $h$ that possess continuity will cluster them all on top or near the same hash value.</p>

<h2>-- ADDITIONAL EDIT --</h2>

<p>The only place I've read so far that mentions continuity as a desirable property has been in the Wikipedia article for hash functions. I've consulted all the other books I have on Algorithms (CLRS, Sedgewick's, Berman and Paul's and others), and I've not found any such assertion on hash functions.</p>

<p>So it seems to me that continuity is not a desirable trait despite what the Wiki entry says. Or am I missing something????</p>

CC BY-SA 3.0
2012-08-04T05:59:52.083Z
what is $\delta$?
Sasho Nikolov
CC BY-SA 3.0
2012-08-04T12:42:22.477Z
I fail to see why the hash function used in hash tables with linear probing should exhibit continuity at all.  Continuity is required if you want to find a value in a hash table that is similar to a given value efficiently, but this is unrelated to linear probing.
Tsuyoshi Ito
CC BY-SA 3.0
2012-08-04T17:54:34.993Z
@SashoNikolov - thanks for asking that. I forgot to mention what $\delta$ was for (in this context, it's the number of characters by which the two strings differ.) I've updated my question to reflect that.
luis.espinal
CC BY-SA 3.0
2012-08-04T18:00:25.453Z
@TsuyoshiIto - I see. So we could say that continuity is required if we are searching for a value (in a hash table) that is very similar to another (an approximate match per say.)
luis.espinal
CC BY-SA 3.0
2012-08-04T18:02:48.600Z
@TsuyoshiIto - one thing I noticed is that the Wiki entry says "linear search" not "linear probing". My bad. This still does not make sense since one does not a linear search of a hashed value on a hash table (we simply do some open addressing scheme instead, linear probing, quadratic, double hashing, etc.) But I have to update my question to reflect the fact I misread "linear search" for "linear probing".
luis.espinal
CC BY-SA 3.0
2012-08-04T21:20:18.107Z
(1) Hmm, I agree that that sentence in the Wikipedia article still does not make sense.  (2) The choice of hash function depends on the application, and in your application, you want to balance between uniformity and continuity.  I do not know a good hash function on strings in that sense, but maybe someone who has experiences in this field knows much better.
Tsuyoshi Ito
CC BY-SA 3.0
2012-08-04T21:24:24.373Z
@luis.espinal: Continuity is something hash functions strive AGAINST in the general case and instead try to separate similar data (to prevent consecutive ids to be hashed in a clump, for instance). Only some very specific applications (which most people have never heard of) instead find use in having hash functions which map similar data to similar hash values.
Jérémie
<ds.data-structures><hash-function>
Continuity vs Uniformity when designing Hash functions
luis.espinal
# Continuity vs Uniformity when designing Hash functions

(Asked by: username_0 on 2012-08-04)

Reading available literature (yep, including wikipedia), I see that hash functions should have ([continuity](http://en.wikipedia.org/wiki/Hash_function#Continuity)) and map values that differ very little to similar/same hash codes, in particular for ([hash tables that use linear search](http://en.wikipedia.org/wiki/Hash_function#Continuity)) ... if that makes any sense. Supposedly, this would help linear search benefits from locality of reference.

OTH, a hash function should be [uniform](http://en.wikipedia.org/wiki/Hash_function#Uniformity), in the sense that hash values are uniformly distributed. Unless I'm misunderstanding something, it looks to me as if these two properties are in conflict.

Let's say we have a sequence $S$ of input values $(s\_1, ..., s\_n)$ such that the delta between any two value pairs is small and similar across the set.

That is

$$\forall s\_i, s\_j \in S\,\,|\,\varepsilon\_{min} < \delta(s\_i, s\_j) < \varepsilon\_{max}$$

and with the differences between any two values ($\delta(s\_i,s\_j)$) being uniformly distributed. In such a case the values in the sequence have a uniform distribution.

Let's suppose we aim to store these values in a hash table with open addressing (linear probing) and a hash function $h$. Obviously, we assume that the hash table is large enough to accommodate all input values (and to be below an initial load factor.)

If the hash function $h$ exhibits continuity, then it will map all the values in $S$ very close to one another (if not, to the same hash code) ergo increasing collisions. That is, it would exhibit *clustering*.

Wouldn't that then, deteriorate the search performance (via liner probing) as a result of the resulting collisions. If continuity is desirable when using linear probing, then how does one cope with the collisions when a large part of the values are close to one another?

## --- EDIT ---

Sorry, I left some things out. $\delta(s\_i,s\_j)$ represents the number of symbols by which the strings $s\_i$ and $s\_j$ differ (or the number of changes that need to occur to convert $s\_i$ into $s\_j$.)

For a better example, let $\Sigma = {\sigma\_1, \sigma\_2, ... , \sigma\_n}$ be an alphabet, and $L(i, n)$ be a language over $\Sigma$ such that

$$
L(i,n)\,\triangleq \{s| (s=\sigma\_i^\*\sigma\_k\sigma\_i)\wedge (|s| = n) \wedge (\{\sigma\_i,\sigma\_k\} \subset \Sigma)\}
$$

So, if we were to consider input strings, all of length $n$ such as
$$aaa..., baaa..., abaa..., ....aaab, caaa..., acaa$$

that only differ on one character, the value and position of which is uniformly distributed among the input strings, then a hash function $h$ that possess continuity will cluster them all on top or near the same hash value.

## -- ADDITIONAL EDIT --

The only place I've read so far that mentions continuity as a desirable property has been in the Wikipedia article for hash functions. I've consulted all the other books I have on Algorithms (CLRS, Sedgewick's, Berman and Paul's and others), and I've not found any such assertion on hash functions.

So it seems to me that continuity is not a desirable trait despite what the Wiki entry says. Or am I missing something????

**Tags:** ds.data-structures, hash-function
> **Comments:**
> **username_1**: what is $\delta$?  
> **username_2**: I fail to see why the hash function used in hash tables with linear probing should exhibit continuity at all.  Continuity is required if you want to find a value in a hash table that is similar to a given value efficiently, but this is unrelated to linear probing.  
> **username_0**: @SashoNikolov - thanks for asking that. I forgot to mention what $\delta$ was for (in this context, it's the number of characters by which the two strings differ.) I've updated my question to reflect that.  
> **username_0**: @TsuyoshiIto - I see. So we could say that continuity is required if we are searching for a value (in a hash table) that is very similar to another (an approximate match per say.)  
> **username_0**: @TsuyoshiIto - one thing I noticed is that the Wiki entry says "linear search" not "linear probing". My bad. This still does not make sense since one does not a linear search of a hashed value on a hash table (we simply do some open addressing scheme instead, linear probing, quadratic, double hashing, etc.) But I have to update my question to reflect the fact I misread "linear search" for "linear probing".  
> **username_2**: (1) Hmm, I agree that that sentence in the Wikipedia article still does not make sense.  (2) The choice of hash function depends on the application, and in your application, you want to balance between uniformity and continuity.  I do not know a good hash function on strings in that sense, but maybe someone who has experiences in this field knows much better.  
> **username_3**: @luis.espinal: Continuity is something hash functions strive AGAINST in the general case and instead try to separate similar data (to prevent consecutive ids to be hashed in a clump, for instance). Only some very specific applications (which most people have never heard of) instead find use in having hash functions which map similar data to similar hash values.  

---

## 1 answers

### Answer by username_1 on 2012-08-05. Score: 2 | Accepted answer

While continuity in a strong sense is probably too much to ask for, locality sensitive hashing (LSH) achieves a very similar goal: given a set of items $x\_1, \ldots, x\_n$ which belong to some metric space with metric $d$, an LSH family of hash functions (parametrized by $D, c, p, q$) maps $x\_i$ and $x\_j$ into the same bucket with probability $\geq p$ if $d(x\_i, x\_j) \leq D$ and maps $x\_i$ and $x\_j$ into different buckets with probablity $\geq q$ if $d(x\_i, x\_j) > cD$. Of course there is tension in the requirements, so one looks for tradeoffs. Look at the [wiki page](http://en.wikipedia.org/wiki/Locality-sensitive_hashing) and Alex Andoni's [webpage](http://www.mit.edu/~andoni/LSH/), especially the CACM survey.

For context, LSH is a tool to solve the approximate nearest neighbor problem in high dimensional metric spaces. There exist solutions for various metrics, like hamming distance and $\ell\_p$ norms, see the CACM survey. The approximate nearest neighbor problem is an important problem in computational geometry, and the high-dimensional version comes up in a variety of applications, for example in computer vision and information retrieval.

The way LSH is used to solve approximate near neighbor search is a fairly natural two-level hashing scheme. As you note, there is tension between the "continuity" property and bounding the number of collisions. For that reason, usually the gap between $p$ and $q$ (as I defined them above) is pretty small. To overcome this we construct a new hash function by choosing several functions from an LSH family and taking their concatenation. The resulting universe is too big, so it is reduced using traditional hashing. Then we build several hash tables using these hash functions. A query is answered by hashing the query point and inspecting the other points in the same bucket(s) to determine if they are close enough (this is the linear search part). Parameters can be chosen so that a query is answered in sublinear time.

> **Comments:**
> **username_3**: As far as I know, LSH is a recent application most people have never heard of. Furthermore, LSH is not a type of hashing function, but rather an algorithm that computes indices using standard uniform hash functions. It is an outgrowth of Broder's original [MinHash](http://dx.doi.org/10.1109%2FSEQUEN.1997.666900), itself the spiritual successor of [Flajolet and Martin's observation](http://algo.inria.fr/flajolet/Publications/FlMa85.pdf) that cardinality estimation schemes based on projection are very good at estimating unions of stream (see "Applications to Distributed Computing" p.204).  
> **username_1**: @Jérémie Of course LSH is a type of hash function. **By definition** a family of hash functions is locality sensitive if collision probability is bounded from above and below as described in my answer (see CACM survey section 2.3). If this is not a type of hash function, then pairwise independence isn't either! You are talking about a particular construction of an LSH family, and not all constructions fit your description, for example the constructions based on $p$-stable distributions from Datar et al., SCG 2004  
> **username_1**: I find your claims that LSH is obscure puzzling too, but that's not a constructive discussion. In any case, notice that the application is near-neighbor search in high dimension, and that's what I am saying is far from unheard of (it's one of the most prominent problems in computational geometry). But, since this comment upsets you, and I admit I was too curt, I will edit that part of my answer. I'd appreciate it if you explain the downvote, or at least what you understand by type of hashing function.  
> **username_3**: I think that in the vast context of hash functions and their applications, LSH is a fascinating but relatively minor topic. For the main part of their history, and for the bulk of the problems that they solve, we ask that hash function separate data -- in other words I believe that "continuity" is by far the exception not the rule. In particular, luis.espinal seems to be focused on the application of hash functions to hash tables, and I feel like this is not something you have factored in when responding (although everything you say is perfectly true).  
> **username_3**: Another way of seeing it is: you are answering "Is continuity possible?" when the question was "Why the heck would we want continuity in hash functions for hash tables?" But perhaps I am wrong? :-/  
> **username_1**: @Jérémie i believe i am answering "why would you want continuity"; a hash table to me is underspecified as a data structure. nevertheless, the common way to use LSH in a data structure is a sort of hash table. i have added more context.  
> **username_0**: Thank you both. Jeremy, yep, my original question was pretty much like that, why the heck have continuity in a hash function for a hash table :)  

---

