<p>The data-structure is encoding a set which means it is not encoding an order.</p>

<p>If you draw numbers uniformly at random from some universe then sort the result (thus destroying the order information), the differences between the numbers follow an exponential distribution. The GCS uses a golomb coder to efficiently store these differences. The sort method used is irrelevant so long as its consistent with the differencing method used.</p>

<p>There is nothing probabilistic about the set encoding itself. But for BIP158 the set being encoded is not TXIDs but short hashes of relevant outputs. Because the hashes are short they can have collisions, making the result probabilistic.</p>

CC BY-SA 4.0
2018-10-28T20:38:31.877Z
Can you elaborate on short hashes for "relevant" outputs? This maybe warrant a separate question -- more related to BIP158 -- but what are "short hashes of relevant outputs"? Isn't the entire block encoded into a golomb coded set -- and then relayed to your client peer? How does the server compute "relevant outputs" when it doesn't understand what that is?
Chris Stewart
CC BY-SA 4.0
2018-10-28T21:28:58.027Z
The GCS's in BIP158 encode for each block which scripts are used in it (whether the block contains at least one output a given script, or whether it spends at least one output assigned to that script). It does that by encoding short hashes of those scripts. A client will download the whole filter for the block, and try to match it against the short hashes of all scripts it is interested in. If it finds a match, it downloads the full block, otherwise it can skip the block and continue with the next one. The filters are generally only a few kilobytes in size.
Pieter Wuille
CC BY-SA 4.0
2018-10-29T08:37:56.747Z
Pieter's answer covers it, by saying relevant I was thinking about  empty/op_return not being included.
G. Maxwell
CC BY-SA 4.0
2018-10-28T17:41:58.473Z
2018-10-28T17:41:58.473Z
G. Maxwell
<p>From my understanding Golomb coded sets are a probabilistic data structure that encodes the <em>deltas</em> of an order set of elements.</p>

<p>With things like txids, which are <em>evenly distributed</em>, Golomb coded sets are an efficient way to encode things inside of a block. </p>

<p>What is the ordering used to determine how txids are encoded in the set? It seems we need a <code>base</code> txid, and then compute deltas from that original txid? Is it just as simple as the lowest big endian numeric interpretation of a txid in a block? </p>

<compact-client-side-block-filters>
How do Golomb Coded Sets work?
Chris Stewart
# How do Golomb Coded Sets work?

(Asked by: username_0 on 2018-10-28)

From my understanding Golomb coded sets are a probabilistic data structure that encodes the *deltas* of an order set of elements.

With things like txids, which are *evenly distributed*, Golomb coded sets are an efficient way to encode things inside of a block.

What is the ordering used to determine how txids are encoded in the set? It seems we need a `base` txid, and then compute deltas from that original txid? Is it just as simple as the lowest big endian numeric interpretation of a txid in a block?

**Tags:** compact-client-side-block-filters


---

## 1 answers

### Answer by username_1 on 2018-10-28. Score: 4 | Accepted answer

The data-structure is encoding a set which means it is not encoding an order.

If you draw numbers uniformly at random from some universe then sort the result (thus destroying the order information), the differences between the numbers follow an exponential distribution. The GCS uses a golomb coder to efficiently store these differences. The sort method used is irrelevant so long as its consistent with the differencing method used.

There is nothing probabilistic about the set encoding itself. But for BIP158 the set being encoded is not TXIDs but short hashes of relevant outputs. Because the hashes are short they can have collisions, making the result probabilistic.

> **Comments:**
> **username_0**: Can you elaborate on short hashes for "relevant" outputs? This maybe warrant a separate question -- more related to BIP158 -- but what are "short hashes of relevant outputs"? Isn't the entire block encoded into a golomb coded set -- and then relayed to your client peer? How does the server compute "relevant outputs" when it doesn't understand what that is?  
> **username_2**: The GCS's in BIP158 encode for each block which scripts are used in it (whether the block contains at least one output a given script, or whether it spends at least one output assigned to that script). It does that by encoding short hashes of those scripts. A client will download the whole filter for the block, and try to match it against the short hashes of all scripts it is interested in. If it finds a match, it downloads the full block, otherwise it can skip the block and continue with the next one. The filters are generally only a few kilobytes in size.  
> **username_1**: username_2's answer covers it, by saying relevant I was thinking about  empty/op_return not being included.  

---

