<p>I think this is a fair summary. I would hesitate to phase out unit tests, rather supplement them with property based tests. In a library I maintain called <a href="https://github.com/bitcoin-s/bitcoin-s" rel="nofollow noreferrer">bitcoin-s</a> we use them in tandem. </p>

<h3>Concepts</h3>

<p>There are two primary concepts in property based testing systems. </p>

<p>The first one is a <code>Generator</code>. The point of a generator is to generate values you want to run through your tests. Think of a generator as specify the boundaries of what random values you want to be generated for your test. </p>

<p>The second concept is the test itself. In english, property based tests say that </p>

<p>"Given a generator X, i want to generate random values from X and run them through my test case. My assertion should always hold true for the value generated from my generator". </p>

<h3>An example</h3>

<p>I'm going to pull an example from <a href="https://github.com/bitcoin/bitcoin/pull/14430" rel="nofollow noreferrer">this</a> PR i've had open for a long time now. If you want to see some more extensive tests I recommend taking a lookg at it. </p>

<p>The "property" we are going to assert is that </p>

<p><strong>"If we generate two random private keys in bitcoin, they should NOT be the same"</strong></p>

<p>If this property fails, we have an indicatin that something is probably wrong with our pseudo random number generator. </p>

<p><a href="https://github.com/bitcoin/bitcoin/blob/92c08f56bfa3abd98d6c7493303c8093337cd122/src/test/key_properties.cpp#L22" rel="nofollow noreferrer">Here</a> is what that test case looks like in C++ with bitcoin core:</p>



<pre><code>/** Check CKey uniqueness */
RC_BOOST_PROP(key_uniqueness, (const CKey&amp; key1, const CKey&amp; key2))
{
    RC_ASSERT(!(key1 == key2));
}
</code></pre>

<p>That test case can be run <code>N</code> number of times (by default 10 in the <a href="https://github.com/emil-e/rapidcheck" rel="nofollow noreferrer">rapidcheck library</a>. </p>

<p>The next logical question would be "where do <code>key1</code> and <code>key2</code> come from"? The answer: They come from the generator! The beauty of property based testing is leverage the framework to do the generation of values for you. </p>

<p><a href="https://github.com/bitcoin/bitcoin/blob/92c08f56bfa3abd98d6c7493303c8093337cd122/src/test/gen/crypto_gen.h#L20" rel="nofollow noreferrer">Here</a> is what the generator for <code>CKey</code> looks like: </p>



<pre><code>/** Generator for a new CKey */
template &lt;&gt;
struct Arbitrary&lt;CKey&gt; {
    static Gen&lt;CKey&gt; arbitrary()
    {
        return rc::gen::map&lt;int&gt;([](int x) {
            CKey key;
            key.MakeNewKey(true);
            return key;
        });
    };
};
</code></pre>

<p>This gives <a href="https://github.com/emil-e/rapidcheck" rel="nofollow noreferrer">rapidcheck</a> the ability to generate arbitrary valid <code>CKey</code>s to pass in as a parameter to the properties that we are asserting holds true.</p>

<p>You can see more complex <a href="https://github.com/bitcoin/bitcoin/blob/92c08f56bfa3abd98d6c7493303c8093337cd122/src/test/gen/script_gen.cpp" rel="nofollow noreferrer">generators</a> and <a href="https://github.com/bitcoin/bitcoin/blob/92c08f56bfa3abd98d6c7493303c8093337cd122/src/test/transaction_properties.cpp#L76" rel="nofollow noreferrer">assertions</a> related to Script in that PR too if you want to see non trivial generators/assertions.</p>

<h3>Summary</h3>

<p>It's really useful to have multiple runs of a test case with newly generated values every time. Typically in software testing, you write a unit test asserting that your code works as expected for one or two values. This is a really limited approach to testing as you want your code to work for a range of values. With generators, you can get the computer to generate a range of valid values that your property should assert true on.  </p>

<p>I think this is particularly useful for consensus systems like bitcoin where we need to reason about a wide range of values being fed as input into the consensus system. Property based testing is a great tool to use to offload some of that cognitive load when reasoning about corner cases.</p>

CC BY-SA 4.0
2019-11-19T13:10:17.987Z
2019-11-19T13:10:17.987Z
Chris Stewart
<p>Property based tests have been added to <a href="https://github.com/bitcoin/bitcoin/blob/master/doc/rapidcheck.md" rel="nofollow noreferrer">Bitcoin Core</a>.</p>

<p>This <a href="https://dev.to/jdsteinhauser/intro-to-property-based-testing-2cj8" rel="nofollow noreferrer">Dev article</a> describes property based tests as randomized unit tests that in some cases could replace unit tests.</p>

<p>"Generally speaking, property-based tests require only a few lines of code (like unit tests), but unlike unit tests they test a different set of inputs each time. Because of this, you end up covering more domain space with roughly the same amount of test code."</p>

<p>Is this a good summary? Could we phase out some of the unit tests in Bitcoin Core and replace them with property based tests? </p>

<bitcoin-core-development><testing>
Property based tests have been added to Bitcoin Core. What are they and could they replace some of the unit tests in Bitcoin Core?
Michael Folkson
# Property based tests have been added to Bitcoin Core. What are they and could they replace some of the unit tests in Bitcoin Core?

(Asked by: username_0 on 2019-11-19)

Property based tests have been added to [Bitcoin Core](https://github.com/bitcoin/bitcoin/blob/master/doc/rapidcheck.md).

This [Dev article](https://dev.to/jdsteinhauser/intro-to-property-based-testing-2cj8) describes property based tests as randomized unit tests that in some cases could replace unit tests.

"Generally speaking, property-based tests require only a few lines of code (like unit tests), but unlike unit tests they test a different set of inputs each time. Because of this, you end up covering more domain space with roughly the same amount of test code."

Is this a good summary? Could we phase out some of the unit tests in Bitcoin Core and replace them with property based tests?

**Tags:** bitcoin-core-development, testing


---

## 1 answers

### Answer by username_1 on 2019-11-19. Score: 1 | Accepted answer

I think this is a fair summary. I would hesitate to phase out unit tests, rather supplement them with property based tests. In a library I maintain called [bitcoin-s](https://github.com/bitcoin-s/bitcoin-s) we use them in tandem.

### Concepts

There are two primary concepts in property based testing systems.

The first one is a `Generator`. The point of a generator is to generate values you want to run through your tests. Think of a generator as specify the boundaries of what random values you want to be generated for your test.

The second concept is the test itself. In english, property based tests say that

"Given a generator X, i want to generate random values from X and run them through my test case. My assertion should always hold true for the value generated from my generator".

### An example

I'm going to pull an example from [this](https://github.com/bitcoin/bitcoin/pull/14430) PR i've had open for a long time now. If you want to see some more extensive tests I recommend taking a lookg at it.

The "property" we are going to assert is that

**"If we generate two random private keys in bitcoin, they should NOT be the same"**

If this property fails, we have an indicatin that something is probably wrong with our pseudo random number generator.

[Here](https://github.com/bitcoin/bitcoin/blob/92c08f56bfa3abd98d6c7493303c8093337cd122/src/test/key_properties.cpp#L22) is what that test case looks like in C++ with bitcoin core:

```
/** Check CKey uniqueness */
RC_BOOST_PROP(key_uniqueness, (const CKey& key1, const CKey& key2))
{
    RC_ASSERT(!(key1 == key2));
}

```

That test case can be run `N` number of times (by default 10 in the [rapidcheck library](https://github.com/emil-e/rapidcheck).

The next logical question would be "where do `key1` and `key2` come from"? The answer: They come from the generator! The beauty of property based testing is leverage the framework to do the generation of values for you.

[Here](https://github.com/bitcoin/bitcoin/blob/92c08f56bfa3abd98d6c7493303c8093337cd122/src/test/gen/crypto_gen.h#L20) is what the generator for `CKey` looks like:

```
/** Generator for a new CKey */
template <>
struct Arbitrary<CKey> {
    static Gen<CKey> arbitrary()
    {
        return rc::gen::map<int>([](int x) {
            CKey key;
            key.MakeNewKey(true);
            return key;
        });
    };
};

```

This gives [rapidcheck](https://github.com/emil-e/rapidcheck) the ability to generate arbitrary valid `CKey`s to pass in as a parameter to the properties that we are asserting holds true.

You can see more complex [generators](https://github.com/bitcoin/bitcoin/blob/92c08f56bfa3abd98d6c7493303c8093337cd122/src/test/gen/script_gen.cpp) and [assertions](https://github.com/bitcoin/bitcoin/blob/92c08f56bfa3abd98d6c7493303c8093337cd122/src/test/transaction_properties.cpp#L76) related to Script in that PR too if you want to see non trivial generators/assertions.

### Summary

It's really useful to have multiple runs of a test case with newly generated values every time. Typically in software testing, you write a unit test asserting that your code works as expected for one or two values. This is a really limited approach to testing as you want your code to work for a range of values. With generators, you can get the computer to generate a range of valid values that your property should assert true on.

I think this is particularly useful for consensus systems like bitcoin where we need to reason about a wide range of values being fed as input into the consensus system. Property based testing is a great tool to use to offload some of that cognitive load when reasoning about corner cases.


---

