<p>You could look at tools such as <a href="https://www.getpostman.com/" rel="nofollow noreferrer">Postman</a> which focuses on testing REST APIs with JavaScript - it has some nice features but you lose the use of Python.</p>

<p>Instead, I'd suggest looking at REST-related plugins for <a href="https://docs.pytest.org/en/latest/" rel="nofollow noreferrer">pytest</a>, a Python test framework that simplifies your test code, while still running tests written using <code>unittest</code>. </p>

<ul>
<li>For example, writing <a href="http://pythontesting.net/framework/pytest/pytest-fixtures-nuts-bolts/#real" rel="nofollow noreferrer">parameterised tests</a> avoids boring and error-prone duplicate test code.</li>
</ul>

<p>Pytest has a huge set of plugins that simplify various tasks, including:</p>

<ul>
<li><p><a href="https://taverntesting.github.io/" rel="nofollow noreferrer">Tavern</a>, which specialises in testing REST APIs and seems highly relevant here - sort of "Postman for Python unit tests". </p></li>
<li><p><a href="https://pypi.python.org/pypi/pytest-curl-report" rel="nofollow noreferrer">pytest-curl-report</a> - when  testing with the <code>requests</code> library, will print a <code>curl</code> command you can use to reproduce the error from shell. </p></li>
<li><p><a href="https://testinfra.readthedocs.io/en/latest/" rel="nofollow noreferrer">Testinfra</a> - focuses on server testing (e.g. state of OS packages, files, processes, etc, usually tested on remote servers) - strongly recommended if you also need this type  e.g. to test Ansible code.</p>

<ul>
<li>For those who use Puppet or Chef, Testinfra is similar to Beaker (with RSpec), ServerSpec or InSpec.</li>
</ul></li>
</ul>

<p>If you don't like Tavern, you can of course use <code>pycurl</code> with <code>pytest</code>, which makes it easier to diagnose exactly what failed. This example from the <a href="https://pypi.python.org/pypi/pytest-curl-report" rel="nofollow noreferrer">pytest-curl-report site</a> uses only generic pytest features:</p>

<pre><code>$ py.test test.py
============================= test session starts ==============================
platform darwin -- Python 2.7.9 -- py-1.4.27 -- pytest-2.6.4
plugins: curl-report, httpbin, cache, capturelog, cov, flakes, pep8
collected 1 items

test.py F

=================================== FAILURES ===================================
______________________________ test_requests_get _______________________________

    def test_requests_get():
        r = requests.get('http://httpbin.org/get')
&gt;       assert False
E       assert False

test.py:7: AssertionError
</code></pre>

<p>Pytest lets you write all tests with a plain <code>assert</code>, and optionally include a helpful message as part of the output.  For example, one of your tests could be written:</p>

<pre><code>def test_2():
    assert pycurl.response("some_url").matches ("xxx"), "xxx not found in response"
</code></pre>

CC BY-SA 4.0
2017-11-21T18:08:44.130Z
2018-12-04T13:21:25.477Z
2018-12-04T13:21:25.477Z
RichVel
<p>I have noticed during integration testing that I actually work on recurring use cases with restful APIs (or in general HTTP interfaces) I check here and there with bash+cURL. </p>

<p>It starts looking quite messy and gets harder to maintain. Why deliver mess?</p>

<p>Typical use cases are:</p>

<ul>
<li>Check that an URL returns http response code e.g. 200</li>
<li>Check that content type matches some MIME you need in that case</li>
<li>Check that returned content matches some pattern, or passes an abstract validation procedure</li>
</ul>

<p>What I have found so far and consider a workable option without reinventing the wheel, is:</p>

<ul>
<li>have a go with PyCurl - in a hope it fully implements all cURL options esp. proxying but also other switches I might need</li>
<li>use Python's built in unit testng</li>
</ul>

<p>Then I could have, for example, one unit test per service I'd like to check:</p>

<pre><code>import unittest, pycurl

class TestService (unittest.TestCase):

    def test_1(self):
        self.assertEqual(pycurl.returncode("some_url"), 200)

    def test_2(self):
        self.assertTrue(pycurl.response("some_url").matches ("xxx") )

    def test_3(self):
        self.assertTrue (pycurl.ContentType("some_url").equal("xxx"))

if __name__ == '__main__':
    unittest.main()
</code></pre>

<p>Does this make sense or is there a more high-level (but not too complex to pick up and integrate) tooling? </p>

CC BY-SA 3.0
2017-11-16T14:09:34.410Z
What is the API code ? there's a bunch of integration test systems and choosing one is usually done on the same language as the code being tested so the same dev can update both.
Tensibai
CC BY-SA 3.0
2017-11-16T14:14:27.800Z
this can be heterogeneuos and I have no control
Ta Mu
CC BY-SA 3.0
2017-11-16T14:16:26.990Z
So what is the language your team is the more comfortable with ? Every tool will do to test this kind of things, from bash+curl to capistrano post deploy hooks...
Tensibai
CC BY-SA 3.0
2017-11-16T14:19:13.887Z
In chef ecosystem you'll find [inspec](https://www.inspec.io/) which can answer your question with the [http resource](https://www.inspec.io/docs/reference/resources/http/), but it's based on serverspec/rspec, so it's more a ruby world than python world but may worth the try.
Tensibai
CC BY-SA 3.0
2017-11-16T16:42:37.320Z
I've been [working on some python code to do REST calls](https://github.com/jdshewey/salt-formula-katello/blob/master/_modules/katello.py) using [Python's requests library](http://docs.python-requests.org/en/master/). It will let you add any header you want and supports HTTP(s) proxy support, so it should be capable of supporting all of curl's features. You are welcome to borrow my code for your needs or use it as an example if it is helpful.
James Shewey
CC BY-SA 3.0
2017-11-16T17:07:02.330Z
@JamesShewey so maybe the answer is redirects + unit test?
Ta Mu
CC BY-SA 4.0
2019-12-23T08:48:52.503Z
Could you indicate whether you solved the issue?
030
<testing><continuous-deployment><python><curl>
A systematic way to test RESTful APIs with curl?
Ta Mu
# A systematic way to test RESTful APIs with curl?

(Asked by: username_0 on 2017-11-16)

I have noticed during integration testing that I actually work on recurring use cases with restful APIs (or in general HTTP interfaces) I check here and there with bash+cURL. 
It starts looking quite messy and gets harder to maintain. Why deliver mess?
Typical use cases are:

Check that an URL returns http response code e.g. 200
Check that content type matches some MIME you need in that case
Check that returned content matches some pattern, or passes an abstract validation procedure

What I have found so far and consider a workable option without reinventing the wheel, is:

have a go with PyCurl - in a hope it fully implements all cURL options esp. proxying but also other switches I might need
use Python's built in unit testng

Then I could have, for example, one unit test per service I'd like to check:
import unittest, pycurl

class TestService (unittest.TestCase):

    def test_1(self):
        self.assertEqual(pycurl.returncode("some_url"), 200)

    def test_2(self):
        self.assertTrue(pycurl.response("some_url").matches ("xxx") )

    def test_3(self):
        self.assertTrue (pycurl.ContentType("some_url").equal("xxx"))

if __name__ == '__main__':
    unittest.main()

Does this make sense or is there a more high-level (but not too complex to pick up and integrate) tooling?

**Tags:** testing, continuous-deployment, python, curl

> **Comments:**
> **username_1**: What is the API code ? there's a bunch of integration test systems and choosing one is usually done on the same language as the code being tested so the same dev can update both.  
> **username_0**: this can be heterogeneuos and I have no control  
> **username_1**: So what is the language your team is the more comfortable with ? Every tool will do to test this kind of things, from bash+curl to capistrano post deploy hooks...  
> **username_1**: In chef ecosystem you'll find [inspec](https://www.inspec.io/) which can answer your question with the [http resource](https://www.inspec.io/docs/reference/resources/http/), but it's based on serverspec/rspec, so it's more a ruby world than python world but may worth the try.  
> **username_2**: I've been [working on some python code to do REST calls](https://github.com/jdshewey/salt-formula-katello/blob/master/_modules/katello.py) using [Python's requests library](http://docs.python-requests.org/en/master/). It will let you add any header you want and supports HTTP(s) proxy support, so it should be capable of supporting all of curl's features. You are welcome to borrow my code for your needs or use it as an example if it is helpful.  
> **username_0**: @JamesShewey so maybe the answer is redirects + unit test?  
> **username_3**: Could you indicate whether you solved the issue?  

---

## 1 answers

### Answer by username_4 on 2017-11-21. Score: 9

You could look at tools such as Postman which focuses on testing REST APIs with JavaScript - it has some nice features but you lose the use of Python.
Instead, I'd suggest looking at REST-related plugins for pytest, a Python test framework that simplifies your test code, while still running tests written using unittest. 

For example, writing parameterised tests avoids boring and error-prone duplicate test code.

Pytest has a huge set of plugins that simplify various tasks, including:

Tavern, which specialises in testing REST APIs and seems highly relevant here - sort of "Postman for Python unit tests". 
pytest-curl-report - when  testing with the requests library, will print a curl command you can use to reproduce the error from shell. 
Testinfra - focuses on server testing (e.g. state of OS packages, files, processes, etc, usually tested on remote servers) - strongly recommended if you also need this type  e.g. to test Ansible code.

For those who use Puppet or Chef, Testinfra is similar to Beaker (with RSpec), ServerSpec or InSpec.


If you don't like Tavern, you can of course use pycurl with pytest, which makes it easier to diagnose exactly what failed. This example from the pytest-curl-report site uses only generic pytest features:
$ py.test test.py
============================= test session starts ==============================
platform darwin -- Python 2.7.9 -- py-1.4.27 -- pytest-2.6.4
plugins: curl-report, httpbin, cache, capturelog, cov, flakes, pep8
collected 1 items

test.py F

=================================== FAILURES ===================================
______________________________ test_requests_get _______________________________

    def test_requests_get():
        r = requests.get('http://httpbin.org/get')
>       assert False
E       assert False

test.py:7: AssertionError

Pytest lets you write all tests with a plain assert, and optionally include a helpful message as part of the output.  For example, one of your tests could be written:
def test_2():
    assert pycurl.response("some_url").matches ("xxx"), "xxx not found in response"


---

