<p>Checkout this solution with codepipeline.  <a href="https://aws.amazon.com/blogs/big-data/implement-continuous-integration-and-delivery-of-serverless-aws-glue-etl-applications-using-aws-developer-tools/" rel="nofollow noreferrer">https://aws.amazon.com/blogs/big-data/implement-continuous-integration-and-delivery-of-serverless-aws-glue-etl-applications-using-aws-developer-tools/</a></p>

CC BY-SA 4.0
2019-12-20T08:39:19.997Z
The link looks good. Could you summarize the important elements in the answer to prevent that the information will be lost if the link is deprecated.
030
CC BY-SA 4.0
2019-09-19T19:26:35.810Z
2019-09-19T19:26:35.810Z
user17183
<p>If you are unit testing, you should be mocking external dependencies like Spark and AWS Glue.  Look into Python mocking frameworks like <a href="https://docs.python.org/3/library/unittest.mock.html" rel="nofollow noreferrer">unittest.mock</a>, <a href="https://docs.pytest.org/en/2.9.2/monkeypatch.html" rel="nofollow noreferrer">monkeypatch</a> and <a href="https://pypi.org/project/pytest-mock/" rel="nofollow noreferrer">pytest-mock</a>.  Your unit tests should just test your code and how it interacts with Spark and AWS Glue.  Your unit tests should not test Spark and AWS Glue functionality.  Do that in your component and integration testing.</p>

CC BY-SA 4.0
2020-09-22T19:46:50.680Z
2020-09-22T19:46:50.680Z
David R. Willson
<p>I have an AWS Glue job written in Python that I would like to perform pyunit tests on. However, from reading and experimenting on my own, there seem to be quite a few dependencies on the Glue service, making this task very impractical if not impossible. My only thought would be to separate the non glue dependent functions in the job into a separate module to perform unit testing on.</p>

<p>Is unit testing with AWS Glue possible, and if so, what are the best practices for doing so? </p>

CC BY-SA 4.0
2019-09-19T19:26:35.810Z
Checkout this solution with codepipeline. https://aws.amazon.com/blogs/big-data/implement-continuous-integration-and-delivery-of-serverless-aws-glue-etl-applications-using-aws-developer-tools/
user17183
CC BY-SA 4.0
2019-12-19T13:57:42.503Z
@user17183 Please add this as an answer by explaining the details rather than only adding a link
030
<amazon-web-services><python><aws-codepipeline><unit-testing>
Unit Test AWS Glue job in CodePipeline
Preston Martin
# Unit Test AWS Glue job in CodePipeline

(Asked by: username_0 on 2019-07-08)

I have an AWS Glue job written in Python that I would like to perform pyunit tests on. However, from reading and experimenting on my own, there seem to be quite a few dependencies on the Glue service, making this task very impractical if not impossible. My only thought would be to separate the non glue dependent functions in the job into a separate module to perform unit testing on.

Is unit testing with AWS Glue possible, and if so, what are the best practices for doing so?

**Tags:** amazon-web-services, python, aws-codepipeline, unit-testing

> **Comments:**
> **username_1**: Checkout this solution with codepipeline. https://aws.amazon.com/blogs/big-data/implement-continuous-integration-and-delivery-of-serverless-aws-glue-etl-applications-using-aws-developer-tools/  
> **username_2**: @username_1 Please add this as an answer by explaining the details rather than only adding a link  

---

## 2 answers

### Answer by username_1 on 2019-09-19. Score: 2

Checkout this solution with codepipeline. <https://aws.amazon.com/blogs/big-data/implement-continuous-integration-and-delivery-of-serverless-aws-glue-etl-applications-using-aws-developer-tools/>

> **Comments:**
> **username_2**: The link looks good. Could you summarize the important elements in the answer to prevent that the information will be lost if the link is deprecated.  

---

### Answer by username_3 on 2020-09-22. Score: 1

If you are unit testing, you should be mocking external dependencies like Spark and AWS Glue. Look into Python mocking frameworks like [unittest.mock](https://docs.python.org/3/library/unittest.mock.html), [monkeypatch](https://docs.pytest.org/en/2.9.2/monkeypatch.html) and [pytest-mock](https://pypi.org/project/pytest-mock/). Your unit tests should just test your code and how it interacts with Spark and AWS Glue. Your unit tests should not test Spark and AWS Glue functionality. Do that in your component and integration testing.


---

