<NME> README.md
<BEF> # arxiv.py
Python wrapper for the arXiv API: http://arxiv.org/help/api/index

## About arXiv
[arXiv](http://arxiv.org/) is a project by the Cornell University Library that provides open access to 1,000,000+ articles in Physics, Mathematics, Computer Science, Quantitative Biology, Quantitative Finance, and Statistics.

They have [an API](http://arxiv.org/help/api/index) that uses ATOM feeds to serve simple database queries.
Unfortunately, handling these ATOM requsts can be clumsy (especially given inconsistency in data between different result objects, even in the same query).
This is where arxiv.py comes it: it constructs requests for arXiv and gets ATOM feeds via a simple handful of methods, and parses the results into an intuitive format.

## Usage


## About arXiv

$ pip install arxiv
```

In your Python script, include the line

```python
import arxiv
```

Verify the installation with

```bash
$ python setup.py test
```

### Query

```python
arxiv.query(search_query, id_list, prune, start, max_results, sort_by, sort_order)
```

| **Argument**   | **Type**        | **Default**    | **Required?** |
|----------------|-----------------|----------------|---------------|
| `search_query` | string          | `""`           | No            |
| `id_list`      | list of strings | `[]`           | No            |
| `start`        | int             | 0              | No            |
| `max_results`  | int             | 10             | No            |
| `sort_by`      | string          | `"relevance"`  | No            |
| `sort_order`   | string          | `"descending"` | No            |

+ `search_query` is a query string; details of its usage are documented [here](https://arxiv.org/help/api/user-manual#Quickstart).
  + **Note:** multi-field queries must be space-delimited. `au:balents_leon AND cat:cond-mat.str-el` is valid; `au:balents_leon+AND+cat:cond-mat.str-el` is *not* valid.

+ `id_list` contains arXiv record IDs (typically of the format `"0710.5765v1"`)

+ `start` is the result offset for paging through a long query result. If set to 0, the API response will begin with the first result; if set to 10, the API response will begin with the 11th.

+ `max_results` is the maximum number of results per response.

All of these arguments are documented more comprehensively in the [arXiv API documentation](https://arxiv.org/help/api/user-manual#Quickstart).

**Query examples:**

```python
import arxiv
# Keyword queries
arxiv.query(search_query="quantum")
# Multi-field queries
arxiv.query(search_query="au:balents_leon AND cat:cond-mat.str-el")
# Get single record by ID
arxiv.query(id_list=["1707.08567"])
# Get multiple records by ID
arxiv.query(id_list=["1707.08567", "1707.08567"])
```

For a more detailed description of the interaction between `search_query` and `id_list`, see [this section of the arXiv documentation](https://arxiv.org/help/api/user-manual#search_query_and_id_list).

**Iterative downloads**

```python
import arxiv
result = arxiv.query(search_query="quantum", iterative=True)

for paper in result():
   print(paper) # a single result
```


### Download PDF

```python
arxiv.download(obj, dirpath, slugify)
```

| **Argument** | **Type** | **Default** | **Required?** |
|--------------|----------|-------------|---------------|
| `obj`        | dict     | N/A         | Yes           |
| `dirpath`    | string   | `"./"`      | No            |
| `slugify`    | boolean  | False       | No            |

+ `obj` is a result object, one of a list returned by query(). This function looks up keys `pdf_url` and `title` in `obj` to make the download request.

+ `dirpath` is the relative directory path to which the downloaded PDF will be saved. It defaults to the present working directory.

+ `slugify` is a function that processes `obj` into a filename. By default, `arxiv.download(obj)` prepends the object ID to the object title.

**Download PDF examples:**

```python
import arxiv
# Query for a paper of interest, then download
# Download the PDF to the PWD with a custom filename.
paper.download_pdf(filename="downloaded-paper.pdf")
# Download the PDF to a specified directory with a custom filename.
paper.download_pdf(dirpath="./mydir", filename="downloaded-paper.pdf")
```

The same interface is available for downloading .tar.gz files of the paper source:

```python
import arxiv

# Download with a specified slugifier function
arxiv.download(paper, slugify=custom_slugify)
```
### Client

A `Client` specifies a strategy for fetching results from arXiv's API; it obscures pagination and retry logic.

For most use cases the default client should suffice. You can construct it explicitly with `arxiv.Client()`, or use it via the `(Search).results()` method.

```python
arxiv.Client(
  page_size: int = 100,
  delay_seconds: int = 3,
  num_retries: int = 3
)
```

+ `page_size`: the number of papers to fetch from arXiv per page of results. Smaller pages can be retrieved faster, but may require more round-trips. The API's limit is 2000 results.
+ `delay_seconds`: the number of seconds to wait between requests for pages. [arXiv's Terms of Use](https://arxiv.org/help/api/tou) ask that you "make no more than one request every three seconds."
+ `num_retries`: The number of times the client will retry a request that fails, either with a non-200 HTTP status code or with an unexpected number of results given the search parameters.

#### Example: fetching results with a custom client

`(Search).results()` uses the default client settings. If you want to use a client you've defined instead of the defaults, use `(Client).results(...)`:

```python
import arxiv

big_slow_client = arxiv.Client(
  page_size = 1000,
  delay_seconds = 10,
  num_retries = 5
)

# Prints 1000 titles before needing to make another request.
for result in big_slow_client.results(arxiv.Search(query="quantum")):
  print(result.title)
```

#### Example: logging

To inspect this package's network behavior and API logic, configure an `INFO`-level logger.

```pycon
>>> import logging, arxiv
>>> logging.basicConfig(level=logging.INFO)
>>> paper = next(arxiv.Search(id_list=["1605.08386v1"]).results())
INFO:arxiv.arxiv:Requesting 100 results at offset 0
INFO:arxiv.arxiv:Requesting page of results
INFO:arxiv.arxiv:Got first page; 1 of inf results available
```

<MSG> Update README with 0.4.0 documentation

<DFF> @@ -1,12 +1,10 @@
-# arxiv.py
-Python wrapper for the arXiv API: http://arxiv.org/help/api/index
+# arxiv.py [![Python 2.7](https://img.shields.io/badge/python-2.7-blue.svg)](https://www.python.org/downloads/release/python-270/) [![Python 3.6](https://img.shields.io/badge/python-3.7-blue.svg)](https://www.python.org/downloads/release/python-370/)
+
+Python wrapper for [the arXiv API](http://arxiv.org/help/api/index).
 
 ## About arXiv
-[arXiv](http://arxiv.org/) is a project by the Cornell University Library that provides open access to 1,000,000+ articles in Physics, Mathematics, Computer Science, Quantitative Biology, Quantitative Finance, and Statistics.
 
-They have [an API](http://arxiv.org/help/api/index) that uses ATOM feeds to serve simple database queries.
-Unfortunately, handling these ATOM requsts can be clumsy (especially given inconsistency in data between different result objects, even in the same query).
-This is where arxiv.py comes it: it constructs requests for arXiv and gets ATOM feeds via a simple handful of methods, and parses the results into an intuitive format.
+[arXiv](http://arxiv.org/) is a project by the Cornell University Library that provides open access to 1,000,000+ articles in Physics, Mathematics, Computer Science, Quantitative Biology, Quantitative Finance, and Statistics.
 
 ## Usage
 
@@ -16,91 +14,99 @@ This is where arxiv.py comes it: it constructs requests for arXiv and gets ATOM
 $ pip install arxiv
 ```
 
-In your Python script, include the line
-
-```python
-import arxiv
-```
-
 Verify the installation with
 
 ```bash
 $ python setup.py test
 ```
 
-### Query
+In your Python script, include the line
 
 ```python
-arxiv.query(search_query, id_list, prune, start, max_results, sort_by, sort_order)
+import arxiv
 ```
 
-| **Argument**   | **Type**        | **Default**    | **Required?** |
-|----------------|-----------------|----------------|---------------|
-| `search_query` | string          | `""`           | No            |
-| `id_list`      | list of strings | `[]`           | No            |
-| `start`        | int             | 0              | No            |
-| `max_results`  | int             | 10             | No            |
-| `sort_by`      | string          | `"relevance"`  | No            |
-| `sort_order`   | string          | `"descending"` | No            |
+### Query
+
+```python
+arxiv.query(search_query="",
+            id_list=[],
+            max_results=None,
+            sort_by="relevance",
+            sort_order="descending",
+            prune=True,
+            iterative=False,
+            max_chunk_results=1000)
+```
 
-+ `search_query` is a query string; details of its usage are documented [here](https://arxiv.org/help/api/user-manual#Quickstart).
+| **Argument**   | **Type**        | **Default**    |
+|----------------|-----------------|----------------|
+| `search_query` | string          | `""`           |
+| `id_list`      | list of strings | `[]`           |
+| `max_results`  | int             | 10             |
+| `sort_by`      | string          | `"relevance"`  |
+| `sort_order`   | string          | `"descending"` |
+| `prune`        | boolean         | `True`         |
+| `iterative`    | boolean         | `False`        |
+| `max_chunk_results` | int        | 1000           |
+
++ `search_query`: an arXiv query string. Format documented [here](https://arxiv.org/help/api/user-manual#Quickstart).
   + **Note:** multi-field queries must be space-delimited. `au:balents_leon AND cat:cond-mat.str-el` is valid; `au:balents_leon+AND+cat:cond-mat.str-el` is *not* valid.
 
-+ `id_list` contains arXiv record IDs (typically of the format `"0710.5765v1"`)
++ `id_list`: list of arXiv record IDs (typically of the format `"0710.5765v1"`).
 
-+ `start` is the result offset for paging through a long query result. If set to 0, the API response will begin with the first result; if set to 10, the API response will begin with the 11th.
++ `max_results`: the maximum number of results returned by the query.
 
-+ `max_results` is the maximum number of results per response.
++ `sort_by`: the arXiv field by which the result should be sorted.
 
-All of these arguments are documented more comprehensively in the [arXiv API documentation](https://arxiv.org/help/api/user-manual#Quickstart).
++ `sort_order`: the sorting order, i.e. "ascending", "descending" or None.
+
++ `prune`: when `True`, received abstract objects will be simplified.
+
++ `iterative`: when `True`, `query()` will return an iterator. Otherwise, `query()` iterates internally and returns the full list of results.
+
++ `max_chunk_results`: the maximum number of abstracts ot be retrieved by a single internal request to the arXiv API.
 
 **Query examples:**
 
 ```python
 import arxiv
+
 # Keyword queries
-arxiv.query(search_query="quantum")
+arxiv.query(search_query="quantum", max_results=100)
 # Multi-field queries
 arxiv.query(search_query="au:balents_leon AND cat:cond-mat.str-el")
 # Get single record by ID
 arxiv.query(id_list=["1707.08567"])
 # Get multiple records by ID
 arxiv.query(id_list=["1707.08567", "1707.08567"])
-```
-
-For a more detailed description of the interaction between `search_query` and `id_list`, see [this section of the arXiv documentation](https://arxiv.org/help/api/user-manual#search_query_and_id_list).
-
-**Iterative downloads**
-
-```python
-import arxiv
-result = arxiv.query(search_query="quantum", iterative=True)
 
+# Get interator over query results
+result = arxiv.query(search_query="quantum", max_chunk_results=10, iterative=True)
 for paper in result():
-   print(paper) # a single result
+   print(paper)
 ```
 
+For a more detailed description of the interaction between `search_query` and `id_list`, see [this section of the arXiv documentation](https://arxiv.org/help/api/user-manual#search_query_and_id_list).
 
-### Download PDF
+### Download article PDF
 
 ```python
-arxiv.download(obj, dirpath, slugify)
+arxiv.download(obj, dirpath="./", slugify=arxiv.slugify)
 ```
 
 | **Argument** | **Type** | **Default** | **Required?** |
 |--------------|----------|-------------|---------------|
 | `obj`        | dict     | N/A         | Yes           |
 | `dirpath`    | string   | `"./"`      | No            |
-| `slugify`    | boolean  | False       | No            |
+| `slugify`    | function | `arxiv.slugify` | No        |
 
-+ `obj` is a result object, one of a list returned by query(). This function looks up keys `pdf_url` and `title` in `obj` to make the download request.
++ `obj` is a result object, one of a list returned by query(). `obj` must at minimum contain values corresponding to `pdf_url` and `title`.
 
 + `dirpath` is the relative directory path to which the downloaded PDF will be saved. It defaults to the present working directory.
 
 + `slugify` is a function that processes `obj` into a filename. By default, `arxiv.download(obj)` prepends the object ID to the object title.
 
-**Download PDF examples:**
-
 ```python
 import arxiv
 # Query for a paper of interest, then download
@@ -118,3 +124,9 @@ def custom_slugify(obj):
 # Download with a specified slugifier function
 arxiv.download(paper, slugify=custom_slugify)
 ```
+
+## Contributors
+
+<a href="https://github.com/lukasschwab/arxiv.py/graphs/contributors">
+  <img src="https://contributors-img.firebaseapp.com/image?repo=lukasschwab/arxiv.py" />
+</a>

Update README with 0.4.0 documentation