<NME> README.md
<BEF> # arxiv.py [![Python 3.6](https://img.shields.io/badge/python-3.7-blue.svg)](https://www.python.org/downloads/release/python-370/) [![PyPI](https://img.shields.io/pypi/v/arxiv)](https://pypi.org/project/arxiv/) [![GitHub Workflow Status (branch)](https://img.shields.io/github/workflow/status/lukasschwab/arxiv.py/python-lint-test/master)](https://github.com/lukasschwab/arxiv.py/actions?query=branch%3Amaster)

Python wrapper for [the arXiv API](http://arxiv.org/help/api/index).

## Quick links

+ [Full package documentation](http://lukasschwab.me/arxiv.py/index.html)
Unfortunately, handling these ATOM requsts can be clumsy (especially given inconsistency in data between different result objects, even in the same query).
This is where arxiv.py comes it: it constructs requests for arXiv and gets ATOM feeds via a simple handful of methods, and parses the results into an intuitive format.

Cool demos hopefully coming soon!

## Docs

To get the arxiv package, simply run `pip install arxiv` at the command line.
[arXiv](http://arxiv.org/) is a project by the Cornell University Library that provides open access to 1,000,000+ articles in Physics, Mathematics, Computer Science, Quantitative Biology, Quantitative Finance, and Statistics.


### Query

`arxiv.query(s, prune=True, start=0, max_results=10)`

Sends arXiv a simple query, and returns a list of results, each of which is a `dict` representing an article that matches the query. The articles are ordered for relevance by arXiv.

+ When bool `prune` is `True` (default), a number of artifacts of the ATOM-to-dict conversion are removed from each result to isolate the useful fields. When `prune` is `False`, `prune_query_result` is not called and those key/value pairs are not removed.
+ Integer `start` identifies a 0-indexed position where the query results begin. For example, `query('term', start=4)` will only request and return results indexed 4-14.
+ Integer `max_results` identifies the number of results to be returned (thus, `query` will return results at positions `start` through `start + max_results`). There are some upper limits involved; if you want to pull >60,000 results at a time you should look at the arXiv [API documentation](http://arxiv.org/help/api/user-manual).

### Clean query results

`arxiv.mod_query_result(result)`

Takes a query result dict representing an article and modifies some keys and values to be more user-readable.
See code for specifics.

`arxiv.prune_query_result(result)`

Takes a query result dict representing an article and removes some keys that are redundant or useless.
See code for specifics.

### Download PDF

`arxiv.download(obj)`

Looks up keys `pdf_url` and `title` on dict `obj`. Downloads the PDF from `pdf_url` and saves it to {title}.pdf in the present working directory.
<!-- TODO: improve this section. -->

The `Result` objects yielded by `(Search).results()` include metadata about each paper and some helper functions for downloading their content.

The meaning of the underlying raw data is documented in the [arXiv API User Manual: Details of Atom Results Returned](https://arxiv.org/help/api/user-manual#_details_of_atom_results_returned).

+ `result.entry_id`: A url `http://arxiv.org/abs/{id}`.
+ `result.updated`: When the result was last updated.
+ `result.published`: When the result was originally published.
+ `result.title`: The title of the result.
+ `result.authors`: The result's authors, as `arxiv.Author`s.
+ `result.summary`: The result abstract.
+ `result.comment`: The authors' comment if present.
+ `result.journal_ref`: A journal reference if present.
+ `result.doi`: A URL for the resolved DOI to an external resource if present.
+ `result.primary_category`: The result's primary arXiv category. See [arXiv: Category Taxonomy](https://arxiv.org/category_taxonomy).
+ `result.categories`: All of the result's categories. See [arXiv: Category Taxonomy](https://arxiv.org/category_taxonomy).
+ `result.links`: Up to three URLs associated with this result, as `arxiv.Link`s.
+ `result.pdf_url`: A URL for the result's PDF if present. Note: this URL also appears among `result.links`.

They also expose helper methods for downloading papers: `(Result).download_pdf()` and `(Result).download_source()`.

#### Example: downloading papers

To download a PDF of the paper with ID "1605.08386v1," run a `Search` and then use `(Result).download_pdf()`:

```python
import arxiv

paper = next(arxiv.Search(id_list=["1605.08386v1"]).results())
# Download the PDF to the PWD with a default filename.
paper.download_pdf()
# Download the PDF to the PWD with a custom filename.
paper.download_pdf(filename="downloaded-paper.pdf")
# Download the PDF to a specified directory with a custom filename.
paper.download_pdf(dirpath="./mydir", filename="downloaded-paper.pdf")
```

The same interface is available for downloading .tar.gz files of the paper source:

```python
import arxiv

paper = next(arxiv.Search(id_list=["1605.08386v1"]).results())
# Download the archive to the PWD with a default filename.
paper.download_source()
# Download the archive to the PWD with a custom filename.
paper.download_source(filename="downloaded-paper.tar.gz")
# Download the archive to a specified directory with a custom filename.
paper.download_source(dirpath="./mydir", filename="downloaded-paper.tar.gz")
```

### Client

A `Client` specifies a strategy for fetching results from arXiv's API; it obscures pagination and retry logic.

For most use cases the default client should suffice. You can construct it explicitly with `arxiv.Client()`, or use it via the `(Search).results()` method.

```python
arxiv.Client(
  page_size: int = 100,
  delay_seconds: int = 3,
  num_retries: int = 3
)
```

+ `page_size`: the number of papers to fetch from arXiv per page of results. Smaller pages can be retrieved faster, but may require more round-trips. The API's limit is 2000 results.
+ `delay_seconds`: the number of seconds to wait between requests for pages. [arXiv's Terms of Use](https://arxiv.org/help/api/tou) ask that you "make no more than one request every three seconds."
+ `num_retries`: The number of times the client will retry a request that fails, either with a non-200 HTTP status code or with an unexpected number of results given the search parameters.

#### Example: fetching results with a custom client

`(Search).results()` uses the default client settings. If you want to use a client you've defined instead of the defaults, use `(Client).results(...)`:

```python
import arxiv

big_slow_client = arxiv.Client(
  page_size = 1000,
  delay_seconds = 10,
  num_retries = 5
)

# Prints 1000 titles before needing to make another request.
for result in big_slow_client.results(arxiv.Search(query="quantum")):
  print(result.title)
```

#### Example: logging

To inspect this package's network behavior and API logic, configure an `INFO`-level logger.

```pycon
>>> import logging, arxiv
>>> logging.basicConfig(level=logging.INFO)
>>> paper = next(arxiv.Search(id_list=["1605.08386v1"]).results())
INFO:arxiv.arxiv:Requesting 100 results at offset 0
INFO:arxiv.arxiv:Requesting page of results
INFO:arxiv.arxiv:Got first page; 1 of inf results available
```

<MSG> Improve documentation significantly

<DFF> @@ -8,8 +8,6 @@ They have [an API](http://arxiv.org/help/api/index) that uses ATOM feeds to serv
 Unfortunately, handling these ATOM requsts can be clumsy (especially given inconsistency in data between different result objects, even in the same query).
 This is where arxiv.py comes it: it constructs requests for arXiv and gets ATOM feeds via a simple handful of methods, and parses the results into an intuitive format.
 
-Cool demos hopefully coming soon!
-
 ## Docs
 
 To get the arxiv package, simply run `pip install arxiv` at the command line.
@@ -18,28 +16,65 @@ At the beginning of your Python script, include the line `import arxiv`.
 
 ### Query
 
-`arxiv.query(s, prune=True, start=0, max_results=10)`
-
-Sends arXiv a simple query, and returns a list of results, each of which is a `dict` representing an article that matches the query. The articles are ordered for relevance by arXiv.
+```python
+arxiv.query(search_query, id_list, prune, start, max_results)
+```
 
-+ When bool `prune` is `True` (default), a number of artifacts of the ATOM-to-dict conversion are removed from each result to isolate the useful fields. When `prune` is `False`, `prune_query_result` is not called and those key/value pairs are not removed.
-+ Integer `start` identifies a 0-indexed position where the query results begin. For example, `query('term', start=4)` will only request and return results indexed 4-14.
-+ Integer `max_results` identifies the number of results to be returned (thus, `query` will return results at positions `start` through `start + max_results`). There are some upper limits involved; if you want to pull >60,000 results at a time you should look at the arXiv [API documentation](http://arxiv.org/help/api/user-manual).
+| **Argument**   | **Type**        | **Default** | **Required?** |
+|----------------|-----------------|-------------|---------------|
+| `search_query` | string          | `""`        | No            |
+| `id_list`      | list of strings | `[]`        | No            |
+| `start`        | int             | 0           | No            |
+| `max_results`  | int             | 10          | No            |
 
-### Clean query results
++ `search_query` is a query string; details of its usage are documented [here](https://arxiv.org/help/api/user-manual#Quickstart).
++ `id_list` contains arXiv record IDs (typically of the format `"0710.5765v1"`)
++ `start` is the result offset for paging through a long query result. If set to 0, the API response will begin with the first result; if set to 10, the API response will begin with the 11th.
++ `max_results` is the maximum number of results per query.
 
-`arxiv.mod_query_result(result)`
+All of these arguments are documented more comprehensively in the [arXiv API documentation](https://arxiv.org/help/api/user-manual#Quickstart).
 
-Takes a query result dict representing an article and modifies some keys and values to be more user-readable.
-See code for specifics.
+**Query examples:**
 
-`arxiv.prune_query_result(result)`
+```python
+import arxiv
+# Keyword search
+arxiv.query(search_query="quantum")
+# Get single record by ID
+arxiv.query(id_list=["1707.08567"])
+# Get multiple records by ID
+arxiv.query(id_list=["1707.08567", "1707.08567"])
+```
 
-Takes a query result dict representing an article and removes some keys that are redundant or useless.
-See code for specifics.
+For a more detailed description of the interaction between `search_query` and `id_list`, see [this section of the arXiv documentation](https://arxiv.org/help/api/user-manual#search_query_and_id_list).
 
 ### Download PDF
 
-`arxiv.download(obj)`
-
-Looks up keys `pdf_url` and `title` on dict `obj`. Downloads the PDF from `pdf_url` and saves it to {title}.pdf in the present working directory.
\ No newline at end of file
+```python
+arxiv.download(obj, dirname, prepend_id, slugify)
+```
+
+| **Argument** | **Type** | **Default** | **Required?** |
+|--------------|----------|-------------|---------------|
+| `obj`        | dict     | N/A         | Yes           |
+| `dirname`    | string   | `"./"`      | No            |
+| `prepend_id` | boolean  | False       | No            |
+| `slugify`    | boolean  | False       | No            |
+
++ `obj` is a result object, one of a list returned by query(). This function looks up keys `pdf_url` and `title` in `obj` to make the download request.
++ `dirname` is the relative directory path to which the downloaded PDF will be saved. It defaults to the present working directory.
++ When `prepend_id` is True, the arXiv record ID will be prepended to the download filename.
++ When `slugify` is True, the paper title will be stripped of non-alphanumeric characters before being used as a filename.
+
+**Download PDF examples:**
+
+```python
+import arxiv
+# Query for a paper of interest, then download
+paper = arxiv.query(id_list=["1707.08567"])[0]
+arxiv.download(paper)
+# You can skip the query step if you have the paper info!
+paper2 = {"pdf_url": "http://arxiv.org/pdf/1707.08567v1",
+         "title": "The Paper Title"}
+arxiv.download(paper2)
+```

Improve documentation significantly