<p>Here is a sample of code that should get you started.</p>

<p>I created a separate <code>get_abstracts</code> function.  Then that function get called repetitively until less than 100K ids are returned.  If you wanted to get fancy this whole process could be called as a recursive function but that seemed overly complex.</p>

<pre><code>BUFFER_SIZE = 100000

def get_abstracts(start, term):
    """Get N number of records from Entrez
    :returns list: List of ids returned from search
    """
    handle_one = Entrez.esearch(db="pubmed", term=term, 
                       retmode="text", retstart=start, retmax=BUFFER_SIZE)
    record = Entrez.read(handle_one)
    idlist = record["IdList"]
    return idlist

list_of_ids = []
start_at = 0

# Iterate the search returning 100000
while true:
    ids = get_abstracts(start_at, 'e coli')

    list_of_ids.extend(ids)

    # Break if less than 100K results were returned 
    if len(ids) &lt; BUFFER_SIZE:
        break
    start_at += BUFFER_SIZE

# Do something with list_of_ids
</code></pre>

CC BY-SA 3.0
2017-11-02T20:20:32.367Z
2017-11-02T20:20:32.367Z
Bioathlete
<p>I want to make function get_abstract, but i don't know what this can look like if the function must return idlist. </p>

<pre><code>from Bio import Entrez, Medline, SeqIO


list_of_ids = []
Entrez.email = 'ski89@gmail.com'

handle_one = Entrez.esearch(db="pubmed", term='e coli', 
                           retmode="text", retmax=100000)
record = Entrez.read(handle_one)
idlist = record["IdList"]
for ID in idlist:
    list_of_ids.append(ID)

    if len(list_of_ids) == 100000:
        handle_one = Entrez.esearch(db="pubmed", term='e coli', 
                           retmode="text", retstart=100000, retmax=100000)
        record = Entrez.read(handle_one)
        idlist = record["IdList"]
        for ID in idlist:
            list_of_ids.append(ID)

            if len(list_of_ids) == 200000:
                handle_one = Entrez.esearch(db="pubmed", term='e coli', 
                           retmode="text", retstart=200000, retmax=100000)
                record = Entrez.read(handle_one)
                idlist = record["IdList"]
                for ID in idlist:
                    list_of_ids.append(ID)
</code></pre>

<p>so the args will be the start and TERM:</p>

<pre><code>get_abstracts(start,TERM): 
handle_one = Entrez.esearch(db="pubmed", term=TERM, 
                               retmode="text", retstart=start, retmax=100000)
                    record = Entrez.read(handle_one)
                    idlist = record["IdList"]
                    for ID in idlist:
                        list_of_ids.append(ID)
</code></pre>

<p>Can someone help refactor this? cheers</p>

CC BY-SA 3.0
2017-11-02T19:29:03.737Z
Do you really mean to have a `for` loop nested inside another `for` loop that both reuse the same variables?
Devon Ryan
CC BY-SA 3.0
2017-11-03T07:10:04.967Z
This type of questions are more suited for code review SE. What is your bioinformatic question? Do you need a new method to do it? A faster one?
llrs
CC BY-SA 3.0
2017-11-03T13:55:45.143Z
Refactoring is for speed up script. I dont know any alternatives for download  many abstracts. Selenium is slow. And wanna be better programmer , thats why I ask. I dont have any helper.
MTG
<python>
Biopython code refactoring
# Biopython code refactoring

(Asked by: username_0 on 2017-11-02)

I want to make function get_abstract, but i don't know what this can look like if the function must return idlist. 
from Bio import Entrez, Medline, SeqIO


list_of_ids = []
Entrez.email = 'ski89@gmail.com'

handle_one = Entrez.esearch(db="pubmed", term='e coli', 
                           retmode="text", retmax=100000)
record = Entrez.read(handle_one)
idlist = record["IdList"]
for ID in idlist:
    list_of_ids.append(ID)

    if len(list_of_ids) == 100000:
        handle_one = Entrez.esearch(db="pubmed", term='e coli', 
                           retmode="text", retstart=100000, retmax=100000)
        record = Entrez.read(handle_one)
        idlist = record["IdList"]
        for ID in idlist:
            list_of_ids.append(ID)

            if len(list_of_ids) == 200000:
                handle_one = Entrez.esearch(db="pubmed", term='e coli', 
                           retmode="text", retstart=200000, retmax=100000)
                record = Entrez.read(handle_one)
                idlist = record["IdList"]
                for ID in idlist:
                    list_of_ids.append(ID)

so the args will be the start and TERM:
get_abstracts(start,TERM): 
handle_one = Entrez.esearch(db="pubmed", term=TERM, 
                               retmode="text", retstart=start, retmax=100000)
                    record = Entrez.read(handle_one)
                    idlist = record["IdList"]
                    for ID in idlist:
                        list_of_ids.append(ID)

Can someone help refactor this? cheers

**Tags:** python
> **Comments:**
> **username_1**: Do you really mean to have a `for` loop nested inside another `for` loop that both reuse the same variables?  
> **username_2**: This type of questions are more suited for code review SE. What is your bioinformatic question? Do you need a new method to do it? A faster one?  
> **username_0**: Refactoring is for speed up script. I dont know any alternatives for download  many abstracts. Selenium is slow. And wanna be better programmer , thats why I ask. I dont have any helper.  

---

## 1 answers

### Answer by username_3 on 2017-11-02. Score: 4 | Accepted answer

Here is a sample of code that should get you started.
I created a separate get_abstracts function.  Then that function get called repetitively until less than 100K ids are returned.  If you wanted to get fancy this whole process could be called as a recursive function but that seemed overly complex.
BUFFER_SIZE = 100000

def get_abstracts(start, term):
    """Get N number of records from Entrez
    :returns list: List of ids returned from search
    """
    handle_one = Entrez.esearch(db="pubmed", term=term, 
                       retmode="text", retstart=start, retmax=BUFFER_SIZE)
    record = Entrez.read(handle_one)
    idlist = record["IdList"]
    return idlist

list_of_ids = []
start_at = 0

# Iterate the search returning 100000
while true:
    ids = get_abstracts(start_at, 'e coli')

    list_of_ids.extend(ids)

    # Break if less than 100K results were returned 
    if len(ids) < BUFFER_SIZE:
        break
    start_at += BUFFER_SIZE

# Do something with list_of_ids


---

