<p>After trying myself in many ways, I have found that the method of creating a checkmark column when a desired column is recorded is more efficient and easier to understand.</p>
<p>Here is my bash script to solve my problem in a verbose way (I guess has others way shorter).</p>
<pre class="lang-bash prettyprint-override"><code>#!/usr/bin/bash

change_index(){
    local in=&quot;<span class="math-container">$(&lt; /dev/stdin)" # Variable assignment from the stdin
    local col_num=$</span>(head -n 1 &lt;(echo &quot;$in&quot;) |awk -F&quot;\t&quot; '{print -1+NF/2}')  # Number of fields

    
    for k in <span class="math-container">$(seq 0 $</span>((<span class="math-container">$col_num-1))); do # Loop through number of field
        in=$</span>(awk -v ia=&quot;<span class="math-container">$k" -F"\t"  '{sub("/[^"ia"]","/"ia"\t-\t/"((ia+1)),$(2*ia+1)) ;print $0}' &lt;(echo "$</span>in&quot;)) # Supplement missing fields
    done
    
    in=<span class="math-container">$(awk -v ia="$</span>col_num&quot; -F&quot;\t&quot;  '{sub(&quot;/[^&quot;ia&quot;]&quot;,&quot;/&quot;ia&quot;\t-&quot;,<span class="math-container">$(2*ia+1)) ;print $</span>0}' &lt;(echo &quot;<span class="math-container">$in")) # Supplement missing the last fields
    in=$</span>(sed 's/\/[[:digit:]]\+//g' &lt;(echo &quot;<span class="math-container">$in")) # Remove markers
    echo "$</span>in&quot;
}

for i in {U47804 U47803 U47802 U47801 U47800 U47799 U47798 X92938}; do \
    efetch -db &quot;nucleotide&quot; -id &quot;$i&quot; -format full; done |    
    xtract -pattern Seq-entry \
        -lbl &quot;/0&quot; -element Org-ref_taxname \
        -block Prot-ref_name -lbl &quot;/1&quot; -element Prot-ref_name_E \
        -block Date-std -position first -lbl &quot;/2&quot; -element Date-std_year |
    change_index| column -t
</code></pre>
<h2>Explanation</h2>
<p>The main idea is that if a field is recorded there will be a marker field lay before the recorded field. This action is done by <code>-lbl</code> tag. And we have the following draft table:</p>
<pre><code>/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
/0  Human  immunodeficiency  virus  1  /2  1995       
</code></pre>
<p>After having a draft table, and cleaning the table by <code>change_index()</code> function I wrote.</p>

CC BY-SA 4.0
2023-02-17T11:59:41.730Z
2023-02-18T08:53:58.943Z
2023-02-18T08:53:58.943Z
Rossy Clair
<p>I am using the <a href="https://dataguide.nlm.nih.gov/edirect/xtract.html" rel="nofollow noreferrer">xtract tool</a> of the Entrez Direct (<a href="https://www.ncbi.nlm.nih.gov/books/NBK179288/" rel="nofollow noreferrer">EDirect</a>) package of NBCI. I have a list of Accession Numbers:</p>
<pre><code>U47804, U47803, U47802, U47801, U47800, U47799, U47798, X92938
</code></pre>
<p>and try to get information about:</p>
<ol>
<li>Name of organism (Org-ref_taxname tag)</li>
<li>Product of gene (Prot-ref_name_E tag)</li>
<li>Public year (Date-std_year tag)</li>
</ol>
<p>I executed the following command:</p>
<pre class="lang-bash prettyprint-override"><code>for i in {U47804 U47803 U47802 U47801 U47800 U47799 U47798 X92938}
do
  efetch -db &quot;nucleotide&quot; -id &quot;$i&quot; -format full; done |
    xtract -pattern Seq-entry -element Org-ref_taxname \
      -block Prot-ref_name -def &quot;-&quot; -element Prot-ref_name_E \
      -block Date-std -position first -element Date-std_year
</code></pre>
<p>The result I got:</p>
<pre><code>Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  1995
</code></pre>
<p>Because the <strong>X92938</strong> doesn't have <em>&lt;Prot-ref_name&gt;</em>, so I was expecting to replace the missing fields with <strong>&quot;-&quot;</strong> by the <code>-def</code> tag of the extract tool. It means my desired result is as follows:</p>
<pre><code>Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1    -     1995
</code></pre>
<p>Can someone have experience with the NBCI tool may help me? Thanks a lot.</p>

CC BY-SA 4.0
2023-02-16T23:40:45.760Z
I saw this on [stackoverflow](https://stackoverflow.com/questions/75469564/default-placeholder-for-missing-fields-doesnt-work-with-block-in-xtract-tool-b) but I couldn't figure out the problem. One of the great things about https://bioinformatics.stackexchange.com/ is that you can include heaps more detail here; if you could please provide more context it might help e.g. Why do you want this data? Have you looked at other sources e.g. [the Rentrez package](https://cran.r-project.org/web/packages/rentrez/vignettes/rentrez_tutorial.html)? Could you post-format the data using e.g. `awk`? Etc
jared_mamrot
CC BY-SA 4.0
2023-02-19T02:37:33.397Z
Thanks for your helpful suggestion. According to your suggestion, I have used [Rentrez](https://cran.r-project.org/web/packages/rentrez/vignettes/rentrez_tutorial.html#web_history) and realized it is much easier to use and more powerful than the Entrez bash. So in some special cases like creating bioinformatics pipelines, using bash Entrez is a compulsory thing.
Rossy Clair
<bash><edirect>
Default placeholder for missing fields doesn't work with block in xtract tool
# Default placeholder for missing fields doesn't work with block in xtract tool

(Asked by: username_0 on 2023-02-16)

I am using the xtract tool of the Entrez Direct (EDirect) package of NBCI. I have a list of Accession Numbers:
U47804, U47803, U47802, U47801, U47800, U47799, U47798, X92938

and try to get information about:

Name of organism (Org-ref_taxname tag)
Product of gene (Prot-ref_name_E tag)
Public year (Date-std_year tag)

I executed the following command:
for i in {U47804 U47803 U47802 U47801 U47800 U47799 U47798 X92938}
do
  efetch -db "nucleotide" -id "$i" -format full; done |
    xtract -pattern Seq-entry -element Org-ref_taxname \
      -block Prot-ref_name -def "-" -element Prot-ref_name_E \
      -block Date-std -position first -element Date-std_year

The result I got:
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  1995

Because the X92938 doesn't have , so I was expecting to replace the missing fields with "-" by the -def tag of the extract tool. It means my desired result is as follows:
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1  gp120   1996
Human immunodeficiency virus 1    -     1995

Can someone have experience with the NBCI tool may help me? Thanks a lot.

**Tags:** bash, edirect
> **Comments:**
> **username_1**: I saw this on [stackoverflow](https://stackoverflow.com/questions/75469564/default-placeholder-for-missing-fields-doesnt-work-with-block-in-xtract-tool-b) but I couldn't figure out the problem. One of the great things about https://bioinformatics.stackexchange.com/ is that you can include heaps more detail here; if you could please provide more context it might help e.g. Why do you want this data? Have you looked at other sources e.g. [the Rentrez package](https://cran.r-project.org/web/packages/rentrez/vignettes/rentrez_tutorial.html)? Could you post-format the data using e.g. `awk`? Etc  
> **username_0**: Thanks for your helpful suggestion. According to your suggestion, I have used [Rentrez](https://cran.r-project.org/web/packages/rentrez/vignettes/rentrez_tutorial.html#web_history) and realized it is much easier to use and more powerful than the Entrez bash. So in some special cases like creating bioinformatics pipelines, using bash Entrez is a compulsory thing.  

---

## 1 answers

### Answer by username_0 on 2023-02-17. Score: 2 | Accepted answer

After trying myself in many ways, I have found that the method of creating a checkmark column when a desired column is recorded is more efficient and easier to understand.
Here is my bash script to solve my problem in a verbose way (I guess has others way shorter).
#!/usr/bin/bash

change_index(){
    local in="$(< /dev/stdin)" # Variable assignment from the stdin
    local col_num=$(head -n 1 <(echo "$in") |awk -F"\t" '{print -1+NF/2}')  # Number of fields

    
    for k in $(seq 0 $(($col_num-1))); do # Loop through number of field
        in=$(awk -v ia="$k" -F"\t"  '{sub("/[^"ia"]","/"ia"\t-\t/"((ia+1)),$(2*ia+1)) ;print $0}' <(echo "$in")) # Supplement missing fields
    done
    
    in=$(awk -v ia="$col_num" -F"\t"  '{sub("/[^"ia"]","/"ia"\t-",$(2*ia+1)) ;print $0}' <(echo "$in")) # Supplement missing the last fields
    in=$(sed 's/\/[[:digit:]]\+//g' <(echo "$in")) # Remove markers
    echo "$in"
}

for i in {U47804 U47803 U47802 U47801 U47800 U47799 U47798 X92938}; do \
    efetch -db "nucleotide" -id "$i" -format full; done |    
    xtract -pattern Seq-entry \
        -lbl "/0" -element Org-ref_taxname \
        -block Prot-ref_name -lbl "/1" -element Prot-ref_name_E \
        -block Date-std -position first -lbl "/2" -element Date-std_year |
    change_index| column -t

Explanation
The main idea is that if a field is recorded there will be a marker field lay before the recorded field. This action is done by -lbl tag. And we have the following draft table:
/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
/0  Human  immunodeficiency  virus  1  /2  1995       

After having a draft table, and cleaning the table by change_index() function I wrote.


---

