<p>Putting the placeholder inside of an input file won't work, since the contents will just be turned into a string when read with <code>read_lines</code> and won't get treated like WDL code. Since the lines from the <code>program_file</code> are simply getting pasted into the command, you can move the contents into the command section. In a placeholder (<code>~{}</code>), adding a string to an optional value will result in an empty string if that optional value is not supplied (see the spec <a href="https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#prepending-a-string-to-an-optional-parameter" rel="nofollow noreferrer">here</a>). So you can use optional inputs to make the different lines non-mandatory:</p>
<pre><code>task trimmomatic {
    input {
        File fastq1
        File fastq2
        String outfile1 = “trimmed_R1.fastq.gz”
        String outfile2 = “trimmed_R2.fastq.gz”
        File? adapter_file
        Int? minlen
        String? slidingwindow
        Int? leading
        Int? trailing
     }
 
     command {
        TrimmomaticPE \
        ~{fastq1} ~{fastq2} \
        ~{outfile1} ~{outfile2} \
        ~{&quot;ILLUMINACLIP:&quot; + adapter_file + &quot;:2:30:10:7:keepBothReads&quot;} \
        ~{&quot;MINLEN:&quot; + minlen} \
        ~{&quot;SLIDINGWINDOW:&quot; + slidingwindow} \
        ~{&quot;LEADING:&quot; + leading} \
        ~{&quot;TRAILING:&quot; + trailing}
     }
 
     output {
        File out1 = outfile1
        File out2 = outfile2
     }
}
</code></pre>
<p>If, say, <code>adapter_file</code> were to be omitted in the inputs, then the produced command should look something like this:</p>
<pre><code>TrimmomaticPE \
R1.fq R2.fq \
trimmed_R1.fastq.gz trimmed_R2.fastq.gz \
 \
MINLEN:36 \
SLIDINGWINDOW:4:15 \
LEADING:3 \
TRAILING:3
</code></pre>
<p>Also note that the output file paths in the input section should be <code>String</code>s. Since they don't exist yet, the execution engine will complain that it can't find them if they are <code>File</code>s</p>
<p>Ps. I'm not familiar with the actual trimmomatic command options, so please do check that this produces the desired command. I suspect that the <code>:2:30:10:7:keepBothReads</code> bit could be split up into additional inputs as well.</p>

CC BY-SA 4.0
2022-10-04T08:34:33.483Z
2022-10-04T08:34:33.483Z
DavyCats
<p>I am developing a gatk pipeline for variant calling and want to include trimmomatic as a preprocessing step. I would like the user to specify a program file of parameters to feed into trimmomatic, and a file of adapter sequences to provide to the ILLUMINACLIP parameter. My goal is to include a variable (“~{adapter_file}” below) in the program file that will expand to include the adapter filename, like so:</p>
<p>Adapter file:</p>
<pre><code>&gt;IL_Multiplex_five_prime_fwd
AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT
&gt;IL_Multiplex_three_prime_rev
AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC
</code></pre>
<p>Program file:</p>
<pre><code>ILLUMINACLIP:~{adapter_file}:2:30:10:7:keepBothReads
MINLEN:36
SLIDINGWINDOW:4:15
LEADING:3
TRAILING:3
</code></pre>
<p>In this way, the user can change the adaptor file without having to alter the program file each time. However, I can’t seem to get the ~{adapter_file} variable to expand inside the ~{trim_program_array} variable. Here’s a simplified version of my trimmomatic WDL file:</p>
<p>Sample workflow:</p>
<pre><code>version 1.0
 
Workflow trim {
    input {
        File fastq1 = “R1.fastq.gz”
        File fastq2 = “R2.fastq.gz”
        File program_file = “trim_program.txt”
        File adapter_file = “adapters.fa”
    }
 
    Array[String] trim_program_array = read_lines(program_file)
 
    call trimmomatic {
        input:
            fastq1 = fastq1,
            fastq2 = fastq2,
            trim_program_array = trim_program_array,
            adapter_file = adaptor_file
    }
}
 
Task trimmomatic {
    input {
        File fastq1
        File fastq2
        File outfile1 = “trimmed_R1.fastq.gz”
        File outfile2 = “trimmed_R2.fastq.gz”
        Array[String] trim_program_array
        File adapter_file
     }
 
     command {
        TrimmomaticPE \
        ~{fastq1} ~{fastq2} \
        ~{outfile1} ~{outfile2} \
        ~{sep = “ “ trim_program_array}
     }
 
     output {
        File out1 = outfile1
        File out2 = outfile2
     }
}
</code></pre>
<p>My goal is for the first element of ~{sep = &quot; &quot; trim_program_array} (aka the first line of trim_program.txt) to evaluate to “ILLUMINACLIP:adapters.fa:2:30:10:7:keepBothReads” rather than “ILLUMINACLIP:~{adapter_file}:2:30:10:7:keepBothReads”. I know this is because ~{adapter_file} is nested inside trim_program_array[0], so it’s only expanding the “outer” variable. How can I convince WDL to expand the variable nested inside the first element of that array before executing the bash command?</p>

CC BY-SA 4.0
2022-09-27T07:48:41.710Z
Hi, I've corrected your question to make the spelling of the word 'adapter' consistent, but noticed that some of those changes were in the configuration file. Can you please re-check to make sure this spelling inconsistency is not causing your problems?
gringer
CC BY-SA 4.0
2022-09-27T07:59:10.613Z
The ~{} syntax only works within the wdl command section or wdl string literals. Since the string is getting read from a file in this case, it is probably getting escaped. It looks like this file is just getting used to provide arguments in the command. Can't you just include the contents of the program file in the command section directly?
DavyCats
CC BY-SA 4.0
2022-09-29T02:11:32.140Z
@DavyCats Yes, that's what I eventually figured out, but it would be nice to have it in the program file so that users of the pipeline aren't left wondering what ":2:30:10:7:keepBothReads" in the first line of the program file refers to, since 'ILLUMINACLIP:~{adaptor_file}' is missing. There are worse problems to have though, so I'm fine with that work-around. Thanks for the suggestion.
user16196
CC BY-SA 4.0
2022-09-30T20:43:17.733Z
@DavyCats Upon thinking about this further, I realized that one advantage to supplying a program file to Trimmomatic is that the user gets to determine which parameters to trim by, rather than just choosing the value for each predetermined parameter. By encoding "ILLUMINACLIP:~{adaptor_file}" into the code block, I'm in effect making that a required parameter. I've seen many cases where a user wouldn't want to supply that parameter to Trimmomatic at all. I'm hoping that someone will come along and help me with a better solution to this problem so that I don't have to hard code any parameters.
user16196
<wdl>
Nested variable expansion in WDL code block
# Nested variable expansion in WDL code block

(Asked by: username_0 on 2022-09-26)

I am developing a gatk pipeline for variant calling and want to include trimmomatic as a preprocessing step. I would like the user to specify a program file of parameters to feed into trimmomatic, and a file of adapter sequences to provide to the ILLUMINACLIP parameter. My goal is to include a variable (“~{adapter_file}” below) in the program file that will expand to include the adapter filename, like so:
Adapter file:
>IL_Multiplex_five_prime_fwd
AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT
>IL_Multiplex_three_prime_rev
AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC

Program file:
ILLUMINACLIP:~{adapter_file}:2:30:10:7:keepBothReads
MINLEN:36
SLIDINGWINDOW:4:15
LEADING:3
TRAILING:3

In this way, the user can change the adaptor file without having to alter the program file each time. However, I can’t seem to get the ~{adapter_file} variable to expand inside the ~{trim_program_array} variable. Here’s a simplified version of my trimmomatic WDL file:
Sample workflow:
version 1.0
 
Workflow trim {
    input {
        File fastq1 = “R1.fastq.gz”
        File fastq2 = “R2.fastq.gz”
        File program_file = “trim_program.txt”
        File adapter_file = “adapters.fa”
    }
 
    Array[String] trim_program_array = read_lines(program_file)
 
    call trimmomatic {
        input:
            fastq1 = fastq1,
            fastq2 = fastq2,
            trim_program_array = trim_program_array,
            adapter_file = adaptor_file
    }
}
 
Task trimmomatic {
    input {
        File fastq1
        File fastq2
        File outfile1 = “trimmed_R1.fastq.gz”
        File outfile2 = “trimmed_R2.fastq.gz”
        Array[String] trim_program_array
        File adapter_file
     }
 
     command {
        TrimmomaticPE \
        ~{fastq1} ~{fastq2} \
        ~{outfile1} ~{outfile2} \
        ~{sep = “ “ trim_program_array}
     }
 
     output {
        File out1 = outfile1
        File out2 = outfile2
     }
}

My goal is for the first element of ~{sep = " " trim_program_array} (aka the first line of trim_program.txt) to evaluate to “ILLUMINACLIP:adapters.fa:2:30:10:7:keepBothReads” rather than “ILLUMINACLIP:~{adapter_file}:2:30:10:7:keepBothReads”. I know this is because ~{adapter_file} is nested inside trim_program_array[0], so it’s only expanding the “outer” variable. How can I convince WDL to expand the variable nested inside the first element of that array before executing the bash command?

**Tags:** wdl
> **Comments:**
> **username_1**: Hi, I've corrected your question to make the spelling of the word 'adapter' consistent, but noticed that some of those changes were in the configuration file. Can you please re-check to make sure this spelling inconsistency is not causing your problems?  
> **username_2**: The ~{} syntax only works within the wdl command section or wdl string literals. Since the string is getting read from a file in this case, it is probably getting escaped. It looks like this file is just getting used to provide arguments in the command. Can't you just include the contents of the program file in the command section directly?  
> **username_0**: @DavyCats Yes, that's what I eventually figured out, but it would be nice to have it in the program file so that users of the pipeline aren't left wondering what ":2:30:10:7:keepBothReads" in the first line of the program file refers to, since 'ILLUMINACLIP:~{adaptor_file}' is missing. There are worse problems to have though, so I'm fine with that work-around. Thanks for the suggestion.  
> **username_0**: @DavyCats Upon thinking about this further, I realized that one advantage to supplying a program file to Trimmomatic is that the user gets to determine which parameters to trim by, rather than just choosing the value for each predetermined parameter. By encoding "ILLUMINACLIP:~{adaptor_file}" into the code block, I'm in effect making that a required parameter. I've seen many cases where a user wouldn't want to supply that parameter to Trimmomatic at all. I'm hoping that someone will come along and help me with a better solution to this problem so that I don't have to hard code any parameters.  

---

## 1 answers

### Answer by username_2 on 2022-10-04. Score: 1

Putting the placeholder inside of an input file won't work, since the contents will just be turned into a string when read with read_lines and won't get treated like WDL code. Since the lines from the program_file are simply getting pasted into the command, you can move the contents into the command section. In a placeholder (~{}), adding a string to an optional value will result in an empty string if that optional value is not supplied (see the spec here). So you can use optional inputs to make the different lines non-mandatory:
task trimmomatic {
    input {
        File fastq1
        File fastq2
        String outfile1 = “trimmed_R1.fastq.gz”
        String outfile2 = “trimmed_R2.fastq.gz”
        File? adapter_file
        Int? minlen
        String? slidingwindow
        Int? leading
        Int? trailing
     }
 
     command {
        TrimmomaticPE \
        ~{fastq1} ~{fastq2} \
        ~{outfile1} ~{outfile2} \
        ~{"ILLUMINACLIP:" + adapter_file + ":2:30:10:7:keepBothReads"} \
        ~{"MINLEN:" + minlen} \
        ~{"SLIDINGWINDOW:" + slidingwindow} \
        ~{"LEADING:" + leading} \
        ~{"TRAILING:" + trailing}
     }
 
     output {
        File out1 = outfile1
        File out2 = outfile2
     }
}

If, say, adapter_file were to be omitted in the inputs, then the produced command should look something like this:
TrimmomaticPE \
R1.fq R2.fq \
trimmed_R1.fastq.gz trimmed_R2.fastq.gz \
 \
MINLEN:36 \
SLIDINGWINDOW:4:15 \
LEADING:3 \
TRAILING:3

Also note that the output file paths in the input section should be Strings. Since they don't exist yet, the execution engine will complain that it can't find them if they are Files
Ps. I'm not familiar with the actual trimmomatic command options, so please do check that this produces the desired command. I suspect that the :2:30:10:7:keepBothReads bit could be split up into additional inputs as well.


---

