<p>I am reluctant to go through this wall of code, but generally it is advisable to always keep bam files and its index in a tuple so the index is staged with its bam file.</p>
<p>For example a process that creates the bam and now emits a tuple with bam and bai file:</p>
<pre><code>(...)
output:
tuple path(&quot;<span class="math-container">${id}.bam"), path("$</span>{id}.bam.bai&quot;), emit: tuple_bam
</code></pre>
<p>and then a process that receives such a <code>tuple_bam</code>:</p>
<pre><code>(...)
input:
tuple path(bam), path(bai)
</code></pre>
<p>The first entry is the bam, the second the index.</p>

CC BY-SA 4.0
2023-07-18T13:46:24.227Z
Thanks it worked.  I cannot understand why only one chromosome is being processed in lancet despite a channel being passed.
Death Metal
CC BY-SA 4.0
2023-07-18T07:08:48.877Z
2023-07-18T07:08:48.877Z
ATpoint
<p>I've a pipeline with many processes.<br />
One of the steps is to run lancet tool. Lancet tool runs per chromosome (1-22, x,y). The input in the lancet step is output from the applybqsr (gatk).</p>
<p>There are tumor and normal bam generated along with the index. human genome FASTA has a index. When I run lancet through the main.nf I run into an error</p>
<blockquote>
<p>ERROR: index not found for BAM file</p>
</blockquote>
<p>I've a config file as:</p>
<pre><code>params {

        hg38genome=&quot;/sc/arion/reference_genome/human/hg38/Homo_sapiens_assembly38.fasta.gz&quot;
        outdir = '/sc/arion/output_pipeline_nextflow/test_tiny_datasets/'
        dbsnp=&quot;/sc/arion/projects/Daphni2_Pipeline_resources/resourceDBsnp146/dbsnp_146.hg38.vcf.gz&quot;
        target=&quot;/sc/arion/projects/Twist_Exome_Target_hg38.padded.interval_list.bed/Twist_Exome_Target_hg38.padded.interval_list.bed&quot;

}
        withName: applybqsr {
        publishDir = [
                [  
                path: &quot;<span class="math-container">${params.outdir}/applybqsr/normal/",
                mode: 'copy',
                pattern:'*normal*'
                ],
                [
                path: "$</span>{params.outdir}/applybqsr/tumor/&quot;,
                mode: 'copy',
                pattern:'*tumor*'
                ]       
        ]
        }
        //applybqsr ends        

        
        withName: lancet {
        publishDir = [  
                path: &quot;${params.outdir}/lancet/&quot;,
                mode: 'copy'
        ]
        }
        //lancet ends
</code></pre>
<p>I have applybqsr as:</p>
<pre><code>process applybqsr{

        maxForks 3
        debug true

        memory '6 GB'

        input:
        tuple val(sample_id_tumor), path(markbam_tumor)
        tuple val(sample_id_tumor), path(recaltable_tumor)
        tuple val(sample_id_normal), path(markbam_normal)
        tuple val(sample_id_normal), path(recaltable_normal)

        output:
        tuple val(sample_id_tumor), path(&quot;<span class="math-container">${sample_id_tumor}.sorted.markdup.recal.bam"), emit: recal_bam_tumor
        tuple val(sample_id_tumor), path("$</span>{sample_id_tumor}.sorted.markdup.recal.bai&quot;),  emit: recal_bai_tumor
        tuple val(sample_id_normal), path(&quot;<span class="math-container">${sample_id_normal}.sorted.markdup.recal.bam"), emit: recal_bam_normal
        tuple val(sample_id_normal), path("$</span>{sample_id_normal}.sorted.markdup.recal.bai&quot;),  emit: recal_bai_normal
        
        script:
        &quot;&quot;&quot;
        ml gatk/4.1.3.0 samtools/1.9
        samtools index <span class="math-container">${markbam_tumor}
        samtools index $</span>{markbam_normal}

        gatk ApplyBQSR \\
        -R <span class="math-container">$params.hg38genome \\
        -I $</span>{markbam_tumor} \\
        --bqsr-recal-file <span class="math-container">${recaltable_tumor} \\
        -O $</span>{sample_id_tumor}.sorted.markdup.recal.bam \\
        -L <span class="math-container">$params.target \\
        --create-output-bam-index true \\
        --java-options -Xmx$</span>{task.memory.toGiga()}g

        gatk ApplyBQSR \\
        -R <span class="math-container">$params.hg38genome \\
        -I $</span>{markbam_normal} \\
        --bqsr-recal-file <span class="math-container">${recaltable_normal} \\
        -O $</span>{sample_id_normal}.sorted.markdup.recal.bam \\
        -L <span class="math-container">$params.target \\
        --create-output-bam-index true \\
        --java-options -Xmx$</span>{task.memory.toGiga()}g


        &quot;&quot;&quot;

}
workflow.onComplete { 
        log.info ( workflow.success ? &quot;Done applyBQSR!&quot; : &quot;Oops .. something went wrong in applyBQSR&quot; )
}
</code></pre>
<p>I've lancet.nf as:</p>
<pre><code>process lancet {
        maxForks 3
        debug true

        input:
        tuple val(sample_id_tumor),path(bqsrbam_tumor)
        tuple val(sample_id_normal),path(bqsrbam_normal)
        val chromosome

        output:
        path &quot;CHR<span class="math-container">${chromosome}_$</span>{sample_name}.lancet.vcf&quot;

        script:

        sample_name=sample_id_normal.split('_normal')[0]

&quot;&quot;&quot;
ml lancet

lancet --tumor <span class="math-container">${bqsrbam_tumor} --normal $</span>{bqsrbam_normal} --ref <span class="math-container">$params.hg38genome --reg "chr"$</span>chromosome --num-threads 8 &gt; &quot;CHR&quot;<span class="math-container">${chromosome}"_$</span>{sample_name}.lancet.vcf&quot; 
&quot;&quot;&quot;

}

workflow.onComplete {
          log.info(workflow.success ? &quot;Done lancet!&quot; : &quot;Oops .. something went wrong in lancet&quot; )
 }
</code></pre>
<p>I have main.nf as:</p>
<pre><code>params.outdir_index=&quot;/sc/arion/output_pipeline_nextflow/test_tiny_datasets/bwa_index&quot;
params.raw_files_normal=&quot;/sc/arion/tiny/all_tumors_normals/*_normal_*{R1,R2}.fastq&quot;
params.raw_files_tumor=&quot;/sc/arion/tiny/all_tumors_normals/*_tumor_*{R1,R2}.fastq&quot;

include { FASTP} from './fastp_process.nf'
include {bwa_index} from './index_process.nf'

include { align_bwa_mem} from './bwamem_process_already_index.nf'
include { gatk_markduplicates} from './gatk_markduplicates_process.nf'
include {setupnmdtags} from './setupnmdtags_process.nf'
include { recalibrate_bam } from './recalibratebam_process.nf'
include { applybqsr } from './applybqsr_process.nf'
include { mutect2 } from './mutect2_process.nf'
include { lancet } from './lancet_process.nf'

workflow {

        normal_read_pairs=Channel.fromFilePairs(params.raw_files_normal)
        tumor_read_pairs=Channel.fromFilePairs(params.raw_files_tumor)
        //FASTP   align_bwa_mem        gatk_markduplicates         setupnmdtags        recalibrate_bam 
        
        applybqsr(gatk_markduplicates.out.sorted_bam_mark_duplicates_tumor,recalibrate_bam.out.recalib_table_tumor,gatk_markduplicates.out.sorted_bam_mark_duplicates_normal,recalibrate_bam.out.recalib_table_normal)

        mutect2(applybqsr.out.recal_bam_tumor,applybqsr.out.recal_bam_normal)
        
        chr=Channel.of( 1..22, 'X', 'Y' )
        lancet(applybqsr.out.recal_bam_tumor,applybqsr.out.recal_bam_normal,chr)
}
</code></pre>
<p>Lancet runs fine if I run it hard coded into a process code block with same input (tumor-normal bam) and human genome FASTA. How do I enable index files to be staged into the lancet process generated one step before?</p>

<nextflow>
nextflow: unable to find index
# nextflow: unable to find index

(Asked by: username_0 on 2023-07-17)

I've a pipeline with many processes.
One of the steps is to run lancet tool. Lancet tool runs per chromosome (1-22, x,y). The input in the lancet step is output from the applybqsr (gatk).
There are tumor and normal bam generated along with the index. human genome FASTA has a index. When I run lancet through the main.nf I run into an error

ERROR: index not found for BAM file

I've a config file as:
params {

        hg38genome="/sc/arion/reference_genome/human/hg38/Homo_sapiens_assembly38.fasta.gz"
        outdir = '/sc/arion/output_pipeline_nextflow/test_tiny_datasets/'
        dbsnp="/sc/arion/projects/Daphni2_Pipeline_resources/resourceDBsnp146/dbsnp_146.hg38.vcf.gz"
        target="/sc/arion/projects/Twist_Exome_Target_hg38.padded.interval_list.bed/Twist_Exome_Target_hg38.padded.interval_list.bed"

}
        withName: applybqsr {
        publishDir = [
                [  
                path: "${params.outdir}/applybqsr/normal/",
                mode: 'copy',
                pattern:'*normal*'
                ],
                [
                path: "${params.outdir}/applybqsr/tumor/",
                mode: 'copy',
                pattern:'*tumor*'
                ]       
        ]
        }
        //applybqsr ends        

        
        withName: lancet {
        publishDir = [  
                path: "${params.outdir}/lancet/",
                mode: 'copy'
        ]
        }
        //lancet ends

I have applybqsr as:
process applybqsr{

        maxForks 3
        debug true

        memory '6 GB'

        input:
        tuple val(sample_id_tumor), path(markbam_tumor)
        tuple val(sample_id_tumor), path(recaltable_tumor)
        tuple val(sample_id_normal), path(markbam_normal)
        tuple val(sample_id_normal), path(recaltable_normal)

        output:
        tuple val(sample_id_tumor), path("${sample_id_tumor}.sorted.markdup.recal.bam"), emit: recal_bam_tumor
        tuple val(sample_id_tumor), path("${sample_id_tumor}.sorted.markdup.recal.bai"),  emit: recal_bai_tumor
        tuple val(sample_id_normal), path("${sample_id_normal}.sorted.markdup.recal.bam"), emit: recal_bam_normal
        tuple val(sample_id_normal), path("${sample_id_normal}.sorted.markdup.recal.bai"),  emit: recal_bai_normal
        
        script:
        """
        ml gatk/4.1.3.0 samtools/1.9
        samtools index ${markbam_tumor}
        samtools index ${markbam_normal}

        gatk ApplyBQSR \\
        -R $params.hg38genome \\
        -I ${markbam_tumor} \\
        --bqsr-recal-file ${recaltable_tumor} \\
        -O ${sample_id_tumor}.sorted.markdup.recal.bam \\
        -L $params.target \\
        --create-output-bam-index true \\
        --java-options -Xmx${task.memory.toGiga()}g

        gatk ApplyBQSR \\
        -R $params.hg38genome \\
        -I ${markbam_normal} \\
        --bqsr-recal-file ${recaltable_normal} \\
        -O ${sample_id_normal}.sorted.markdup.recal.bam \\
        -L $params.target \\
        --create-output-bam-index true \\
        --java-options -Xmx${task.memory.toGiga()}g


        """

}
workflow.onComplete { 
        log.info ( workflow.success ? "Done applyBQSR!" : "Oops .. something went wrong in applyBQSR" )
}

I've lancet.nf as:
process lancet {
        maxForks 3
        debug true

        input:
        tuple val(sample_id_tumor),path(bqsrbam_tumor)
        tuple val(sample_id_normal),path(bqsrbam_normal)
        val chromosome

        output:
        path "CHR${chromosome}_${sample_name}.lancet.vcf"

        script:

        sample_name=sample_id_normal.split('_normal')[0]

"""
ml lancet

lancet --tumor ${bqsrbam_tumor} --normal ${bqsrbam_normal} --ref $params.hg38genome --reg "chr"$chromosome --num-threads 8 > "CHR"${chromosome}"_${sample_name}.lancet.vcf" 
"""

}

workflow.onComplete {
          log.info(workflow.success ? "Done lancet!" : "Oops .. something went wrong in lancet" )
 }

I have main.nf as:
params.outdir_index="/sc/arion/output_pipeline_nextflow/test_tiny_datasets/bwa_index"
params.raw_files_normal="/sc/arion/tiny/all_tumors_normals/*_normal_*{R1,R2}.fastq"
params.raw_files_tumor="/sc/arion/tiny/all_tumors_normals/*_tumor_*{R1,R2}.fastq"

include { FASTP} from './fastp_process.nf'
include {bwa_index} from './index_process.nf'

include { align_bwa_mem} from './bwamem_process_already_index.nf'
include { gatk_markduplicates} from './gatk_markduplicates_process.nf'
include {setupnmdtags} from './setupnmdtags_process.nf'
include { recalibrate_bam } from './recalibratebam_process.nf'
include { applybqsr } from './applybqsr_process.nf'
include { mutect2 } from './mutect2_process.nf'
include { lancet } from './lancet_process.nf'

workflow {

        normal_read_pairs=Channel.fromFilePairs(params.raw_files_normal)
        tumor_read_pairs=Channel.fromFilePairs(params.raw_files_tumor)
        //FASTP   align_bwa_mem        gatk_markduplicates         setupnmdtags        recalibrate_bam 
        
        applybqsr(gatk_markduplicates.out.sorted_bam_mark_duplicates_tumor,recalibrate_bam.out.recalib_table_tumor,gatk_markduplicates.out.sorted_bam_mark_duplicates_normal,recalibrate_bam.out.recalib_table_normal)

        mutect2(applybqsr.out.recal_bam_tumor,applybqsr.out.recal_bam_normal)
        
        chr=Channel.of( 1..22, 'X', 'Y' )
        lancet(applybqsr.out.recal_bam_tumor,applybqsr.out.recal_bam_normal,chr)
}

Lancet runs fine if I run it hard coded into a process code block with same input (tumor-normal bam) and human genome FASTA. How do I enable index files to be staged into the lancet process generated one step before?

**Tags:** nextflow

---

## 1 answers

### Answer by username_1 on 2023-07-18. Score: 2 | Accepted answer

I am reluctant to go through this wall of code, but generally it is advisable to always keep bam files and its index in a tuple so the index is staged with its bam file.
For example a process that creates the bam and now emits a tuple with bam and bai file:
(...)
output:
tuple path("${id}.bam"), path("${id}.bam.bai"), emit: tuple_bam

and then a process that receives such a tuple_bam:
(...)
input:
tuple path(bam), path(bai)

The first entry is the bam, the second the index.

> **Comments:**
> **username_0**: Thanks it worked.  I cannot understand why only one chromosome is being processed in lancet despite a channel being passed.  

---

