Pipeline Outputs¶

This page describes all output files generated by the MAD4HATTER pipeline, organized by output type.

Output Structure¶

Below you will find the folder structure outputs from each of the different workflow options (complete,qc,postprocessing).

Complete Workflow Outputs¶

results/
├── allele_data.txt                    # Main allele table (unmasked and masked sequences)
├── allele_data_collapsed.txt          # Collapsed allele table (masked sequences only)
├── sample_coverage.txt                # Sample-level coverage statistics
├── amplicon_coverage.txt              # Amplicon-level coverage statistics
├── raw_dada2_output/
│   └── dada2.clusters.txt             # Raw DADA2 output (ASV sequences and counts)
├── resistance_marker_module/          # Resistance marker analysis outputs (if enabled)
│   ├── resmarker_table.txt
│   ├── resmarker_table_by_locus.txt
│   ├── resmarker_microhaplotype_table.txt
│   └── all_mutations_table.txt
├── panel_information/                 # Panel configuration files used
│   ├── amplicon_info.tsv
│   ├── reference.fasta
│   └── resmarker_info.tsv
└── run/                               # Run metadata
    ├── parameters.tsv                 # Parameters used in this run
    └── runtime.tsv                    # Runtime information

QC Output Files¶

results/
├── sample_coverage.txt                # Sample-level coverage statistics
├── amplicon_coverage.txt              # Target-level coverage statistics
├── panel_information/                 # Panel configuration files used
│   └── amplicon_info.tsv
└── run/                               # Run metadata
    ├── parameters.tsv                 # Parameters used in this run
    └── runtime.tsv                    # Runtime information

Postprocessing Output Files¶

results/
├── allele_data.txt                    # Main allele table (unmasked and masked sequences)
├── allele_data_collapsed.txt          # Collapsed allele table (masked sequences only)
├── panel_information/                 # Panel configuration files used
│   ├── amplicon_info.tsv
│   └── reference.fasta
└── run/                               # Run metadata
    ├── parameters.tsv                 # Parameters used in this run
    └── runtime.tsv                    # Runtime information

Coverage Files¶

sample_coverage.txt¶

Description: Sample-level coverage statistics showing how many reads pass through each processing stage.

Columns:

Stages:

• Input: Starting number of reads from raw FASTQ files
• No Dimers: Reads remaining after removing Illumina adapter dimers
• Amplicons: Reads with expected primers attached (after cutadapt)

The following stages are only included if the complete workflow is run.

• OutputDada2: Number of denoised sequences (ASVs) after DADA2 processing
• OutputPostprocessing: Final number of sequences after alignment filtering

Example:

sample_name          stage              reads
SRR26819135_S1_L001  Input              50000
SRR26819135_S1_L001  No Dimers          49494
SRR26819135_S1_L001  Amplicons          40259
SRR26819135_S1_L001  OutputDada2        40217
SRR26819135_S1_L001  OutputPostprocessing 40217

amplicon_coverage.txt¶

Description: Amplicon-level coverage statistics showing read counts per target for each sample. The columns OutputDada2 and OutputPostprocessing will only be added if the complete workflow has been run.

Columns:

Example:

sample_name          target_name                        reads  OutputDada2  OutputPostprocessing
SRR26819135_S1_L001  Pf3D7_01_v3-0145420-0145630       54     54           54
SRR26819135_S1_L001  Pf3D7_01_v3-0162888-0163092       400    398          398

Allele Data Files¶

allele_data.txt¶

Description: This is a very informative output file containing all identified alleles (ASVs) with both unmasked and masked sequences, plus mutation annotations.

Columns:

Key Features:

• Unmasked columns (asv, pseudocigar_unmasked): Full sequence information before masking
• Masked columns (asv_masked, pseudocigar_masked): Sequences with low-complexity regions masked (used for downstream analysis). Note that asv_masked contains alignment information to allow for masking.
• PseudoCIGAR format: See PseudoCIGAR Format section below

Example:

sample_name          target_name                        asv                    pseudocigar_unmasked  asv_masked              pseudocigar_masked    reads  pool
SRR26819135_S1_L001  Pf3D7_01_v3-0145420-0145630        GATATGTTTAAATATA...    94A                    GATATGTTTAAATATA...    25+25N94A169+8N188+9N  54     D1.1

allele_data_collapsed.txt¶

Description: Simplified allele table containing only masked sequences amd PseudoCIGARS. This is a collapsed version of allele_data.txt that removes duplicate ASVs where they were only different due to suspected errors in the masked region.

Columns:

When to use: This file is useful when you only need the masked sequences and mutation annotations, without the full unmasked sequences.

Raw DADA2 Output¶

raw_dada2_output/dada2.clusters.txt¶

Description: Raw output from the DADA2 sequence inference step, before alignment and masking.

Columns:

When to use: This file contains the raw DADA2 output before any alignment or masking steps. Useful for understanding initial sequence inference results. This can also be used as input to the postprocessing workflow if you wish to reanalyze the data with alternate parameters.

Example:

sampleID            locus                           asv                    reads  allele                                    norm.reads.locus  n.alleles
SRR26819135_S1_L001 Pf3D7_01_v3-0145420-0145630    GATATGTTTAAATATA...    54     Pf3D7_01_v3-0145420-0145630.1              1                 1

PseudoCIGAR Format¶

The PseudoCIGAR string provides a compact representation of mutations and masked regions in ASV sequences relative to the reference.

Mutation Syntax¶

SNPs (Single Nucleotide Polymorphisms)¶

Format: {position}{base}

• position: Position in the reference sequence (1-based)
• base: The base in the ASV at this position (different from reference)

Example: 94A means the ASV has an A at position 94, while the reference has a different base.

Insertions¶

Format: {position}I={base}

• position: Position in the reference where the insertion occurs
• base: The base inserted in the ASV

Example: 10I=G means a G was inserted at position 10 in the ASV.

Deletions¶

Format: {position}D={base}

• position: Position in the reference sequence
• base: The base that was deleted from the ASV (exists in reference but not in ASV)

Example: 144D=TCT means the sequence TCT was deleted starting at position 144.

Mask Syntax¶

Format: {start_position}+{length}N

• start_position: Where the mask begins (1-based)
• length: Length of the masked region
• N: Indicates masked (low-complexity) region

Example: 25+25N means positions 25-49 (25 bases) are masked.

Multiple Mutations¶

Multiple mutations are included in the PseudoCIGAR string. For example:

25+25N94A169+8N188+9N

This means: - Positions 25-49 are masked (25 bases) - Position 94 has SNP A - Positions 169-176 are masked (8 bases) - Positions 188-196 are masked (9 bases)

Interpreting Masked Sequences¶

In asv_masked, masked regions are replaced with lowercase n characters:

Reference:  ACTTGATTGCACA
ASV:        ACTTGATTGCACA
Masked:     ACTnnnnnnnCACA  (positions 4-10 masked)
PseudoCIGAR: 4+7N

Resistance Marker Module Outputs¶

These files are generated when resistance marker analysis is enabled. They provide codon-level and mutation-level information for resistance-associated genes.

resmarker_table.txt¶

Description: Comprehensive resistance marker table showing codon-level information for all resistance-associated genes.

Columns:

Example:

sample_name          gene_id  gene  aa_position  ref_codon  codon  codon_ref_alt  ref_aa  aa  aa_ref_alt  follows_indel  codon_masked  multiple_loci  reads
SRR26819135_S1_L001  0417200  dhfr  16           GCA        GCA   REF            A       A   REF         False           False         False         127
SRR26819135_S1_L001  0417200  dhfr  51           AAT        ATT   ALT            N       I   ALT         False           False         False         127

resmarker_table_by_locus.txt¶

Description: Resistance marker table organized by target, showing which target each codon call comes from. If you have no overlapping targets then this information will be the same as the above file (resmarker_table.txt).

Columns:

Difference from resmarker_table.txt: This file includes target_name to show which target each codon call came from. This may be useful if you want to filter targets or collapse codons covered by multiple targets in a different way than the pipeline does (e.g., max or mean).

resmarker_microhaplotype_table.txt¶

Description: Table showing combinations of codons from the same microhaplotype for resistance markers of interest.

Columns:

Example:

sample_name          gene_id  gene  target_name                    mhap_aa_positions  ref_mhap  mhap    mhap_ref_alt  reads
SRR26819135_S1_L001  0417200  dhfr  Pf3D7_04_v3-0748128-0748326    16/51/59           A/N/C     A/I/R   ALT            381

all_mutations_table.txt¶

Description: Mutation table showing all single-nucleotide variants (SNVs) in targets that covered at least one marker of interest.

Columns:

When to use: This file provides a simple view of all mutations without codon/amino acid interpretation. Useful for looking for novel mutations, outside of markers of interest.

Example:

sample_name          gene_id  gene  target_name                    target_position  alt  ref  reads
SRR26819135_S1_L001  0417200  dhfr  Pf3D7_04_v3-0748128-0748326    110              T    A    127
SRR26819135_S1_L001  0417200  dhfr  Pf3D7_04_v3-0748128-0748326    133              C    T    127

Panel Information Files¶

panel_information/¶

This directory contains copies of the panel configuration files used during the run:

• amplicon_info.tsv: Target definitions (targets, primers, coordinates)
• reference.fasta: Reference sequences used for alignment
• resmarker_info.tsv: Resistance marker definitions

When to use: Useful when using multiple pre-configured pools. The pipeline will automatically generate these files from the list of pools. These files can be informative for downstream analysis and seeing which resmarkers were identified as being covered by targets.

Run Metadata Files¶

run/parameters.tsv¶

Description: Tab-separated file containing all parameters used in this pipeline run.

Format: Two columns: parameter name and parameter value.

Example:

pools                D1,R1,R2
readDIR              tests/example_data/example_fastq
workflow_name        complete
cutadapt_minlen      100
quality_score        20
...

When to use: Useful for reproducing runs or understanding what settings were used.

run/runtime.tsv¶

Description: Runtime information about the pipeline execution.

Columns:

When to use: Useful for tracking pipeline versions, runtime, and debugging issues.

Understanding Output File Relationships¶

Raw FASTQ files
    ↓
sample_coverage.txt (tracks reads through stages)
    ↓
raw_dada2_output/dada2.clusters.txt (DADA2 inference)
    ↓
allele_data.txt (alignment, masking, mutation calling)
    ↓
allele_data_collapsed.txt (simplified version)
    ↓
resistance_marker_module/ (if enabled)
    ├── resmarker_table.txt (codon-level)
    ├── resmarker_table_by_locus.txt (with target info)
    ├── resmarker_microhaplotype_table.txt (haplotypes)
    └── all_mutations_table.txt (discovery of novel mutations)

Next Steps¶

• Analyze results: Use the allele tables to identify variants in your samples
• Check quality: Review coverage files to ensure sufficient read depth
• Resistance analysis: If enabled, review resistance marker tables for drug resistance mutations
• Reproducibility: Use run/parameters.tsv to reproduce your analysis