Windowed Adaptive Trimming for fastq files using quality (paired-end)
@E00361:399:HFGF2CCX2:8:1101:4909:2118 1:N:0:GGACTCCT+GCGATCTA
NTGTTAAGTTCATCTAAGTAATAAGTAATTGCAATTGTTGTAAAGCACTAGGATTATTTACATAAACTGTCTTTGTAATAAAATAAAATATAAGAATAAGAGTTAGCTGATTAATATTTAGTTACGTACCTTTCAACATACAAAATTAATC
+
#<AAFAAA7F-77FAJJJJJJFFJJJAJFJJJJJ--FFF--<FFJ-FFJJF-F<JJJ<FJJAJAAFJJJJJ-F<JFJJJJJJJFJFJJJJJJJ-<J-<7-AA-FFF-FA7J--------A7-7FA-AF---7-A<-FFFFFA7F7-7AF--
@E00361:399:HFGF2CCX2:8:1101:6898:2118 1:N:0:GGACTCCT+GCGATCTA
NTCTTATTCTTTTTTACAAGCTCATTTTGAAGTCTCATCAACTCTTCTACTTTGGAGCTAAATCGCCTGAATGACTGTCCCATTCAGATAAATGGTTGACCTCTGGAGTAAATAGATTTTTGACCTTTGCAGATACATGCATGGATACTAG
+
#<AAFJJJJJFJJFJJJJJJJJJJJJ-J-FJJJJJJFFJFJFJFJ-A-<JJ-JFFFJJJJJJJJAJJJJJJJJJJJJJJJJJJJFJJA<JJ<FJ-<-AJ--AJJFFJA-7F-<F7F-FJJJFFAF<J<AJJ77A--AF--AJJJJA-AAF<
@E00361:399:HFGF2CCX2:8:1101:7466:2118 1:N:0:GGACTCCT+GCGATCTA
NTTTTATAGAATATTTCTAGTACGATATCGAAATTGTGACAAAGCAATTCCCCCTCCATGCTCTTTCGATTTGTGAATTTAAGGCACCCCATAAACGAGACTTATTATACTCTCAAAAAATTAAAATAATAAATCATTGGAAATCAGAAAT
@E00361:399:HFGF2CCX2:8:1101:4909:2118 2:N:0:GGACTCCT+GCGATCTA
GGATAATGTCTCATTCTACAACAATGGTTGTTGCGCGATGGATTATAAGTTTAATTGTGTATGTTGAAAGGTAAGTGACTATATATTAATCAACTAACAAGTCTGCGGAGATGTGCTTTGAGTACAAAGACAGTTTCTGTAAATAATTCTA
+
A<AFF<FJJAAFAJJJ<JJFJ77FJJFJ--FFFJF<--7AFFAF-7-7F-7F--7--77-FJJJJJFJ<FFJ<FJJ--FA--7FAFA--7A---77--7--7---7)))))--7--7A-7---7------------------------7--
@E00361:399:HFGF2CCX2:8:1101:6898:2118 2:N:0:GGACTCCT+GCGATCTA
ATCCAAAGCCTGTCTTTTAACATATATTGTGAACACTACATATTTGAATTACACATGTTTTTGAGAAACTGGGTGGGTTAATTAGCAAGTATAGATGGATGTATTTGCAGAGGGCAAAAATATATTTACTCTAGTGGTCAAACATTTATAT
+
AAAFF-<JJ<AJFJAJJJJJJJJJJJJJJJJJJJF-J----<FFJFJJJA-<FJJJ-F-JJJJJ-FJ-AF7<-7A-7-FFJ<-<F--7A-77-77----77A<<-7A-7--7))-))-7---7---AF7-7-7----7)----7-77A--7
@E00361:399:HFGF2CCX2:8:1101:7466:2118 2:N:0:GGACTCCT+GCGATCTA
GTAATGGATTAAACTAATTTTCATTATTTTGTATGGGATAACATTATTCGGAGTTCGTACACCAATTTAGAAAGGAGGATGGTGAGAGTCCTAGGTTCCACTGTAATTTCAATTTCTCATTTACAATGATCAATTACTTTAATTTTTTGAG
trimming~sickle-pe -c 8 -m 32 input_1/test_1.fq input_2/test_2.fq
@E00361:399:HFGF2CCX2:8:1101:4909:2118 1:N:0:GGACTCCT+GCGATCTA
TGTTAAGTTCATCTAAGTAATAAGTAATTGCAATTGTTGTAAAGCACTAGGATTATTTACATAAACTGTCTTTGTAATAAAATAAAATATAAGAATAAGAGTTAG
+
<AAFAAA7F-77FAJJJJJJFFJJJAJFJJJJJ--FFF--<FFJ-FFJJF-F<JJJ<FJJAJAAFJJJJJ-F<JFJJJJJJJFJFJJJJJJJ-<J-<7-AA-FFF
@E00361:399:HFGF2CCX2:8:1101:6898:2118 1:N:0:GGACTCCT+GCGATCTA
TCTTATTCTTTTTTACAAGCTCATTTTGAAGTCTCATCAACTCTTCTACTTTGGAGCTAAATCGCCTGAATGACTGTCCCATTCAGATAAATGGTTGACCTCTGGAGTAAATAGATTTTTGACCTTTGCAGATACATGCATGGATACTAG
+
<AAFJJJJJFJJFJJJJJJJJJJJJ-J-FJJJJJJFFJFJFJFJ-A-<JJ-JFFFJJJJJJJJAJJJJJJJJJJJJJJJJJJJFJJA<JJ<FJ-<-AJ--AJJFFJA-7F-<F7F-FJJJFFAF<J<AJJ77A--AF--AJJJJA-AAF<
@E00361:399:HFGF2CCX2:8:1101:7466:2118 1:N:0:GGACTCCT+GCGATCTA
TTTTATAGAATATTTCTAGTACGATATCGAAATTGTGACAAAGCAATTCCCCCTCCATGCTCTTTCGATTTGTGAATTTAAGGCACCCCATAAACGAGACTTATTATACTCTCAAAAAATTAAAATAATAAATCATTGGAAATCAGAAAT
@E00361:399:HFGF2CCX2:8:1101:4909:2118 2:N:0:GGACTCCT+GCGATCTA
GGATAATGTCTCATTCTACAACAATGGTTGTTGCGCGATGGATT
+
A<AFF<FJJAAFAJJJ<JJFJ77FJJFJ--FFFJF<--7AFFAF
@E00361:399:HFGF2CCX2:8:1101:6898:2118 2:N:0:GGACTCCT+GCGATCTA
ATCCAAAGCCTGTCTTTTAACATATATTGTGAACACTACATATTTGAATTACACATGTTTTTGAGAAACTGGGTGGGTTAAT
+
AAAFF-<JJ<AJFJAJJJJJJJJJJJJJJJJJJJF-J----<FFJFJJJA-<FJJJ-F-JJJJJ-FJ-AF7<-7A-7-FFJ<
@E00361:399:HFGF2CCX2:8:1101:7466:2118 2:N:0:GGACTCCT+GCGATCTA
GTAATGGATTAAACTAATTTTCATTATTTTGTATGGGATAACATTATTCGGAGTTCGTACACCAATTTAGAAAGGAGGATGGTGAGAGTCCTAGGTT
@E00361:399:HFGF2CCX2:8:1101:20750:2135 1:N:0:GGACTCCT+GCGATCTA
TTAGGCAACACAAGGCCCCCATCCCATTTGCTTCTTTGCAGTGCTGCTCTACATGCTCTTGGGTTCGTGCCAGCCCAAATAAACTATGAAATTGTTTTGTCTAACAATCGAATAAACAATTTTGGTAGGGATATTGGATGACAATGGAAA
+
A-A-77FJ<JJA7F<JJJJJJJJJJ<-<FFJJFJJAJF-<FF-77FFAAFFFFJFFFJ7FJJJJJJFJJAJFJJJJF<<FAF-A7<JJ7F<FJF-7AAF-<AJFFJF-<FJ-F-7-7-777F<J7<-<F--<-77AFJ-AFJJ7AAF<FJ
@E00361:399:HFGF2CCX2:8:1101:3802:2170 1:N:0:GGACTCCT+GCGATCTA
ATCTTATCCACCTCTTTCTTTTTCAATAACCTCCACTGGTCCGTCATTCTCGCTCCAGTCTACGGAGGTGGTTGCGGAGGTTGGAGACA
+
AA<<AFJFJAJJ7-FFAJFJJJAJJJFJF<JFJJ7F-FJ-7-7--7-<F--<J7-<<-<JJ<-A<FJA-A7-AF77-FF-A<<---AJ<
@E00361:399:HFGF2CCX2:8:1101:28270:2188 1:N:0:GGACTCCT+GCGATCTA
GTATGACGATTGTTACGTGAAATTGTTCGTGGAATTTTTCCGGTAATACTTTCCTACAAAAAAATATAATTTTATCTTATAATTGAAAAAAACCTAAAAGCAGATCTTTATCGTTGTTGCTGCATATAGCGATAAGATGTACAAATGCGCG
pp trimming~sickle-pe -c 8 -m 32 input_1/test_1.fq input_2/test_2.fq PID: 1280203 /home/yoshitake.kazutoshi/files/m256y/pp-dev/yoshitake/PortablePipeline/PortablePipeline/scripts/pp 'trimming~sickle-pe' -c 8 -m 32 input_1/test_1.fq input_2/test_2.fq Checking the realpath of input files. 1 script: /suikou/files/m256y/yoshitake.kazutoshi/work/pp-dev/yoshitake/PortablePipeline/PortablePipeline/scripts/trimming~sickle-pe Containers: centos:centos6 quay.io/biocontainers/sickle-trim:1.33--2 using docker ++ set -o pipefail + set -eux + set -o pipefail ++ echo input_1/test_1.fq ++ grep '[.]gz$' ++ wc -l ++ true + '[' 0 = 1 ']' ++ grep '[.]gz$' /suikou/files/m256y/yoshitake.kazutoshi/work/pp-dev/yoshitake/PortablePipeline/PortablePipeline/scripts/trimming~sickle-pe: 行 25: input_2: 未割り当ての変数です ++ wc -l ++ true + '[' 0 = 1 ']' /suikou/files/m256y/yoshitake.kazutoshi/work/pp-dev/yoshitake/PortablePipeline/PortablePipeline/scripts/trimming~sickle-pe: 行 26: input_2: 未割り当ての変数です PID: 1280988 /home/yoshitake.kazutoshi/files/m256y/pp-dev/yoshitake/PortablePipeline/PortablePipeline/scripts/pp 'trimming~sickle-pe' -c 8 -m 32 input_1/test_1.fq input_2/test_2.fq Checking the realpath of input files. 1 script: /suikou/files/m256y/yoshitake.kazutoshi/work/pp-dev/yoshitake/PortablePipeline/PortablePipeline/scripts/trimming~sickle-pe Containers: centos:centos6 quay.io/biocontainers/sickle-trim:1.33--2 using docker ++ set -o pipefail + set -eux + set -o pipefail ++ echo input_1/test_1.fq ++ grep '[.]gz$' ++ wc -l ++ true + '[' 0 = 1 ']' ++ echo input_2/test_2.fq ++ grep '[.]gz$' ++ wc -l ++ true + '[' 0 = 1 ']' + FUNC_RUN_DOCKER quay.io/biocontainers/sickle-trim:1.33--2 sickle pe -f input_1/test_1.fq -r input_2/test_2.fq -o output.sickle_1.fastq -p output.sickle_2.fastq -s + PP_RUN_IMAGE=quay.io/biocontainers/sickle-trim:1.33--2 + shift + PP_RUN_DOCKER_CMD=("${@}") ++ date +%Y%m%d_%H%M%S_%3N + PPDOCNAME=pp20241105_094124_413_17962 + echo pp20241105_094124_413_17962 ++ id -u ++ id -g + docker run --name pp20241105_094124_413_17962 -v /suikou/files/m256y/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/trimming~sickle-pe:/suikou/files/m256y/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/trimming~sickle-pe -w /suikou/files/m256y/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/trimming~sickle-pe -v /suikou/files/m256y/yoshitake.kazutoshi:/suikou/files/m256y/yoshitake.kazutoshi -u 2007:600 -i --rm quay.io/biocontainers/sickle-trim:1.33--2 sickle pe -f input_1/test_1.fq -r input_2/test_2.fq -o output.sickle_1.fastq -p output.sickle_2.fastq -s sickle: option requires an argument -- 's' If you have separate files for forward and reverse reads: Usage: sickle pe [options] -f-r -t -o -p -s If you have one file with interleaved forward and reverse reads: Usage: sickle pe [options] -c -t -m -s If you have one file with interleaved reads as input and you want ONLY one interleaved file as output: Usage: sickle pe [options] -c -t -M Options: Paired-end separated reads -------------------------- -f, --pe-file1, Input paired-end forward fastq file (Input files must have same number of records) -r, --pe-file2, Input paired-end reverse fastq file -o, --output-pe1, Output trimmed forward fastq file -p, --output-pe2, Output trimmed reverse fastq file. Must use -s option. Paired-end interleaved reads ---------------------------- -c, --pe-combo, Combined (interleaved) input paired-end fastq -m, --output-combo, Output combined (interleaved) paired-end fastq file. Must use -s option. -M, --output-combo-all, Output combined (interleaved) paired-end fastq file with any discarded read written to output file as a single N. Cannot be used with the -s option. Global options -------------- -t, --qual-type, Type of quality values (solexa (CASAVA < 1.3), illumina (CASAVA 1.3 to 1.7), sanger (which is CASAVA >= 1.8)) (required) -s, --output-single, Output trimmed singles fastq file -q, --qual-threshold, Threshold for trimming based on average quality in a window. Default 20. -l, --length-threshold, Threshold to keep a read based on length after trimming. Default 20. -x, --no-fiveprime, Don't do five prime trimming. -n, --truncate-n, Truncate sequences at position of first N. -g, --gzip-output, Output gzipped files. --quiet, do not output trimming info --help, display this help and exit --version, output version information and exit PID: 1282987 /home/yoshitake.kazutoshi/files/m256y/pp-dev/yoshitake/PortablePipeline/PortablePipeline/scripts/pp 'trimming~sickle-pe' -c 8 -m 32 input_1/test_1.fq input_2/test_2.fq Checking the realpath of input files. 1 script: /suikou/files/m256y/yoshitake.kazutoshi/work/pp-dev/yoshitake/PortablePipeline/PortablePipeline/scripts/trimming~sickle-pe Containers: centos:centos6 quay.io/biocontainers/sickle-trim:1.33--2 using docker ++ set -o pipefail + set -eux + set -o pipefail ++ echo input_1/test_1.fq ++ grep '[.]gz$' ++ wc -l ++ true + '[' 0 = 1 ']' ++ echo input_2/test_2.fq ++ grep '[.]gz$' ++ wc -l ++ true + '[' 0 = 1 ']' + FUNC_RUN_DOCKER quay.io/biocontainers/sickle-trim:1.33--2 sickle pe -f input_1/test_1.fq -r input_2/test_2.fq -o output.sickle_1.fastq -p output.sickle_2.fastq + PP_RUN_IMAGE=quay.io/biocontainers/sickle-trim:1.33--2 + shift + PP_RUN_DOCKER_CMD=("${@}") ++ date +%Y%m%d_%H%M%S_%3N + PPDOCNAME=pp20241105_094317_831_6162 + echo pp20241105_094317_831_6162 ++ id -u ++ id -g + docker run --name pp20241105_094317_831_6162 -v /suikou/files/m256y/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/trimming~sickle-pe:/suikou/files/m256y/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/trimming~sickle-pe -w /suikou/files/m256y/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/trimming~sickle-pe -v /suikou/files/m256y/yoshitake.kazutoshi:/suikou/files/m256y/yoshitake.kazutoshi -u 2007:600 -i --rm quay.io/biocontainers/sickle-trim:1.33--2 sickle pe -f input_1/test_1.fq -r input_2/test_2.fq -o output.sickle_1.fastq -p output.sickle_2.fastq If you have separate files for forward and reverse reads: Usage: sickle pe [options] -f -r -t -o -p -s If you have one file with interleaved forward and reverse reads: Usage: sickle pe [options] -c -t -m -s If you have one file with interleaved reads as input and you want ONLY one interleaved file as output: Usage: sickle pe [options] -c -t -M Options: Paired-end separated reads -------------------------- -f, --pe-file1, Input paired-end forward fastq file (Input files must have same number of records) -r, --pe-file2, Input paired-end reverse fastq file -o, --output-pe1, Output trimmed forward fastq file -p, --output-pe2, Output trimmed reverse fastq file. Must use -s option. Paired-end interleaved reads ---------------------------- -c, --pe-combo, Combined (interleaved) input paired-end fastq -m, --output-combo, Output combined (interleaved) paired-end fastq file. Must use -s option. -M, --output-combo-all, Output combined (interleaved) paired-end fastq file with any discarded read written to output file as a single N. Cannot be used with the -s option. Global options -------------- -t, --qual-type, Type of quality values (solexa (CASAVA < 1.3), illumina (CASAVA 1.3 to 1.7), sanger (which is CASAVA >= 1.8)) (required) -s, --output-single, Output trimmed singles fastq file -q, --qual-threshold, Threshold for trimming based on average quality in a window. Default 20. -l, --length-threshold, Threshold to keep a read based on length after trimming. Default 20. -x, --no-fiveprime, Don't do five prime trimming. -n, --truncate-n, Truncate sequences at position of first N. -g, --gzip-output, Output gzipped files. --quiet, do not output trimming info --help, display this help and exit --version, output version information and exit ****Error: Quality type is required. PID: 1283755 /home/yoshitake.kazutoshi/files/m256y/pp-dev/yoshitake/PortablePipeline/PortablePipeline/scripts/pp 'trimming~sickle-pe' -c 8 -m 32 input_1/test_1.fq input_2/test_2.fq Checking the realpath of input files. 1 script: /suikou/files/m256y/yoshitake.kazutoshi/work/pp-dev/yoshitake/PortablePipeline/PortablePipeline/scripts/trimming~sickle-pe Containers: centos:centos6 quay.io/biocontainers/sickle-trim:1.33--2 using docker ++ set -o pipefail + set -eux + set -o pipefail ++ echo input_1/test_1.fq ++ grep '[.]gz$' ++ wc -l ++ true + '[' 0 = 1 ']' ++ echo input_2/test_2.fq ++ grep '[.]gz$' ++ wc -l ++ true + '[' 0 = 1 ']' + FUNC_RUN_DOCKER quay.io/biocontainers/sickle-trim:1.33--2 sickle pe -f input_1/test_1.fq -r input_2/test_2.fq -o output.sickle_1.fastq -p output.sickle_2.fastq -s output.sickle_single.fastq + PP_RUN_IMAGE=quay.io/biocontainers/sickle-trim:1.33--2 + shift + PP_RUN_DOCKER_CMD=("${@}") ++ date +%Y%m%d_%H%M%S_%3N + PPDOCNAME=pp20241105_094347_116_15584 + echo pp20241105_094347_116_15584 ++ id -u ++ id -g + docker run --name pp20241105_094347_116_15584 -v /suikou/files/m256y/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/trimming~sickle-pe:/suikou/files/m256y/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/trimming~sickle-pe -w /suikou/files/m256y/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/trimming~sickle-pe -v /suikou/files/m256y/yoshitake.kazutoshi:/suikou/files/m256y/yoshitake.kazutoshi -u 2007:600 -i --rm quay.io/biocontainers/sickle-trim:1.33--2 sickle pe -f input_1/test_1.fq -r input_2/test_2.fq -o output.sickle_1.fastq -p output.sickle_2.fastq -s output.sickle_single.fastq If you have separate files for forward and reverse reads: Usage: sickle pe [options] -f -r -t -o -p -s If you have one file with interleaved forward and reverse reads: Usage: sickle pe [options] -c -t -m -s If you have one file with interleaved reads as input and you want ONLY one interleaved file as output: Usage: sickle pe [options] -c -t -M Options: Paired-end separated reads -------------------------- -f, --pe-file1, Input paired-end forward fastq file (Input files must have same number of records) -r, --pe-file2, Input paired-end reverse fastq file -o, --output-pe1, Output trimmed forward fastq file -p, --output-pe2, Output trimmed reverse fastq file. Must use -s option. Paired-end interleaved reads ---------------------------- -c, --pe-combo, Combined (interleaved) input paired-end fastq -m, --output-combo, Output combined (interleaved) paired-end fastq file. Must use -s option. -M, --output-combo-all, Output combined (interleaved) paired-end fastq file with any discarded read written to output file as a single N. Cannot be used with the -s option. Global options -------------- -t, --qual-type, Type of quality values (solexa (CASAVA < 1.3), illumina (CASAVA 1.3 to 1.7), sanger (which is CASAVA >= 1.8)) (required) -s, --output-single, Output trimmed singles fastq file -q, --qual-threshold, Threshold for trimming based on average quality in a window. Default 20. -l, --length-threshold, Threshold to keep a read based on length after trimming. Default 20. -x, --no-fiveprime, Don't do five prime trimming. -n, --truncate-n, Truncate sequences at position of first N. -g, --gzip-output, Output gzipped files. --quiet, do not output trimming info --help, display this help and exit --version, output version information and exit ****Error: Quality type is required. PID: 1284909 /home/yoshitake.kazutoshi/files/m256y/pp-dev/yoshitake/PortablePipeline/PortablePipeline/scripts/pp 'trimming~sickle-pe' -c 8 -m 32 input_1/test_1.fq input_2/test_2.fq Checking the realpath of input files. 1 script: /suikou/files/m256y/yoshitake.kazutoshi/work/pp-dev/yoshitake/PortablePipeline/PortablePipeline/scripts/trimming~sickle-pe Containers: centos:centos6 quay.io/biocontainers/sickle-trim:1.33--2 using docker ++ set -o pipefail + set -eux + set -o pipefail ++ echo input_1/test_1.fq ++ grep '[.]gz$' ++ wc -l ++ true + '[' 0 = 1 ']' ++ echo input_2/test_2.fq ++ grep '[.]gz$' ++ wc -l ++ true + '[' 0 = 1 ']' + FUNC_RUN_DOCKER quay.io/biocontainers/sickle-trim:1.33--2 sickle pe -f input_1/test_1.fq -r input_2/test_2.fq -t sanger -o output.sickle_1.fastq -p output.sickle_2.fastq -s output.sickle_single.fastq + PP_RUN_IMAGE=quay.io/biocontainers/sickle-trim:1.33--2 + shift + PP_RUN_DOCKER_CMD=("${@}") ++ date +%Y%m%d_%H%M%S_%3N + PPDOCNAME=pp20241105_094417_555_31823 + echo pp20241105_094417_555_31823 ++ id -u ++ id -g + docker run --name pp20241105_094417_555_31823 -v /suikou/files/m256y/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/trimming~sickle-pe:/suikou/files/m256y/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/trimming~sickle-pe -w /suikou/files/m256y/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/trimming~sickle-pe -v /suikou/files/m256y/yoshitake.kazutoshi:/suikou/files/m256y/yoshitake.kazutoshi -u 2007:600 -i --rm quay.io/biocontainers/sickle-trim:1.33--2 sickle pe -f input_1/test_1.fq -r input_2/test_2.fq -t sanger -o output.sickle_1.fastq -p output.sickle_2.fastq -s output.sickle_single.fastq FastQ paired records kept: 19260 (9630 pairs) FastQ single records kept: 369 (from PE1: 369, from PE2: 0) FastQ paired records discarded: 2 (1 pairs) FastQ single records discarded: 369 (from PE1: 0, from PE2: 369) + post_processing + '[' 1 = 1 ']' + rm -f /home/yoshitake.kazutoshi/files/m256y/pp-dev/yoshitake/test/trimming~sickle-pe/pp-singularity-flag + '[' '' = y ']' + echo 0 + exit