[2024-11-05 00:27:49] root: INFO: Starting Flye 2.9.5-b1801 [2024-11-05 00:27:49] root: DEBUG: Cmd: /usr/local/bin/flye --nano-raw input_1//SRR27458461.fastq.gz --out-dir output --threads 8 --genome-size 10M [2024-11-05 00:27:49] root: DEBUG: Python version: 3.9.19 | packaged by conda-forge | (main, Mar 20 2024, 12:50:21) [GCC 12.3.0] [2024-11-05 00:27:49] root: INFO: >>>STAGE: configure [2024-11-05 00:27:49] root: INFO: Configuring run [2024-11-05 00:27:54] root: INFO: Total read length: 264541423 [2024-11-05 00:27:54] root: INFO: Input genome size: 10000000 [2024-11-05 00:27:54] root: INFO: Estimated coverage: 26 [2024-11-05 00:27:54] root: INFO: Reads N50/N90: 9108 / 4635 [2024-11-05 00:27:54] root: INFO: Minimum overlap set to 5000 [2024-11-05 00:27:54] root: INFO: >>>STAGE: assembly [2024-11-05 00:27:54] root: INFO: Assembling disjointigs [2024-11-05 00:27:54] root: DEBUG: -----Begin assembly log------ [2024-11-05 00:27:54] root: DEBUG: Running: flye-modules assemble --reads /data/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/assemble~flye/input_1/SRR27458461.fastq.gz --out-asm /data/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/assemble~flye/output/00-assembly/draft_assembly.fasta --config /usr/local/lib/python3.9/site-packages/flye/config/bin_cfg/asm_raw_reads.cfg --log /data/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/assemble~flye/output/flye.log --threads 8 --genome-size 10000000 --min-ovlp 5000 [2024-11-05 00:27:54] DEBUG: Build date: Aug 30 2024 21:35:52 [2024-11-05 00:27:54] DEBUG: Total RAM: 754 Gb [2024-11-05 00:27:54] DEBUG: Available RAM: 497 Gb [2024-11-05 00:27:54] DEBUG: Total CPUs: 16 [2024-11-05 00:27:54] DEBUG: Loading /usr/local/lib/python3.9/site-packages/flye/config/bin_cfg/asm_raw_reads.cfg [2024-11-05 00:27:54] DEBUG: Loading /usr/local/lib/python3.9/site-packages/flye/config/bin_cfg/asm_defaults.cfg [2024-11-05 00:27:54] DEBUG: big_genome_threshold=29000000 [2024-11-05 00:27:54] DEBUG: meta_read_filter_kmer_freq=100 [2024-11-05 00:27:54] DEBUG: chain_large_gap_penalty=2 [2024-11-05 00:27:54] DEBUG: chain_small_gap_penalty=0.5 [2024-11-05 00:27:54] DEBUG: chain_gap_jump_threshold=100 [2024-11-05 00:27:54] DEBUG: max_jump_gap=500 [2024-11-05 00:27:54] DEBUG: max_coverage_drop_rate=5 [2024-11-05 00:27:54] DEBUG: max_extensions_drop_rate=5 [2024-11-05 00:27:54] DEBUG: chimera_window=100 [2024-11-05 00:27:54] DEBUG: chimera_overhang=1000 [2024-11-05 00:27:54] DEBUG: min_reads_in_disjointig=4 [2024-11-05 00:27:54] DEBUG: max_inner_reads=10 [2024-11-05 00:27:54] DEBUG: max_inner_fraction=0.25 [2024-11-05 00:27:54] DEBUG: aggressive_dup_filter=1 [2024-11-05 00:27:54] DEBUG: max_separation=500 [2024-11-05 00:27:54] DEBUG: unique_edge_length=50000 [2024-11-05 00:27:54] DEBUG: min_repeat_res_support=0.51 [2024-11-05 00:27:54] DEBUG: out_paths_ratio=5 [2024-11-05 00:27:54] DEBUG: graph_cov_drop_rate=5 [2024-11-05 00:27:54] DEBUG: coverage_estimate_window=100 [2024-11-05 00:27:54] DEBUG: max_bubble_length=50000 [2024-11-05 00:27:54] DEBUG: loop_coverage_rate=1.5 [2024-11-05 00:27:54] DEBUG: repeat_edge_cov_mult=1.75 [2024-11-05 00:27:54] DEBUG: weak_detach_rate=5 [2024-11-05 00:27:54] DEBUG: tip_coverage_rate=2 [2024-11-05 00:27:54] DEBUG: tip_length_rate=2 [2024-11-05 00:27:54] DEBUG: output_gfa_before_rr=1 [2024-11-05 00:27:54] DEBUG: remove_alt_edges=0 [2024-11-05 00:27:54] DEBUG: low_cutoff_warning=1 [2024-11-05 00:27:54] DEBUG: kmer_size=17 [2024-11-05 00:27:54] DEBUG: use_minimizers=0 [2024-11-05 00:27:54] DEBUG: reads_base_alignment=0 [2024-11-05 00:27:54] DEBUG: meta_read_top_kmer_rate=0.40 [2024-11-05 00:27:54] DEBUG: maximum_jump=1500 [2024-11-05 00:27:54] DEBUG: maximum_overhang=1500 [2024-11-05 00:27:54] DEBUG: repeat_kmer_rate=100 [2024-11-05 00:27:54] DEBUG: assemble_ovlp_divergence=0.10 [2024-11-05 00:27:54] DEBUG: assemble_divergence_relative=1 [2024-11-05 00:27:54] DEBUG: repeat_graph_ovlp_divergence=0.08 [2024-11-05 00:27:54] DEBUG: read_align_ovlp_divergence=0.25 [2024-11-05 00:27:54] DEBUG: hpc_scoring_on=0 [2024-11-05 00:27:54] DEBUG: add_unassembled_reads=0 [2024-11-05 00:27:54] DEBUG: extend_contigs_with_repeats=0 [2024-11-05 00:27:54] DEBUG: min_read_cov_cutoff=3 [2024-11-05 00:27:54] DEBUG: short_tip_length=20000 [2024-11-05 00:27:54] DEBUG: long_tip_length=100000 [2024-11-05 00:27:54] DEBUG: Running with k-mer size: 17 [2024-11-05 00:27:54] DEBUG: Running with minimum overlap 5000 [2024-11-05 00:27:54] DEBUG: Metagenome mode: N [2024-11-05 00:27:54] DEBUG: Short mode: N [2024-11-05 00:27:54] INFO: Reading sequences [2024-11-05 00:27:57] DEBUG: Building positional index [2024-11-05 00:27:57] DEBUG: Total sequence: 231413392 bp [2024-11-05 00:27:58] INFO: Counting k-mers: [2024-11-05 00:28:05] DEBUG: Updating k-mer histogram [2024-11-05 00:29:03] DEBUG: Hash size: 2844562 [2024-11-05 00:29:03] DEBUG: Total k-mers 8833950 [2024-11-05 00:29:03] INFO: Filling index table (1/2) [2024-11-05 00:29:18] DEBUG: Mean k-mer frequency: 47.597 [2024-11-05 00:29:18] DEBUG: Repetitive k-mer frequency: 4759 [2024-11-05 00:29:18] DEBUG: Filtered 0 repetitive k-mers (0) [2024-11-05 00:29:19] INFO: Filling index table (2/2) [2024-11-05 00:29:35] DEBUG: Sorting k-mer index [2024-11-05 00:29:35] DEBUG: Selected k-mers: 2188124 [2024-11-05 00:29:35] DEBUG: Index size: 100518227 [2024-11-05 00:29:35] DEBUG: Mean k-mer index frequency: 45.9381 [2024-11-05 00:29:35] DEBUG: Peak RAM usage: 8 Gb [2024-11-05 00:29:35] DEBUG: Estimating k-mer identity bias [2024-11-05 00:29:59] DEBUG: Initial divergence estimate : 0.0437429 [2024-11-05 00:29:59] DEBUG: Relative threshold: Y [2024-11-05 00:29:59] DEBUG: Max divergence threshold set to 0.143743 [2024-11-05 00:29:59] INFO: Extending reads [2024-11-05 00:29:59] DEBUG: Estimating overlap coverage [2024-11-05 00:32:14] INFO: Overlap-based coverage: 73 [2024-11-05 00:32:14] INFO: Median overlap divergence: 0.0452029 [2024-11-05 00:32:14] DEBUG: Sequence divergence distribution: | * | | ** | | ** | | *** | | **** | | **** | | **** | | ***** | | ******* | | ******** | | ******** | | ********** | | ********** | | ********** | | ************ | | ************* | | ************** | | *************** | |***************** * | |****************************** **** *** * * ---------------------------------------------------------------------------------------------------- 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% Q25 = 0.031, Q50 = 0.045, Q75 = 0.056 [2024-11-05 00:32:55] DEBUG: Assembled disjointig 1 With 336 reads Start read: +SRR27458461.17827.1 At position: 74 leftTip: 1 rightTip: 1 Suspicious: 4 Short ext: 2 Mean extensions: 48 Avg overlap len: 11917 Min overlap len: 3493 Inner reads: 0 Length: 1107192 [2024-11-05 00:32:55] DEBUG: Inner: 16656 covered: 16724 total: 50858 [2024-11-05 00:32:55] DEBUG: Discarded disjointig with 168 reads and 166 inner overlaps [2024-11-05 00:32:55] DEBUG: Discarded disjointig with 192 reads and 190 inner overlaps [2024-11-05 00:34:02] DEBUG: Assembled disjointig 2 With 84 reads Start read: +SRR27458461.454.1 At position: 60 leftTip: 1 rightTip: 1 Suspicious: 2 Short ext: 1 Mean extensions: 55 Avg overlap len: 12241 Min overlap len: 1605 Inner reads: 0 Length: 271393 [2024-11-05 00:34:02] DEBUG: Inner: 20956 covered: 21056 total: 50858 [2024-11-05 00:34:02] DEBUG: Discarded disjointig with 618 reads and 84 inner overlaps [2024-11-05 00:34:02] DEBUG: Discarded disjointig with 641 reads and 106 inner overlaps [2024-11-05 00:34:02] DEBUG: Discarded disjointig with 617 reads and 84 inner overlaps [2024-11-05 00:34:02] DEBUG: Discarded disjointig with 637 reads and 106 inner overlaps [2024-11-05 00:34:03] DEBUG: Discarded disjointig with 24 reads and 22 inner overlaps [2024-11-05 00:34:03] DEBUG: Discarded disjointig with 622 reads and 84 inner overlaps [2024-11-05 00:34:03] DEBUG: Assembled disjointig 3 With 527 reads Start read: +SRR27458461.7235.1 At position: 107 leftTip: 0 rightTip: 0 Suspicious: 6 Short ext: 3 Mean extensions: 55 Avg overlap len: 12253 Min overlap len: 3289 Inner reads: 0 Length: 1649334 [2024-11-05 00:34:03] DEBUG: Inner: 49522 covered: 49608 total: 50858 [2024-11-05 00:34:03] DEBUG: Discarded disjointig with 521 reads and 519 inner overlaps [2024-11-05 00:34:03] DEBUG: Discarded disjointig with 14 reads and 12 inner overlaps [2024-11-05 00:34:03] DEBUG: Discarded disjointig with 3 reads and 1 inner overlaps [2024-11-05 00:34:03] DEBUG: Discarded disjointig with 189 reads and 187 inner overlaps [2024-11-05 00:34:03] DEBUG: Discarded disjointig with 4 reads and 2 inner overlaps [2024-11-05 00:34:03] DEBUG: Discarded disjointig with 3 reads and 1 inner overlaps [2024-11-05 00:34:03] DEBUG: Discarded disjointig with 3 reads and 1 inner overlaps [2024-11-05 00:34:07] DEBUG: Assembled disjointig 4 With 4 reads Start read: +SRR27458461.28437.1 At position: 2 leftTip: 0 rightTip: 0 Suspicious: 2 Short ext: 1 Mean extensions: 61 Avg overlap len: 5099 Min overlap len: 4595 Inner reads: 0 Length: 25854 [2024-11-05 00:34:07] DEBUG: Inner: 49526 covered: 49754 total: 50858 [2024-11-05 00:34:08] DEBUG: Assembled disjointig 5 With 3 reads Start read: +SRR27458461.12063.1 At position: 2 leftTip: 0 rightTip: 0 Suspicious: 1 Short ext: 0 Mean extensions: 60 Avg overlap len: 8647 Min overlap len: 8560 Inner reads: 0 Length: 17287 [2024-11-05 00:34:08] DEBUG: Inner: 49528 covered: 49770 total: 50858 [2024-11-05 00:34:11] DEBUG: Assembled disjointig 6 With 4 reads Start read: +SRR27458461.12799.1 At position: 3 leftTip: 0 rightTip: 0 Suspicious: 2 Short ext: 0 Mean extensions: 50 Avg overlap len: 8197 Min overlap len: 6618 Inner reads: 0 Length: 22829 [2024-11-05 00:34:11] DEBUG: Inner: 49532 covered: 49868 total: 50858 [2024-11-05 00:34:18] DEBUG: Assembled disjointig 7 With 4 reads Start read: +SRR27458461.32838.1 At position: 3 leftTip: 0 rightTip: 0 Suspicious: 2 Short ext: 0 Mean extensions: 60 Avg overlap len: 6042 Min overlap len: 6042 Inner reads: 0 Length: 22602 [2024-11-05 00:34:18] DEBUG: Inner: 49536 covered: 50100 total: 50858 [2024-11-05 00:34:21] DEBUG: Assembled disjointig 8 With 3 reads Start read: +SRR27458461.11451.1 At position: 2 leftTip: 0 rightTip: 0 Suspicious: 1 Short ext: 0 Mean extensions: 56 Avg overlap len: 8437 Min overlap len: 7807 Inner reads: 0 Length: 17619 [2024-11-05 00:34:21] DEBUG: Inner: 49538 covered: 50228 total: 50858 [2024-11-05 00:34:22] DEBUG: Assembled disjointig 9 With 4 reads Start read: +SRR27458461.25288.1 At position: 2 leftTip: 0 rightTip: 0 Suspicious: 2 Short ext: 2 Mean extensions: 19 Avg overlap len: 4913 Min overlap len: 2991 Inner reads: 0 Length: 17168 [2024-11-05 00:34:22] DEBUG: Inner: 49542 covered: 50238 total: 50858 [2024-11-05 00:34:24] INFO: Assembled 9 disjointigs [2024-11-05 00:34:24] INFO: Generating sequence [2024-11-05 00:34:24] DEBUG: Building positional index [2024-11-05 00:34:24] DEBUG: Total sequence: 3150769 bp [2024-11-05 00:34:25] DEBUG: Mean k-mer frequency: 1.10148 [2024-11-05 00:34:25] DEBUG: Repetitive k-mer frequency: 110 [2024-11-05 00:34:25] DEBUG: Filtered 0 repetitive k-mers (0) [2024-11-05 00:34:25] DEBUG: Sorting k-mer index [2024-11-05 00:34:25] DEBUG: Selected k-mers: 2860346 [2024-11-05 00:34:25] DEBUG: K-mer index size: 3150616 [2024-11-05 00:34:25] DEBUG: Mean k-mer frequency: 1.10148 [2024-11-05 00:34:25] DEBUG: Minimizer rate: 1.00005 [2024-11-05 00:34:25] INFO: Filtering contained disjointigs [2024-11-05 00:34:26] DEBUG: Computing transitive closure for overlaps [2024-11-05 00:34:26] DEBUG: Found 250 overlaps [2024-11-05 00:34:26] DEBUG: Left 100 overlaps after filtering [2024-11-05 00:34:26] INFO: Contained seqs: 4 [2024-11-05 00:34:26] DEBUG: Writing FASTA [2024-11-05 00:34:26] DEBUG: Peak RAM usage: 8 Gb -----------End assembly log------------ [2024-11-05 00:34:26] root: DEBUG: Disjointigs length: 3070423, N50: 1649226 [2024-11-05 00:34:26] root: INFO: >>>STAGE: consensus [2024-11-05 00:34:26] root: INFO: Running Minimap2 [2024-11-05 00:34:49] root: INFO: Computing consensus [2024-11-05 00:36:35] root: INFO: Alignment error rate: 0.008413 [2024-11-05 00:36:35] root: INFO: >>>STAGE: repeat [2024-11-05 00:36:35] root: INFO: Building and resolving repeat graph [2024-11-05 00:36:35] root: DEBUG: -----Begin repeat analyser log------ [2024-11-05 00:36:35] root: DEBUG: Running: flye-modules repeat --disjointigs /data/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/assemble~flye/output/10-consensus/consensus.fasta --reads /data/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/assemble~flye/input_1/SRR27458461.fastq.gz --out-dir /data/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/assemble~flye/output/20-repeat --config /usr/local/lib/python3.9/site-packages/flye/config/bin_cfg/asm_raw_reads.cfg --log /data/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/assemble~flye/output/flye.log --threads 8 --min-ovlp 5000 [2024-11-05 00:36:35] DEBUG: Build date: Aug 30 2024 21:36:17 [2024-11-05 00:36:35] DEBUG: Total RAM: 754 Gb [2024-11-05 00:36:35] DEBUG: Available RAM: 497 Gb [2024-11-05 00:36:35] DEBUG: Total CPUs: 16 [2024-11-05 00:36:35] DEBUG: Loading /usr/local/lib/python3.9/site-packages/flye/config/bin_cfg/asm_raw_reads.cfg [2024-11-05 00:36:35] DEBUG: Loading /usr/local/lib/python3.9/site-packages/flye/config/bin_cfg/asm_defaults.cfg [2024-11-05 00:36:35] DEBUG: big_genome_threshold=29000000 [2024-11-05 00:36:35] DEBUG: meta_read_filter_kmer_freq=100 [2024-11-05 00:36:35] DEBUG: chain_large_gap_penalty=2 [2024-11-05 00:36:35] DEBUG: chain_small_gap_penalty=0.5 [2024-11-05 00:36:35] DEBUG: chain_gap_jump_threshold=100 [2024-11-05 00:36:35] DEBUG: max_jump_gap=500 [2024-11-05 00:36:35] DEBUG: max_coverage_drop_rate=5 [2024-11-05 00:36:35] DEBUG: max_extensions_drop_rate=5 [2024-11-05 00:36:35] DEBUG: chimera_window=100 [2024-11-05 00:36:35] DEBUG: chimera_overhang=1000 [2024-11-05 00:36:35] DEBUG: min_reads_in_disjointig=4 [2024-11-05 00:36:35] DEBUG: max_inner_reads=10 [2024-11-05 00:36:35] DEBUG: max_inner_fraction=0.25 [2024-11-05 00:36:35] DEBUG: aggressive_dup_filter=1 [2024-11-05 00:36:35] DEBUG: max_separation=500 [2024-11-05 00:36:35] DEBUG: unique_edge_length=50000 [2024-11-05 00:36:35] DEBUG: min_repeat_res_support=0.51 [2024-11-05 00:36:35] DEBUG: out_paths_ratio=5 [2024-11-05 00:36:35] DEBUG: graph_cov_drop_rate=5 [2024-11-05 00:36:35] DEBUG: coverage_estimate_window=100 [2024-11-05 00:36:35] DEBUG: max_bubble_length=50000 [2024-11-05 00:36:35] DEBUG: loop_coverage_rate=1.5 [2024-11-05 00:36:35] DEBUG: repeat_edge_cov_mult=1.75 [2024-11-05 00:36:35] DEBUG: weak_detach_rate=5 [2024-11-05 00:36:35] DEBUG: tip_coverage_rate=2 [2024-11-05 00:36:35] DEBUG: tip_length_rate=2 [2024-11-05 00:36:35] DEBUG: output_gfa_before_rr=1 [2024-11-05 00:36:35] DEBUG: remove_alt_edges=0 [2024-11-05 00:36:35] DEBUG: low_cutoff_warning=1 [2024-11-05 00:36:35] DEBUG: kmer_size=17 [2024-11-05 00:36:35] DEBUG: use_minimizers=0 [2024-11-05 00:36:35] DEBUG: reads_base_alignment=0 [2024-11-05 00:36:35] DEBUG: meta_read_top_kmer_rate=0.40 [2024-11-05 00:36:35] DEBUG: maximum_jump=1500 [2024-11-05 00:36:35] DEBUG: maximum_overhang=1500 [2024-11-05 00:36:35] DEBUG: repeat_kmer_rate=100 [2024-11-05 00:36:35] DEBUG: assemble_ovlp_divergence=0.10 [2024-11-05 00:36:35] DEBUG: assemble_divergence_relative=1 [2024-11-05 00:36:35] DEBUG: repeat_graph_ovlp_divergence=0.08 [2024-11-05 00:36:35] DEBUG: read_align_ovlp_divergence=0.25 [2024-11-05 00:36:35] DEBUG: hpc_scoring_on=0 [2024-11-05 00:36:35] DEBUG: add_unassembled_reads=0 [2024-11-05 00:36:35] DEBUG: extend_contigs_with_repeats=0 [2024-11-05 00:36:35] DEBUG: min_read_cov_cutoff=3 [2024-11-05 00:36:35] DEBUG: short_tip_length=20000 [2024-11-05 00:36:35] DEBUG: long_tip_length=100000 [2024-11-05 00:36:35] DEBUG: Running with k-mer size: 17 [2024-11-05 00:36:35] DEBUG: Selected minimum overlap 5000 [2024-11-05 00:36:35] DEBUG: Metagenome mode: N [2024-11-05 00:36:35] INFO: Parsing disjointigs [2024-11-05 00:36:35] DEBUG: Building positional index [2024-11-05 00:36:35] DEBUG: Total sequence: 3069589 bp [2024-11-05 00:36:35] INFO: Building repeat graph [2024-11-05 00:36:36] DEBUG: Mean k-mer frequency: 1.07927 [2024-11-05 00:36:36] DEBUG: Repetitive k-mer frequency: 107 [2024-11-05 00:36:36] DEBUG: Filtered 0 repetitive k-mers (0) [2024-11-05 00:36:36] DEBUG: Sorting k-mer index [2024-11-05 00:36:36] DEBUG: Selected k-mers: 2844065 [2024-11-05 00:36:36] DEBUG: K-mer index size: 3069504 [2024-11-05 00:36:36] DEBUG: Mean k-mer frequency: 1.07927 [2024-11-05 00:36:36] DEBUG: Minimizer rate: 1.00003 [2024-11-05 00:36:37] DEBUG: Computing transitive closure for overlaps [2024-11-05 00:36:37] DEBUG: Found 100 overlaps [2024-11-05 00:36:37] DEBUG: Left 40 overlaps after filtering [2024-11-05 00:36:37] INFO: Median overlap divergence: 0.00219487 [2024-11-05 00:36:37] DEBUG: Sequence divergence distribution: |* | |* | |* | |* | |* | |* | |* | |* | |* | |* | |* | |* | |* | |* | |* | |* * | |* * | |* * * | |* * * | |* * * | ---------------------------------------------------------------------------------------------------- 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% Q25 = 0.0012, Q50 = 0.0022, Q75 = 0.021 [2024-11-05 00:36:37] DEBUG: Computing gluepoints [2024-11-05 00:36:37] DEBUG: Added 0 gluepoint projections [2024-11-05 00:36:37] DEBUG: Created 64 gluepoints [2024-11-05 00:36:37] DEBUG: Artificial loops removed: 0 left, 0 right, 0 both [2024-11-05 00:36:37] DEBUG: Initializing edges [2024-11-05 00:36:37] DEBUG: Edges length checksum: 5751843703 [2024-11-05 00:36:37] DEBUG: Filtered 0 singleton segments [2024-11-05 00:36:37] DEBUG: Removed 0 simple and 0 double chimeric junctions [2024-11-05 00:36:37] DEBUG: Collapsed 6 edges [2024-11-05 00:36:37] DEBUG: * 17 +disjointig_1 0 1107221 1107221 [2024-11-05 00:36:37] DEBUG: * 20 +disjointig_2 13741 271385 257644 [2024-11-05 00:36:37] DEBUG: * 18 +disjointig_3 10329 663741 653412 [2024-11-05 00:36:37] DEBUG: 10 +disjointig_3 663741 669210 5469 [2024-11-05 00:36:37] DEBUG: * -11 +disjointig_3 669210 759711 90501 [2024-11-05 00:36:37] DEBUG: 10 +disjointig_3 759711 764897 5186 [2024-11-05 00:36:37] DEBUG: * 12 +disjointig_3 764897 1416818 651921 [2024-11-05 00:36:37] DEBUG: 13 +disjointig_3 1416818 1421897 5079 [2024-11-05 00:36:37] DEBUG: * 15 +disjointig_3 1421897 1539730 117833 [2024-11-05 00:36:37] DEBUG: -13 +disjointig_3 1539730 1545005 5275 [2024-11-05 00:36:37] DEBUG: * 14 +disjointig_3 1545005 1590706 45701 [2024-11-05 00:36:37] DEBUG: 13 +disjointig_3 1590706 1595782 5076 [2024-11-05 00:36:37] DEBUG: * 19 +disjointig_3 1595782 1647936 52154 [2024-11-05 00:36:37] DEBUG: Total edges: 20 [2024-11-05 00:36:37] INFO: Parsing reads [2024-11-05 00:36:41] DEBUG: Building positional index [2024-11-05 00:36:41] DEBUG: Total sequence: 264541423 bp [2024-11-05 00:36:41] DEBUG: Building positional index [2024-11-05 00:36:41] DEBUG: Total sequence: 3002472 bp [2024-11-05 00:36:41] INFO: Aligning reads to the graph [2024-11-05 00:36:41] DEBUG: Mean k-mer frequency: 1.05611 [2024-11-05 00:36:41] DEBUG: Repetitive k-mer frequency: 105 [2024-11-05 00:36:41] DEBUG: Filtered 0 repetitive k-mers (0) [2024-11-05 00:36:42] DEBUG: Sorting k-mer index [2024-11-05 00:36:42] DEBUG: Selected k-mers: 2842742 [2024-11-05 00:36:42] DEBUG: K-mer index size: 3002251 [2024-11-05 00:36:42] DEBUG: Mean k-mer frequency: 1.05611 [2024-11-05 00:36:42] DEBUG: Minimizer rate: 1.00007 [2024-11-05 00:37:07] DEBUG: Total reads : 25429 [2024-11-05 00:37:07] DEBUG: Read with aligned parts : 25363 [2024-11-05 00:37:07] DEBUG: Aligned in one piece : 25356 [2024-11-05 00:37:07] INFO: Aligned read sequence: 230426379 / 231413392 (0.995735) [2024-11-05 00:37:07] INFO: Median overlap divergence: 0.00109482 [2024-11-05 00:37:07] DEBUG: Sequence divergence distribution: |* | |* | |* | |* | |* | |* | |* | |* | |* | |* | |* | |* | |* | |* | |* | |* | |* | |** | |** | |******** | ---------------------------------------------------------------------------------------------------- 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% Q25 = 0.00049, Q50 = 0.0011, Q75 = 0.0027 [2024-11-05 00:37:07] INFO: Mean edge coverage: 76 [2024-11-05 00:37:07] DEBUG: 10 len:5327 cov:184 mult:2.42105 [2024-11-05 00:37:07] DEBUG: -10 len:5327 cov:184 mult:2.42105 [2024-11-05 00:37:07] DEBUG: 11 len:90501 cov:83 mult:1.09211 [2024-11-05 00:37:07] DEBUG: -11 len:90501 cov:83 mult:1.09211 [2024-11-05 00:37:07] DEBUG: 12 len:651921 cov:81 mult:1.06579 [2024-11-05 00:37:07] DEBUG: -12 len:651921 cov:81 mult:1.06579 [2024-11-05 00:37:07] DEBUG: 13 len:5143 cov:195 mult:2.56579 [2024-11-05 00:37:07] DEBUG: -13 len:5143 cov:195 mult:2.56579 [2024-11-05 00:37:07] DEBUG: 14 len:45701 cov:79 mult:1.03947 [2024-11-05 00:37:07] DEBUG: -14 len:45701 cov:79 mult:1.03947 [2024-11-05 00:37:07] DEBUG: 15 len:117833 cov:76 mult:1 [2024-11-05 00:37:07] DEBUG: -15 len:117833 cov:76 mult:1 [2024-11-05 00:37:07] DEBUG: 17 len:1107221 cov:69 mult:0.907895 [2024-11-05 00:37:07] DEBUG: -17 len:1107221 cov:69 mult:0.907895 [2024-11-05 00:37:07] DEBUG: 18 len:653412 cov:81 mult:1.06579 [2024-11-05 00:37:07] DEBUG: -18 len:653412 cov:81 mult:1.06579 [2024-11-05 00:37:07] DEBUG: 19 len:52154 cov:86 mult:1.13158 [2024-11-05 00:37:07] DEBUG: -19 len:52154 cov:86 mult:1.13158 [2024-11-05 00:37:07] DEBUG: 20 len:257644 cov:75 mult:0.986842 [2024-11-05 00:37:07] DEBUG: -20 len:257644 cov:75 mult:0.986842 [2024-11-05 00:37:07] DEBUG: Unique coverage threshold 145 [2024-11-05 00:37:07] INFO: Simplifying the graph [2024-11-05 00:37:07] DEBUG: Read coverage cutoff: 15 [2024-11-05 00:37:07] DEBUG: [SIMPL] Removed 0 paths with low coverage [2024-11-05 00:37:07] DEBUG: [SIMPL] Masked 0 heterozygous loops [2024-11-05 00:37:07] DEBUG: [SIMPL] Masked 0 simple bubbles [2024-11-05 00:37:07] DEBUG: Finding repeats [2024-11-05 00:37:07] DEBUG: Read coverage cutoff: 15 [2024-11-05 00:37:07] DEBUG: High-cov: 10 5327 184 [2024-11-05 00:37:07] DEBUG: High-cov: 13 5143 195 [2024-11-05 00:37:07] DEBUG: Repeat detection iteration 1 [2024-11-05 00:37:07] DEBUG: Writing Dot [2024-11-05 00:37:07] DEBUG: Writing FASTA [2024-11-05 00:37:07] DEBUG: Writing Gfa [2024-11-05 00:37:07] DEBUG: [SIMPL] == Iteration 1 == [2024-11-05 00:37:07] DEBUG: Splitting nodes [2024-11-05 00:37:07] DEBUG: [SIMPL] Split 0 nodes [2024-11-05 00:37:07] DEBUG: [SIMPL] Clipped 0 short and 0 long tips [2024-11-05 00:37:07] DEBUG: [SIMPL] Masked 0 heterozygous loops [2024-11-05 00:37:07] DEBUG: [SIMPL] Masked 0 simple bubbles [2024-11-05 00:37:07] DEBUG: Finding repeats [2024-11-05 00:37:07] DEBUG: Read coverage cutoff: 15 [2024-11-05 00:37:07] DEBUG: High-cov: 10 5327 184 [2024-11-05 00:37:07] DEBUG: High-cov: 13 5143 195 [2024-11-05 00:37:07] DEBUG: Repeat detection iteration 1 [2024-11-05 00:37:07] DEBUG: Total unique edges: 8 [2024-11-05 00:37:07] DEBUG: Connection -19 -14 18 0.972973 [2024-11-05 00:37:07] DEBUG: Connection -12 11 33 1 [2024-11-05 00:37:07] DEBUG: Connection 18 -11 27 1 [2024-11-05 00:37:07] DEBUG: Connection -14 -15 30 0.983607 [2024-11-05 00:37:07] DEBUG: Connection -15 -12 19 0.95 [2024-11-05 00:37:07] DEBUG: [SIMPL] Resolved repeats: 5 [2024-11-05 00:37:07] DEBUG: RR links: 258 [2024-11-05 00:37:07] DEBUG: Unresolved: 0 [2024-11-05 00:37:07] DEBUG: Removed 0 simple and 0 double chimeric junctions [2024-11-05 00:37:07] DEBUG: [SIMPL] == Iteration 2 == [2024-11-05 00:37:07] DEBUG: Splitting nodes [2024-11-05 00:37:07] DEBUG: [SIMPL] Split 0 nodes [2024-11-05 00:37:07] DEBUG: [SIMPL] Clipped 0 short and 0 long tips [2024-11-05 00:37:07] DEBUG: [SIMPL] Masked 0 heterozygous loops [2024-11-05 00:37:07] DEBUG: [SIMPL] Masked 0 simple bubbles [2024-11-05 00:37:07] DEBUG: Finding repeats [2024-11-05 00:37:07] DEBUG: Read coverage cutoff: 15 [2024-11-05 00:37:07] DEBUG: Repeat detection iteration 1 [2024-11-05 00:37:07] DEBUG: Total unique edges: 13 [2024-11-05 00:37:07] DEBUG: [SIMPL] Resolved repeats: 0 [2024-11-05 00:37:07] DEBUG: RR links: 0 [2024-11-05 00:37:07] DEBUG: Unresolved: 0 [2024-11-05 00:37:07] DEBUG: Removed 0 simple and 0 double chimeric junctions [2024-11-05 00:37:07] DEBUG: [SIMPL] Collapsed 0 haplotypes [2024-11-05 00:37:07] DEBUG: [SIMPL] Resolved 0 simple repeats [2024-11-05 00:37:07] DEBUG: Read coverage cutoff: 15 [2024-11-05 00:37:07] DEBUG: [SIMPL] Removed 0 paths with low coverage [2024-11-05 00:37:07] DEBUG: Finding repeats [2024-11-05 00:37:07] DEBUG: Read coverage cutoff: 15 [2024-11-05 00:37:07] DEBUG: Repeat detection iteration 1 [2024-11-05 00:37:07] DEBUG: Writing Dot [2024-11-05 00:37:07] DEBUG: Writing FASTA [2024-11-05 00:37:07] DEBUG: Peak RAM usage: 0 Gb -----------End assembly log------------ [2024-11-05 00:37:07] root: INFO: >>>STAGE: contigger [2024-11-05 00:37:07] root: INFO: Generating contigs [2024-11-05 00:37:07] root: DEBUG: -----Begin contigger analyser log------ [2024-11-05 00:37:07] root: DEBUG: Running: flye-modules contigger --graph-edges /data/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/assemble~flye/output/20-repeat/repeat_graph_edges.fasta --reads /data/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/assemble~flye/input_1/SRR27458461.fastq.gz --out-dir /data/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/assemble~flye/output/30-contigger --config /usr/local/lib/python3.9/site-packages/flye/config/bin_cfg/asm_raw_reads.cfg --repeat-graph /data/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/assemble~flye/output/20-repeat/repeat_graph_dump --graph-aln /data/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/assemble~flye/output/20-repeat/read_alignment_dump --log /data/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/assemble~flye/output/flye.log --threads 8 --min-ovlp 5000 [2024-11-05 00:37:07] DEBUG: Build date: Aug 30 2024 21:36:44 [2024-11-05 00:37:07] DEBUG: Total RAM: 754 Gb [2024-11-05 00:37:07] DEBUG: Available RAM: 497 Gb [2024-11-05 00:37:07] DEBUG: Total CPUs: 16 [2024-11-05 00:37:07] DEBUG: Loading /usr/local/lib/python3.9/site-packages/flye/config/bin_cfg/asm_raw_reads.cfg [2024-11-05 00:37:07] DEBUG: Loading /usr/local/lib/python3.9/site-packages/flye/config/bin_cfg/asm_defaults.cfg [2024-11-05 00:37:07] DEBUG: big_genome_threshold=29000000 [2024-11-05 00:37:07] DEBUG: meta_read_filter_kmer_freq=100 [2024-11-05 00:37:07] DEBUG: chain_large_gap_penalty=2 [2024-11-05 00:37:07] DEBUG: chain_small_gap_penalty=0.5 [2024-11-05 00:37:07] DEBUG: chain_gap_jump_threshold=100 [2024-11-05 00:37:07] DEBUG: max_jump_gap=500 [2024-11-05 00:37:07] DEBUG: max_coverage_drop_rate=5 [2024-11-05 00:37:07] DEBUG: max_extensions_drop_rate=5 [2024-11-05 00:37:07] DEBUG: chimera_window=100 [2024-11-05 00:37:07] DEBUG: chimera_overhang=1000 [2024-11-05 00:37:07] DEBUG: min_reads_in_disjointig=4 [2024-11-05 00:37:07] DEBUG: max_inner_reads=10 [2024-11-05 00:37:07] DEBUG: max_inner_fraction=0.25 [2024-11-05 00:37:07] DEBUG: aggressive_dup_filter=1 [2024-11-05 00:37:07] DEBUG: max_separation=500 [2024-11-05 00:37:07] DEBUG: unique_edge_length=50000 [2024-11-05 00:37:07] DEBUG: min_repeat_res_support=0.51 [2024-11-05 00:37:07] DEBUG: out_paths_ratio=5 [2024-11-05 00:37:07] DEBUG: graph_cov_drop_rate=5 [2024-11-05 00:37:07] DEBUG: coverage_estimate_window=100 [2024-11-05 00:37:07] DEBUG: max_bubble_length=50000 [2024-11-05 00:37:07] DEBUG: loop_coverage_rate=1.5 [2024-11-05 00:37:07] DEBUG: repeat_edge_cov_mult=1.75 [2024-11-05 00:37:07] DEBUG: weak_detach_rate=5 [2024-11-05 00:37:07] DEBUG: tip_coverage_rate=2 [2024-11-05 00:37:07] DEBUG: tip_length_rate=2 [2024-11-05 00:37:07] DEBUG: output_gfa_before_rr=1 [2024-11-05 00:37:07] DEBUG: remove_alt_edges=0 [2024-11-05 00:37:07] DEBUG: low_cutoff_warning=1 [2024-11-05 00:37:07] DEBUG: kmer_size=17 [2024-11-05 00:37:07] DEBUG: use_minimizers=0 [2024-11-05 00:37:07] DEBUG: reads_base_alignment=0 [2024-11-05 00:37:07] DEBUG: meta_read_top_kmer_rate=0.40 [2024-11-05 00:37:07] DEBUG: maximum_jump=1500 [2024-11-05 00:37:07] DEBUG: maximum_overhang=1500 [2024-11-05 00:37:07] DEBUG: repeat_kmer_rate=100 [2024-11-05 00:37:07] DEBUG: assemble_ovlp_divergence=0.10 [2024-11-05 00:37:07] DEBUG: assemble_divergence_relative=1 [2024-11-05 00:37:07] DEBUG: repeat_graph_ovlp_divergence=0.08 [2024-11-05 00:37:07] DEBUG: read_align_ovlp_divergence=0.25 [2024-11-05 00:37:07] DEBUG: hpc_scoring_on=0 [2024-11-05 00:37:07] DEBUG: add_unassembled_reads=0 [2024-11-05 00:37:07] DEBUG: extend_contigs_with_repeats=0 [2024-11-05 00:37:07] DEBUG: min_read_cov_cutoff=3 [2024-11-05 00:37:07] DEBUG: short_tip_length=20000 [2024-11-05 00:37:07] DEBUG: long_tip_length=100000 [2024-11-05 00:37:07] DEBUG: Running with k-mer size: 17 [2024-11-05 00:37:07] DEBUG: Selected minimum overlap 5000 [2024-11-05 00:37:07] INFO: Reading sequences [2024-11-05 00:37:10] DEBUG: Building positional index [2024-11-05 00:37:10] DEBUG: Total sequence: 264541423 bp [2024-11-05 00:37:11] DEBUG: Flipped 0 [2024-11-05 00:37:11] DEBUG: UPath 1: -20 -> -19 -> -21 -> -14 -> 24 -> -15 -> -25 -> -12 -> 22 -> 11 -> -23 -> -18 -> -17 [2024-11-05 00:37:11] DEBUG: Final graph contains 1 egdes [2024-11-05 00:37:11] DEBUG: Extending contigs into repeats [2024-11-05 00:37:11] DEBUG: Covered 0 repetitive contigs [2024-11-05 00:37:11] INFO: Generated 1 contigs [2024-11-05 00:37:11] DEBUG: Writing FASTA [2024-11-05 00:37:11] DEBUG: Generating scaffold connections [2024-11-05 00:37:11] INFO: Added 0 scaffold connections [2024-11-05 00:37:11] DEBUG: Writing Dot [2024-11-05 00:37:11] DEBUG: Writing FASTA [2024-11-05 00:37:11] DEBUG: Writing Gfa [2024-11-05 00:37:11] DEBUG: Peak RAM usage: 0 Gb -----------End assembly log------------ [2024-11-05 00:37:11] root: INFO: >>>STAGE: polishing [2024-11-05 00:37:11] root: INFO: Polishing genome (1/1) [2024-11-05 00:37:11] root: INFO: Running minimap2 [2024-11-05 00:37:33] root: INFO: Separating alignment into bubbles [2024-11-05 00:39:02] root: DEBUG: Generated 190386 bubbles [2024-11-05 00:39:02] root: DEBUG: Split 0 long bubbles [2024-11-05 00:39:02] root: DEBUG: Skipped 0 empty bubbles [2024-11-05 00:39:02] root: DEBUG: Skipped 0 bubbles with long branches [2024-11-05 00:39:02] root: INFO: Alignment error rate: 0.002672 [2024-11-05 00:39:02] root: INFO: Correcting bubbles [2024-11-05 00:39:26] root: DEBUG: Mean contig coverage: 89, selected threshold: 18 [2024-11-05 00:39:26] root: DEBUG: Filtered 0 contigs of total length 0 [2024-11-05 00:39:26] root: DEBUG: Generating polished GFA [2024-11-05 00:39:28] root: DEBUG: 0 sequences remained unpolished [2024-11-05 00:39:28] root: INFO: >>>STAGE: finalize [2024-11-05 00:39:28] root: DEBUG: ---Output dir contents:---- [2024-11-05 00:39:28] root: DEBUG: output/ [2024-11-05 00:39:28] root: DEBUG: 35.0 K flye.log [2024-11-05 00:39:28] root: DEBUG: 92.0 B params.json [2024-11-05 00:39:28] root: DEBUG: 369.0 B assembly_graph.gv [2024-11-05 00:39:28] root: DEBUG: 2.0 M assembly_graph.gfa [2024-11-05 00:39:28] root: DEBUG: 2.0 M assembly.fasta [2024-11-05 00:39:28] root: DEBUG: 00-assembly/ [2024-11-05 00:39:28] root: DEBUG: 2.0 M draft_assembly.fasta [2024-11-05 00:39:28] root: DEBUG: 165.0 B draft_assembly.fasta.fai [2024-11-05 00:39:28] root: DEBUG: 10-consensus/ [2024-11-05 00:39:28] root: DEBUG: 1011.0 B minimap.stderr [2024-11-05 00:39:28] root: DEBUG: 64.0 K minimap.bam.bai [2024-11-05 00:39:28] root: DEBUG: 2.0 M consensus.fasta [2024-11-05 00:39:28] root: DEBUG: 20-repeat/ [2024-11-05 00:39:28] root: DEBUG: 1.0 K graph_before_rr.gv [2024-11-05 00:39:28] root: DEBUG: 2.0 M graph_before_rr.fasta [2024-11-05 00:39:28] root: DEBUG: 2.0 M graph_before_rr.gfa [2024-11-05 00:39:28] root: DEBUG: 1.0 K graph_after_rr.gv [2024-11-05 00:39:28] root: DEBUG: 2.0 K repeat_graph_dump [2024-11-05 00:39:28] root: DEBUG: 6.0 M read_alignment_dump [2024-11-05 00:39:28] root: DEBUG: 2.0 M repeat_graph_edges.fasta [2024-11-05 00:39:28] root: DEBUG: 30-contigger/ [2024-11-05 00:39:28] root: DEBUG: 2.0 M contigs.fasta [2024-11-05 00:39:28] root: DEBUG: 112.0 B contigs_stats.txt [2024-11-05 00:39:28] root: DEBUG: 0.0 B scaffolds_links.txt [2024-11-05 00:39:28] root: DEBUG: 369.0 B graph_final.gv [2024-11-05 00:39:28] root: DEBUG: 2.0 M graph_final.fasta [2024-11-05 00:39:28] root: DEBUG: 2.0 M graph_final.gfa [2024-11-05 00:39:28] root: DEBUG: 26.0 B contigs.fasta.fai [2024-11-05 00:39:28] root: DEBUG: 40-polishing/ [2024-11-05 00:39:28] root: DEBUG: 952.0 B minimap.stderr [2024-11-05 00:39:28] root: DEBUG: 62.0 K minimap_1.bam.bai [2024-11-05 00:39:28] root: DEBUG: 46.0 B contigs_stats.txt [2024-11-05 00:39:28] root: DEBUG: 937.0 K base_coverage.bed.gz [2024-11-05 00:39:28] root: DEBUG: 2.0 M filtered_contigs.fasta [2024-11-05 00:39:28] root: DEBUG: 46.0 B filtered_stats.txt [2024-11-05 00:39:28] root: DEBUG: 26.0 B filtered_contigs.fasta.fai [2024-11-05 00:39:28] root: DEBUG: 1.0 K edges_aln.bam.bai [2024-11-05 00:39:28] root: DEBUG: 2.0 M polished_edges.gfa [2024-11-05 00:39:28] root: DEBUG: -------------------------- [2024-11-05 00:39:28] root: INFO: Assembly statistics: Total length: 3002544 Fragments: 1 Fragments N50: 3002544 Largest frg: 3002544 Scaffolds: 0 Mean coverage: 89 [2024-11-05 00:39:28] root: INFO: Final assembly: /data/yoshitake.kazutoshi/work/pp-dev/yoshitake/test/assemble~flye/output/assembly.fasta