Streaming Support for Data Intensive Cloud-Based Sequence Analysis

Table 3

Running times in minutes for mapping NGS reads to a reference genome using Bowtie based on the use of traditional computer cluster. The column titled upload_speed specifies the upload speed. The column titled “upload” includes the time in minutes for uploading the data to the cloud with the respective upload speed. The column titled “compTime” includes the computation time in minutes of the whole dataset after being uploaded to the cloud. The column titled totalTimeS includes the experiment time in streaming mode and the column titled totalTimeT includes the time in nonstreaming mode, where all the data is first transferred and then processed. The numbers in brackets in this column are the respective monetary cost.

Upload_speed	Read size	Nodes	Upload	CompTime	TotalTimeT	TotalTimeS
E. coli reads
250 KB/s	1.4 G	1	100	3	103 ($1.32)	102 ($1.32)
250 KB/s	1.4 G	2	100	3	103 ($1.98)	102 ($2.64 )
250 KB/s	1.4 G	4	100	3	103 ($3.3)	102 ($5.28)
1 GB human reads
250 KB/s	1 G	1	71	17	88 ($1.32)	72 ($1.32)
250 KB/s	1 G	2	71	11	82 ($1.98)	72 ($2.64)
250 KB/s	1 G	4	71	7	73 ($3.3)	72 ($5.28)
10 GB human reads
250 KB/s	10 G	1	800 (13.3 h)	220	1021 ($11.88)	832 ($9.24)
250 KB/s	10 G	2	800 (13.3 h)	130	930 ($12.54)	818 ($9.24)
250 KB/s	10 G	4	800 (13.3 h)	60	860 ($11.88)	818 ($9.24)
E. coli reads
1 MB/s	1.4 G	1	25	3	28 ($0.66)	27 ($0.66)
1 MB/s	1.4 G	2	25	3	28 ($1.32)	27 ($1.32)
1 MB/s	1.4 G	4	25	3	28 ($2.64)	27 ($2.64)
1 GB human reads
1 MB/s	1 G	1	18	17	35 ($0.66)	21 ($0.66)
1 MB/s	1 G	2	18	11	29 ($1.32)	21 ($1.32)
1 MB/s	1 G	4	18	7	25 ($2.64)	21 ($2.64)
10 GB human reads
1 MB/s	10 G	1	200	220	421 ($5.28)	231 ($2.64)
1 MB/s	10 G	2	200	130	330 ($5.94)	215 ($5.28)
1 MB/s	10 G	4	200	60	261 ($5.28)	215 ($10.56)
40 GB human reads
1 MB/s	40 G	1	690	590	1280 ($14.52)	1100 ($12.54)
1 MB/s	40 G	2	690	325	1015 ($15.18)	695 ($15.84)
1 MB/s	40 G	4	690	180	870 ($15.84)	695 ($31.68)
130 GB human reads
1 MB/s	130 G	1	2220	1720	3940 ($43.56)	3600 ($39.6)
1 MB/s	130 G	2	2220	940	3160 ($45.54)	2400 ($52.8)
1 MB/s	130 G	4	2220	520	2740 ($48.18)	2400 ($105.6)
1 MB/s	130 G	8	2220	284	2504 ($50.82)	2400 ($211.2)
E. coli reads
10 MB/s	1.4 G	1	2.5	3	5.5 ($0.66)	5 ($0.66)
10 MB/s	1.4 G	2	2.5	3	5.5 ($1.32)	5 ($1.32)
10 MB/s	1.4 G	4	2.5	3	5.5 ($2.64)	5 ($2.64)
10 GB human reads
10 MB/s	10 G	1	18	220	238 ($2.64)	180 ($1.98)
10 MB/s	10 G	2	18	130	148 ($3.96)	85 ($2.64)
10 MB/s	10 G	4	18	60	78 ($3.3)	50 ($2.64)
40 GB human reads
10 MB/s	40 G	1	70	590	660 ($7.26)	686 ($7.92)
10 MB/s	40 G	2	70	310	380 ($8.58)	350 ($7.92)
10 MB/s	40 G	4	70	170	240 ($8.58)	180 ($7.92)
10 MB/s	40 G	8	70	95	165 ($11.22)	100 ($10.56)
10 MB/s	40 G	16	70	53	123 ($11.88)	73 ($21.12)
130 GB human reads
10 MB/s	130 G	1	224	1720	1944 ($21.78)	2050 ($23.1)
10 MB/s	130 G	2	224	950	1174 ($23.76)	1100 ($25.08)
10 MB/s	130 G	4	224	520	744 ($26.4)	580 ($26.4)
10 MB/s	130 G	8	224	284	508 ($33.66)	320 ($31.68)
10 MB/s	130 G	16	224	160	384 ($34.32)	235 ($42.24)