Streaming Support for Data Intensive Cloud-Based Sequence Analysis

Table 4

Running times in minutes for mapping NGS reads to a reference genome using Crossbow. The column titled speed specifies the upload speed. The column titled cluster includes the number of created EMR clusters. The column titled compTime incudes the computation time of the whole data after uploading it. The column titled totalTimeS includes the experiment time in streaming mode, and the column titled totalTimeT includes the time in nonstreaming mode, where all the data is first transferred and then processed. The number in brackets in this column is the respective monetary cost.

Upload_speed	Read_size	Clusters	Upload_time	CompTime	TotalTimeT	TotalTimeS
E. coli reads
250 KB/s	1.4 G	1	100	9	109 ($0.72)	110 ($0.96)
250 KB/s	1.4 G	2	100	7	107 ($1.2)	110 ($1.92)
250 KB/s	1.4 G	4	100	7	107 ($2.26)	110 ($3.84)
1 GB human reads
250 KB/s	1 G	1	71	6	77 ($0.72)	84 ($0.96)
250 KB/s	1 G	2	71	5	76 ($1.2)	84 ($1.92)
250 KB/s	1 G	4	71	5	76 ($1.2)	84 ($3.84)
10 GB human reads
250 KB/s	10 G	1	800	60	860 ($2.16)	820 ($6.82)
250 KB/s	10 G	2	800	31	831 ($2.64)	820 ($13.44)
250 KB/s	10 G	4	800	18	818 ($3.6)	820 ($26.88)
E. coli reads
1 MB/s	1.4 G	1	25	9	34 ($0.6)	65 ($0.96)
1 MB/s	1.4 G	2	25	7	32 ($1.08)	65 ($1.92)
1 MB/s	1.4 G	4	25	7	32 ($2.04)	65 ($3.84)
1 GB human reads
1 MB/s	1 G	1	18	6	24 ($0.6)	31 ($0.48)
1 MB/s	1 G	2	18	5	23 ($1.09)	32 ($0.96)
1 MB/s	1 G	4	18	5	23 ($2.04)	31 ($0.1.92)
10 GB human reads
1 MB/s	10 G	1	200	60	260 ($0.96)	220 ($1.92)
1 MB/s	10 G	2	200	31	231 ($1.54)	220 ($3.84)
1 MB/s	10 G	4	200	18	218 ($2.40)	220 ($7.68)
40 GB human reads
1 MB/s	40 G	1	690	580	1270 ($6.24)	630 ($5.28)
1 MB/s	40 G	2	690	300	990 ($6.24)	630 ($10.56)
1 MB/s	40 G	4	690	150	840 ($7.2)	630 ($21.12)
1 MB/s	40 G	8	690	80	770 ($9.12)	630 ($42.24)
130 GB human reads
1 MB/s	130 G	1	2220	1890	4110 ($19.8)	2360 ($19.2)
1 MB/s	130 G	2	2220	945	3165 ($19.8)	2320 ($37.44)
1 MB/s	130 G	4	2220	476	2696 ($19.8)	2320 ($74.88)
1 MB/s	130 G	8	2220	184	2404 ($19.8)	2320 ($149.76)
1 MB/s	130 G	16	2220	126	2346 ($27.48)	2320 ($299.52)
10 GB human reads
10 MB/s	10 G	1	20	60	80 ($1.08)	160 ($1.44)
10 MB/s	10 G	2	20	31	51 ($1.08)	95 ($1.92)
10 MB/s	10 G	4	20	18	38 ($2.04)	65 ($3.84)
40 GB human reads
10 MB/s	40 G	1	70	580	650 ($5.04)	610 ($5.28)
10 MB/s	40 G	2	70	300	370 ($5.04)	310 ($5.76)
10 MB/s	40 G	4	70	150	220 ($6.00)	170 ($5.76)
10 MB/s	40 G	8	70	80	150 ($7.92)	110 ($7.68)
130 GB human reads
10 MB/s	130 G	1	224	1890	2114 ($15.84)	1960 ($15.84)
10 MB/s	130 G	2	224	945	1169 ($15.84)	1000 ($16.32)
10 MB/s	130 G	4	224	476	700 ($15.84)	520 ($17.28)
10 MB/s	130 G	8	224	184	470 ($15.84)	300 ($19.2)
10 MB/s	130 G	16	224	126	350 ($23.52)	300 ($38.4)