Meteorological Data Analysis Using MapReduce

Table 2

The configuration items of Hadoop key parameters.

Configuration parameter name	Parameter value	Description
io.sort.mb	256	Maximum Memory to store temporary data in the phase of arrangement, overflow to the disk if excess, unit: M
dfs.replication	3	Number of file backup
dfs.block.size	409600	The maximum value of each file: the file is read and stored in block if excess unit: bit
mapred.local.dir	/mapred/local	Data stored path when MapReduce task executes
mapred.tasktracker.; map.tasks.maximun	2	The maximum number of Map tasks can be run on a TaskTracker; these tasks run at the same time
mapred.tasktracker.; reduce.tasks.maximun	1	The maximum number of Reduce tasks can be run on a TaskTracker; these tasks run at the same time
mapred.reduce.; parallel.copies	30	Reduce startup more parallel copies for a large number of output map
io.sort.factor	100	More streams will be merged while sorting files
fs.default.name	hdfs://aiken:9000	The host IP and port of JobTracker
hadoop.tmp.dir	/root/data1	Hadoop default temporary path