Search⌘ K
AI Features
Home
Courses
Introduction to Big Data and Hadoop

Exploring MapReduce Runs

Explore how changing the number of reducers influences MapReduce job output. Learn to run jobs with multiple reducers, examine output files, and navigate the Job History Server UI to track job execution on a Hadoop cluster.

We'll cover the following...

Exploring MapReduce Runs

In this lesson, we vary the number of reducers and see how that affects the output. Here we’ll increase the number of reducers to 3.

Connect to the terminal below and execute the commands. Each command is explained later in the lesson. You can read the explanation first and then execute the commands in the terminal.

Shell
# Click on the terminal below and execute the commands in order
./DataJek/startHadoop.sh
hdfs dfs -copyFromLocal /DataJek/cars.data /
hadoop jar JarDependencies/MapReduceJarDependencies/MapReduce-1.0-SNAPSHOT.jar io.datajek.mapreduce.Driver /cars.data /MultipleReducers 3
hdfs dfs -ls /MultipleReducers
hdfs dfs -cat /MultipleReducers/part-r-00000
hdfs dfs -cat /MultipleReducers/part-r-00001
hdfs dfs -cat /MultipleReducers/part-r-00002
Terminal 1
Terminal
Loading...

  1. Start-up the hadoop cluster by running the command below:

    /DataJek/startHado
...