Dr Abir KHALDI

Setup a Single-Node Hadoop Cluster Using Docker

Step 1: If you are getting the output you see above, the docker is running properly on your

instance, and now we can setup Hadoop inside a docker container. To do so, run a pull

command to get the docker image on Hadoop. More specifically, what you'll see is a Docker

Image a file with multiple layers, which you'll use to deploy containers.

docker pull sequenceiq/hadoop-docker:2.7.0

2.7.0: Pulling from sequenceiq/hadoop-docker

Step 2: run the below command to the check the list of docker images present on your

system.

docker images

output

Step 3 : run the image in a container

Step 4: In the above output you can see, the container is starting all the Hadoop daemons

one by one. Just to make sure all the daemons are up and running, run the jps command.

Dr Abir KHALDI

Step 5: If you get the above output after running the jps command, then you can be assured

that all the hadoop daemons are running correctly. After that, run the docker command

shown below to get the details on the docker container.

Step 6 : Run the below command to get the IP address on which the container is running.

Step 7: From the above output, we know that the docker container

is running

at 172.17.0.3. Following this, the Hadoop cluster web interface can be accessed on port

50070. So, you can open your browser, specifically Mozilla Firefox browser, in your ubuntu

machine and go to 172.17.0.3:50070. Your Hadoop Overview Interface will open. You can

see from below output that Hadoop is running at port 9000, which is the default port.

Dr Abir KHALDI

Step 8: Now you're inside a docker container. You will find bash shell inside the

container, not the default ubuntu terminal shell. Let's run a wordcount mapreduce

program on this Hadoop cluster running inside a docker container. This program will

take the input containing text and give output as key value pair where key will the

work and value will be the number of occurrences of that word. First thing you'll want

to do is go to the Hadoop home directory.

bash-4.1# cd $HADOOP_PREFIX

Step 9: Next, run the hadoop-mapreduce-examples-2.7.0.jar file, which has a wordcount

program pre-installed. The output is as follows:

bash-4.1# bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-

2.7.0.jar grep input output 'dfs[a-z.]+'

output

Dr Abir KHALDI

End output

Step 10: After the mapreduce program has finished executing the operation, run the below

command to check the output.

bash-4.1# bin/hdfs dfs -cat output/*

output