i
Characteristics of Big Data
Application of Big Data Processing
Introduction to BIG DATA
Where to get Big Data?
Types of Big Data
Storage layer - HDFS (Hadoop Distributed File System)
MapReduce
YARN
How Hadoop works?
Hadoop Eco System
Hadoop Architecture
Hadoop Installation & Environment Setup
Setting Up A Single Node Hadoop Cluster
Ubuntu User Configuration
SSH Setup With Key Generation
Disable IPv6
Download and Install Hadoop 3.1.2
Working with Configuration Files
Start The Hadoop instances
Hadoop Distributed File System (HDFS)
HDFS Features and Goals
HDFS Architecture
Read Operations in HDFS
Write Operations In HDFS
HDFS Operations
YARN
YARN Features
YARN Architecture
Resource Manager
Node Manager
Application Master
Container
Application Workflow in Hadoop YARN
Hadoop MapReduce
How MapReduce Works?
MapReduce Examples with Python
Running The MapReduce Program & Storing The Data File To HDFS
Create A Python Script
Hadoop Environment Setup
Execute The Script
Apache Hive Definition
Why Apache Hive?
Features Of Apache Hive
Hive Architecture
Hive Metastore
Hive Query Language
SQL vs Hive
Hive Installation
Apache Pig Definition
MapReduce vs. Apache Pig vs. Hive
Apache Pig Architecture
Installation Process Of Apache Pig
Execute Apache Pig Script
Hadoop Eco Components
NoSQL Data Management
Apache Hbase
Apache Cassandra
Mongodb
Introduction To Kafka
The Architecture of Apache Flume
Apache Spark Ecosystem
In this section, first, we will download Hadoop 3.1.2 (released in February 2019) on the hduser then install it.
Step 12: We are going to search the latest download file for Hadoop on the web, so we google it and go to the Apache Hadoop web page.
In the Apache Hadoop web page, we will get different version options to download. We will go to the binary of Hadoop 3.1.2 Version.
It will land us to download the page and suggest the mirror to download Hadoop from it. We will choose the default mirror.
This download may take a few minutes. After finishing the download, please go to the Downloads folder, and our desired file should be available there.
Step 13: Now, we will move the hadoop zip file to our working folder. We can set any folder, even we can work from this Downloads location, but it is always recommended to work from a safe, isolated location. In my case, I will work from /usr/local. Use the below command to move your Hadoop file to the specific location.
Step 14: Now go to the /usr/ local folder and unzip the hadoop-3.1.2.tar.gz file
Step 15: After successfully execute the command, go to the file location and check it. We will create a shortcut of hadoop-3.1.2 with the name hadoop.
Finally, there will be a unzip the hadoop-3.1.2 directory and a hadoop shortcut directory.
Step 16: We will change the ownership of the folder hadoop-3.1.2 to hduser and give the full authority on it so that we can read, write, and modify this folder.
sudo chown –R hduser:hadoop hadoop-3.1.2 sudo chmod 777 hadoop-3.1.2 |
Don't miss out!