i
Characteristics of Big Data
Application of Big Data Processing
Introduction to BIG DATA
Where to get Big Data?
Types of Big Data
Storage layer - HDFS (Hadoop Distributed File System)
MapReduce
YARN
How Hadoop works?
Hadoop Eco System
Hadoop Architecture
Hadoop Installation & Environment Setup
Setting Up A Single Node Hadoop Cluster
Ubuntu User Configuration
SSH Setup With Key Generation
Disable IPv6
Download and Install Hadoop 3.1.2
Working with Configuration Files
Start The Hadoop instances
Hadoop Distributed File System (HDFS)
HDFS Features and Goals
HDFS Architecture
Read Operations in HDFS
Write Operations In HDFS
HDFS Operations
YARN
YARN Features
YARN Architecture
Resource Manager
Node Manager
Application Master
Container
Application Workflow in Hadoop YARN
Hadoop MapReduce
How MapReduce Works?
MapReduce Examples with Python
Running The MapReduce Program & Storing The Data File To HDFS
Create A Python Script
Hadoop Environment Setup
Execute The Script
Apache Hive Definition
Why Apache Hive?
Features Of Apache Hive
Hive Architecture
Hive Metastore
Hive Query Language
SQL vs Hive
Hive Installation
Apache Pig Definition
MapReduce vs. Apache Pig vs. Hive
Apache Pig Architecture
Installation Process Of Apache Pig
Execute Apache Pig Script
Hadoop Eco Components
NoSQL Data Management
Apache Hbase
Apache Cassandra
Mongodb
Introduction To Kafka
The Architecture of Apache Flume
Apache Spark Ecosystem
We will set up a single Hadoop cluster using a dedicated Hadoop user "hduser”. To do this, we have to log in as root user, then create a user and have to give sudo permission.
Step 1: From your user, log in to root using sudo su command. It will land you to root user.
sudo su
|
Step 2: We will use a dedicated Hadoop user account for running Hadoop, while it is always recommended because it helps to separate the Hadoop installation from other users. We will create a group hadoop and create & add a user hduser on it.
sudo addgroup hadoop sudo adduser hduser
sudo adduser hduser hadoop |
Step 3: Now, we have to add the hduser to the sudo list so that it can have admin properties.
sudo visudo |
This command will take you to the sudo user list. Now go to under # Allow members of group sudo to execute any command, add hduser and set ALL permission on it using:
hduser ALL=(ALL: ALL) ALL |
Now press Ctrl and x to exit, then Y to save and entre to return to the command prompt.
Step4: Now logout your system and log in with the new user hduser. From now on, we only work with the hduser.
Don't miss out!