i
Characteristics of Big Data
Application of Big Data Processing
Introduction to BIG DATA
Where to get Big Data?
Types of Big Data
Storage layer - HDFS (Hadoop Distributed File System)
MapReduce
YARN
How Hadoop works?
Hadoop Eco System
Hadoop Architecture
Hadoop Installation & Environment Setup
Setting Up A Single Node Hadoop Cluster
Ubuntu User Configuration
SSH Setup With Key Generation
Disable IPv6
Download and Install Hadoop 3.1.2
Working with Configuration Files
Start The Hadoop instances
Hadoop Distributed File System (HDFS)
HDFS Features and Goals
HDFS Architecture
Read Operations in HDFS
Write Operations In HDFS
HDFS Operations
YARN
YARN Features
YARN Architecture
Resource Manager
Node Manager
Application Master
Container
Application Workflow in Hadoop YARN
Hadoop MapReduce
How MapReduce Works?
MapReduce Examples with Python
Running The MapReduce Program & Storing The Data File To HDFS
Create A Python Script
Hadoop Environment Setup
Execute The Script
Apache Hive Definition
Why Apache Hive?
Features Of Apache Hive
Hive Architecture
Hive Metastore
Hive Query Language
SQL vs Hive
Hive Installation
Apache Pig Definition
MapReduce vs. Apache Pig vs. Hive
Apache Pig Architecture
Installation Process Of Apache Pig
Execute Apache Pig Script
Hadoop Eco Components
NoSQL Data Management
Apache Hbase
Apache Cassandra
Mongodb
Introduction To Kafka
The Architecture of Apache Flume
Apache Spark Ecosystem
We can classify big data into three main categories.
Structured Data:
Structured data refers to the data that resides in a fixed field within a record or file, which includes data contained in relational databases and spreadsheets. It has the advantage of being easily entered, queried, stored, and analyzed. At one time, relational databases, and spreadsheets using structured data was the only way to manage data effectively, because of the high cost and performance limitations of storage, memory, and processing, relational databases, and However, nowadays, we are foreseeing issues when the size of such data grows to a considerable extent; typical sizes are being in the rage of multiple zettabytes.
Example of Structured Data:
The medical history of Heart patients can be an excellent example of structured data.
Table: Heart Patient Data
Un-Structured Data:
Any data with the unknown form of the structure is classified as unstructured data. Despite the huge size, unstructured data poses many challenges in terms of its processing capabilities.
A typical example of unstructured data is the heterogeneous data source containing a combination of simple text files, images, videos, etc.
Semi-structured Data:
Semi-structured data can contain both structured and unstructured forms of data. It is neither like Relational data nor text or image data.
An excellent example of semi-structured data is data represented in an XML file format.
Don't miss out!