Introduction to Big Data and Hadoop

Course Overview

This course offers a one-of-a-kind rich and interactive experience to learn the fundamentals and basics of Big Data. Throughout this course, you will have plenty of opportunities to get your hands dirty with functioning Hadoop clusters. You will start off by learning about the rise of... See More

How You'll Learn

Hands-on coding environments

You don’t get better at swimming by watching others. Coding is no different. Practice as you learn with live code environments inside your browser.

Faster than videos

Videos are holding you back. The average video tutorial is spoken at 150 words per minute, while you can read at 250. That‘s why our courses are text-based.

No set-up required

Start learning immediately instead of fiddling with SDKs and IDEs. It‘s all on the cloud.

Progress you can show

Built in assessments let you test your skills. Completion certificates let you show them off.

Hands-on coding environments

You don’t get better at swimming by watching others. Coding is no different. Practice as you learn with live code environments inside your browser.

Faster than videos

Videos are holding you back. The average video tutorial is spoken at 150 words per minute, while you can read at 250. That‘s why our courses are text-based.

No set-up required

Start learning immediately instead of fiddling with SDKs and IDEs. It‘s all on the cloud.

Progress you can show

Built in assessments let you test your skills. Completion certificates let you show them off.

Course Contents

Hide All

Hide All Lessons

1. Hadoop

Introduction

Rise of Big Data

Types of Data

Big Data Defined

Big Data vs Data Warehouse

2. YARN

Introduction

Workflow

Scheduling

3. Map Reduce

Basics

Mapper

Testing Mapper

Mapper Input

Reducer

Testing Reducer

Testing MapReduce Program

Running MapReduce End to End

Exploring MapReduce Runs

Combiner and Partitioner

Putting it Together

Resiliency

4. HDFS

Filesystem

The Big Picture

Disk Blocks & HDFS Blocks

Block Replication

Namenode

Datanode

Writing and Reading

High Availability

HDFS in Practice

HDFS in Practice II

Distcp

5. Spark

Introduction

Architecture

Spark Application Life Cycle

Spark API

Resilient Distributed Datasets (RDDs)

DataFrames

Datasets

An Example

Running Spark Applications

Anatomy of a Spark Application

Execution of a Spark Application

6. Input & Output Formats

Sequence File: Intro

Sequence File: Reading & Writing

SerDe

Rows vs Columnar Databases

Avro: Intro

Avro: Code Generation

Avro: IDL & RPC

Parquet: Intro

Parquet: Definition Level

Parquet: Repetition Level

Parquet: Reading & Writing

Parquet: Projection Schema & Misc. Tools

7. Misc

Zookeeper: Intro

Zookeeper: Example

Zookeeper: Practical

Pig: Overview

Summary

8. Quiz

Quiz 1

Quiz 2

Quiz 3

Quiz 4

Quiz 5

Quiz 6

9. Reference: Replication