3 months

Duration

Online Self-Paced

Format

8+

Projects

E&ICT, IIT Roorkee

Certificate

3,421

Ratings

15,200+

Learners

E&ICT

E&ICT Academy, IIT Roorkee

An initiative of Ministry of Electronics and Information Technology (MeitY) Govt. of India

About the Course

This Big Data Engineering Certification Program is a self-paced online course. This gives you complete freedom about your schedule and convenience.

This course has over 60+ hours of video content. As humans, we are immersed in data in our every-day lives. As per IBM, the data doubles every two years on this planet. The value that data holds can only be understood when we can start to identify patterns and trends in the data. Normal computing principles do not work when data becomes huge. There is massive growth in the big data space, and job opportunities are skyrocketing, making this the perfect time to launch your career in this space. In this specialization, you will learn Hadoop and Spark to drive better business decisions and solve real-world problems.

Additionally, this course comes with our cloud lab access to gain the much needed hands-on experience to solve the real-world problems.

Upon successfully completing the course, you will get the certificate from E&ICT Academy, IIT Roorkee which you can use for progressing in your career and finding better opportunities.

Program Highlights

  • Certificate of Completion by E&ICT Academy, IIT Roorkee

  • 60+ Hours of Learning

  • Work on about 8+ projects to get hands-on experience

  • Timely Doubt Resolution

  • Cloud Lab Access

Start Learning Today

Apply Now Request a Callback

Our Students Work At

Certificate

What is the certificate like?

  • Why E&ICT, IIT Roorkee?

    The Electronics & ICT Academy program is sponsored by the Ministry of Electronics and Information Technology, Govt. of India. It conducts short courses/FDPs in the emerging areas to enrich & upgrade subject knowledge and technical skills benefiting faculty, working professionals and Govt. employees.

    The trained beneficiaries are expected to create a cascading effect, transforming the competencies and standards in the parent institutes/organizations. E&ICT courses are at par with QIP for recognition/credits. As of now the E&ICT Academy, IIT Roorkee has conducted 91 courses and trained over 5,000 beneficiaries.

  • Why Cloudxlab?

    CloudxLab (CxL) has been a pioneer in the edtech space for the past few years. Founded in 2015 by Sandeep Giri, an alumnus of IIT Roorkee, CxL has successfully transformed 1,000's of students' careers by offering world-class certification courses in big data, machine learning and artificial intelligence.

    Some of the unique features of CxL are an exclusive gamified learning environment through the lab (read as CloudxLab), highest rated faculty, excellent student support and more.

Programming Languages and Tools

Hands-on Learning

hands-on lab

  • Gamified Learning Platform


  • Auto-assessment Tests


  • No Installation Required

Instructors

Instructor Sandeep Giri

Sandeep Giri

Founder at CloudxLab

Past: Amazon, InMobi, D.E.Shaw

Instructor Abhinav Singh

Abhinav Singh

Co-Founder at CloudxLab

Past: Byjus

Instructor Sanjeev Manhas

Sanjeev Manhas

Associate Professor

IIT Roorkee

Instructor R. Balasubramania

R. Balasubramanian

Professor

IIT Roorkee

Instructor Partha Pratim Roy

Partha Pratim Roy

Assistant Professor

IIT Roorkee

Instructor Jatin Shah

Jatin Shah

Ex-LinkedIn, Yahoo, Yale CS Ph.D., IITB

Course Advisor

Curriculum

60+
Hours of Training
90
Days of Lab Access
8+
Projects
15K+
Learners
Download Curriculum
1. Big Data Introduction
1. Introduction
2. Distributed systems
3. Big Data Use Cases
4. Various Solutions
5. Overview of Hadoop Ecosystem
6. Spark Ecosystem Walkthrough
7. Quiz
2. Foundation & Environment
1. Quiz
2. Getting Started - Hands on
3. Hadoop & Spark Hands-on
4. Understanding Regular Expressions
5. Setting up VM
3. Zookeeper
1. ZooKeeper - Race Condition
2. ZooKeeper - Deadlock
3. How does election happen - Paxos Algorithm?
4. Use cases
5. When not to use
6. Quiz & Assessment
4. HDFS
1. Why HDFS?
2. NameNode & DataNodes
3. Advance HDFS Concepts (HA, Federation)
4. Hands-on with HDFS (Upload, Download, SetRep)
5. Data Locality (Rack Awareness)
6. Quiz & Assessment
5. YARN
1. Why YARN?
2. Evolution from MapReduce 1.0
3. Resource Management: YARN Architecture
4. Advance Concepts - Speculative Execution
5. Quiz & Assessment
6. MapReduce Basics
1. Understanding Sorting
2. MapReduce - Overview
3. Word Frequency Problem - Without MR
4. Only Mapper - Image Resizing
5. Temperature Problem
6. Multiple Reducer
7. Java MapReduce
7. MapReduce Advanced
1. Writing MapReduce Code Using Java
2. Apache Ant
3. Concept - Associative & Commutative
4. Combiner
5. Hadoop Streaming
6. Adv. Problem Solving - Anagrams
7. Adv. Problem Solving - Same DNA
8. Adv. Problem Solving - Similar DNA
9. Joins - Voting
10. Limitations of MapReduce
8. Analyzing Data with Pig
1. Pig - Introduction
2. Pig - Modes
3. Example - NYSE Stock Exchange
4. Concept - Lazy Evaluation
9. Processing Data with Hive
1. Hive - Introduction
2. Hive - Data Types
3. Loading Data in Hive (Tables)
4. Movielens Data Processing
5. Connecting Tableau and HiveServer 2
6. Connecting Microsoft Excel and HiveServer 2
7. Project: Sentiment Analyses of Twitter Data
8. Advanced - Partition Tables
9. Understanding HCatalog & Impal
10. NoSQL and HBase
1. NoSQL - Scaling Out / Up
2. ACID Properties and RDBMS Story
3. CAP Theorem
4. HBase Architecture - Region Servers etc
5. Hbase Data Model - Column Family Orientedness
6. Getting Started - Create table, Adding Data
7. Adv Example - Google Links Storage
8. Concept - Bloom Filter
9. Comparison of NOSQL Databases
11. Importing Data with Sqoop and Flume, Oozie
1. Sqoop - Introduction
2. Sqoop Import - MySQL to HDFS
3. Exporting to MySQL from HDFS
4. Concept - Unbounding Dataset Processing or Stream Processing
5. Flume Overview: Agents - Source, Sink, Channel
6. Data from Local network service into HDFS
7. Example - Extracting Twitter Data
8. Example - Creating workflow with Oozier
12. Scala Basics
1. Variables and Methods
2. Interactive, Compilation, SBT
3. Types, Variables & Values
4. Functions
5. Collections
6. Classes
7. Parameters
13. Spark Basics
1. Apache Spark ecosystem
2. Why Spark?
3. Using the Spark Shell on CloudxLab
4. Example 1 - Performing Word Count
5. Understanding Spark Cluster Modes on YARN
6. RDDs (Resilient Distributed Datasets)
7. General RDD Operations: Transformations & Actions
8. RDD lineage
9. RDD Persistence Overview
10. Distributed Persistence
14. Writing and Deploying Spark Applications
1. The Spark Application Web UI
2. Configuring Spark Properties
3. Running Spark on Cluster
4. RDD Partitions
5. Executing Parallel Operations
6. Stages and Tasks
7. RDD Persistence Overview
8. Distributed Persistence
15. Writing and Deploying Spark Applications
1. Types, Variables & Values
2. Functions
3. Collections
4. Classes
5. Parameters
16. Common Patterns in Spark Data Processing
1. Example 2 - Understanding Spark Streaming
2. Understanding Kafka
3. Example 3 - Spark Streaming from Kafka
4. Iterative Algorithms in Spark
5. Project: Real-time analytics of orders in an e-commerce company
17. Data Formats & Management
1. XML
2. AVRO
3. How to store many small files - SequenceFile?
4. Parquet
5. Protocol Buffers
6. Comparing Compressions
7. Understanding Row Oriented and Column Oriented Formats - RCFile?
18. DataFrames and Spark SQL
1. Spark SQL - Introduction
2. Spark SQL - Dataframe Introduction
3. Transforming and Querying DataFrames
4. Saving DataFrames
5. DataFrames and RDDs
6. Comparing Spark SQL, Impala, and Hive-on-Spark
19. Machine Learning with Spark
1. Machine Learning Introduction
2. Applications Of Machine Learning

Testimonials

Enroll Now

Eligibility Criteria for Certifcate

You need to complete at least 60% of the topics from the course. You also need to complete projects - Analyse emails from Python, Sentiment Analysis (Hive) from Hadoop.

All the above requirements need to be met within 90 days from the course enrollment date to be eligible for the certificate from E&ICT Academy, IIT Roorkee.

Audit this course

99

  • Self_Paced
  • 60+ hours of self-paced content
  • 90 days lab
  • Timely Doubt Resolution
  • Certificate from E&ICT, IIT Roorkee
Apply Now

109

  • Self_Paced
  • 60+ hours of self-paced content
  • 120 days lab
  • Timely Doubt Resolution
  • Certificate from E&ICT, IIT Roorkee
Apply Now

119

  • Self_Paced
  • 60+ hours of self-paced content
  • 150 days lab
  • Timely Doubt Resolution
  • Certificate from E&ICT, IIT Roorkee
Apply Now

Related Courses

Data Science

Data Science Specialization
EICT, IIT Roorkee

220+ Hours of Training | 26+ Projects


View Details
Machine Learning Specialization

Machine Learning Specialization
EICT, IIT Roorkee

170+ Hours of Training | 18+ Projects


View Details

Frequently Asked Questions

What things I need to fulfil to claim the certificate?

You need to complete at least 60% of topics from the course and submit required projects within 90 days from the course enrollment date. There are total of 8 projects among which submission of 2 projects is mandatory - Sentiment Analysis (Hive) and Log Parsing (Spark).

After completing the projects, please send an email to reachus@cloudxlab.com. We will forward your details to EICT, IIT Roorkee and EICT will be issuing your certificate. Please note that it may take 3-7 days to issue the certificate.

Do I need to install any software before starting this course?

No, we will provide you with the access to our online lab and BootML so that you do not have to install anything on your local machine

What do you mean by self-paced learning?

In Self-paced learning, you will get,

  1. Lifetime access to the self-paced course including videos, assessments, quizzes, and projects
  2. Recordings of the previous batch of instructor-led training
  3. 24x7 support using the discussion forum

What are the prerequisites for this course?

  1. Basics Of SQL. You should know the basics of SQL and databases. If you know about filters in SQL, you are expected to understand the course.

  2. A know-how of the basics of programming. If you understand 'loops' in any programming language, and if you are able to create a directory and see what's inside a file from the command line, you are good to get the concepts of this course even if you have not really touched programming for the last 10 years! In addition, we will be providing video classes on the basics of Python and Scala.

What is the validity of course material?

We understand that you might need course material for a longer duration to make most out of your subscription. You will get lifetime access to the course material so that you can refer to the course material anytime.

Why is the course available at such a low price?

The E&ICT Academy project is sponsored by the Ministry of Electronics and Information Technology, Govt. of India. The E&ICT courses lay special emphasis on hands-on learning with participation from industry in the emerging areas of the E&ICT domain. Their programs enable the participants and institutes to build industry connects, upgrade lab facilities and create opportunities for collaboration For this course, CloudxLab has joined hands with E&ICT, IIT Roorkee in order to fulfill the vision of The Ministry of Electronics and Information Technology (MeitY), to upskill the faculties across the country on high-end technologies. Hence it is available at such low prices so that every eligible person can pursue it and benefit from it wrt his career growth and country's growth as well.

How to view the course after getting access to it?

Please log in at CloudxLab.com with your Gmail Id and access your course under "My Courses".

What are the hardware and software requirements?

Course requires a good internet connection and a browser to watch videos and do hands-on the lab. We've configured all the tools in the lab so that you can focus on learning and practising in a real-world cluster.

Can I pursue a career in Business Analytics after this course?

These days most of the companies are using Big Data systems like Hive and Spark for analytics. After the course, you will have a good knowledge of mining data using Hive and SparkSQL. Also, you will learn how to run your "R" code in SparkR.

What is the refund policy for courses taken from CloudxLab?

For self-paced course, we provide 100% fees refund if the request is raised within 7 days from enrollment date. Please contact us at reachus@cloudxlab.com to request a refund within the stipulated time. Thereafter, no refund is provided.

Do we have to pay separately for the lab?

No, the lab is available within the course price.

Can I renew my lab subscription?

Yes, you can renew your subscription anytime. Please choose your desired plan for the lab and make payment to renew your subscription.