Certificate Authentication

Spark for Developers Certificate for Klaudia Pietras

Add to LinkedIn

Certificate ID: 
654255
Authentication Code: 
169c8
Certified Person Name: 
Klaudia Pietras
Trainer Name: 
Marek Czuma
Duration Days: 
4
Duration Hours: 
28
Course Name: 
Spark for Developers
Course Date: 
2021-10-19 09:00 to 2021-10-22 16:00
Course Outline: 

  1. Scala primer

    • A quick introduction to Scala
    • Labs : Getting know Scala

  2. Spark Basics

    • Background and history
    • Spark and Hadoop
    • Spark concepts and architecture
    • Spark eco system (core, spark sql, mlib, streaming)
    • Labs : Installing and running Spark

  3. First Look at Spark

    • Running Spark in local mode
    • Spark web UI
    • Spark shell
    • Analyzing dataset – part 1
    • Inspecting RDDs
    • Labs: Spark shell exploration

  4. RDDs

    • RDDs concepts
    • Partitions
    • RDD Operations / transformations
    • RDD types
    • Key-Value pair RDDs
    • MapReduce on RDD
    • Caching and persistence
    • Labs : creating & inspecting RDDs;   Caching RDDs

  5. Spark API programming

    • Introduction to Spark API / RDD API
    • Submitting the first program to Spark
    • Debugging / logging
    • Configuration properties
    • Labs : Programming in Spark API, Submitting jobs

  6. Spark SQL

    • SQL support in Spark
    • Dataframes
    • Defining tables and importing datasets
    • Querying data frames using SQL
    • Storage formats : JSON / Parquet
    • Labs : Creating and querying data frames; evaluating data formats

  7. MLlib

    • MLlib intro
    • MLlib algorithms
    • Labs : Writing MLib applications

  8. GraphX

    • GraphX library overview
    • GraphX APIs
    • Labs : Processing graph data using Spark

  9. Spark Streaming

    • Streaming overview
    • Evaluating Streaming platforms
    • Streaming operations
    • Sliding window operations
    • Labs : Writing spark streaming applications

  10. Spark and Hadoop

    • Hadoop Intro (HDFS / YARN)
    • Hadoop + Spark architecture
    • Running Spark on Hadoop YARN
    • Processing HDFS files using Spark

  11. Spark Performance and Tuning

    • Broadcast variables
    • Accumulators
    • Memory management & caching

  12. Spark Operations

    • Deploying Spark in production
    • Sample deployment templates
    • Configurations
    • Monitoring
    • Troubleshooting