Inicio
All Courses
Partnership Courses
Spark in Scala
Spark in Scala
Currículum
5 Sections
19 Lessons
20 Hours
Expand all sections
Collapse all sections
Module 1: Spark features I: the computational model (4 hours)
The basic principles and concepts behind Spark, as a framework for distributed processing.
3
0.0
Transformations and actions; jobs, stages and tasks
0.1
Cluster managers: Yarn, Standalone, Mesos
0.2
Driver and executors; SparkUI
Module 2: Spark features II: Spark APIs (4 hours)
How do we process massive distributed data sets in a cluster? With high-level APIs! We have two major alternative APIs available at your fingertips: statically typed and dynamically typed.
4
0.0
Spark languages: SparkSQL, RDDs, ML, GraphX
0.1
Dataset: Statically typed
0.2
DataFrame: Dynamically unsafe
0.3
Datasets vs DataFrames
Module 3: Spark features III: Reading and writing in Spark (4 hours)
The last module focused on transformations, whereas this one focuses on the data side: formats, optimizations, management, etc.
3
0.0
File formats & optimizations: JSON, Parquet
0.1
Spark catalog management
0.2
A brief intro to Delta Lake
Module 4: Spark optimizations (4 hours)
Learn how to take the most of the spark optimizations for free.
3
0.0
Datasets vs DataFrames optimizations
0.1
Optimized file formats vs non-optimized
0.2
The standard Catalog API
Module 5. Best practices on performance & modular design (4 hours)
Learn the best ways to optimize and organize your Spark code to make it more robust and performant.
6
0.0
Partitioning issues: Unpartitioned data and over-partitioning
0.1
Fixing memory problems
0.2
How to solve serialization issues
0.3
Caching: when it improves your process, and when is extra work
0.4
Tasks that never finish: detect why this is happening
0.5
Workflow structure: design patterns to properly modularize your ETLs, and improve testability
Spark catalog management
El contenido de la lección esta vacío.
File formats & optimizations: JSON, Parquet
A brief intro to Delta Lake
Leave your thought here
Cancelar la respuesta
Guarda mi nombre, correo electrónico y web en este navegador para la próxima vez que comente.
Δ
Hide similarities
Highlight differences
Select the fields to be shown. Others will be hidden. Drag and drop to rearrange the order.
Image
SKU
Rating
Price
Stock
Availability
Add to cart
Description
Content
Weight
Dimensions
Additional information
Click outside to hide the comparison bar
Compare
Workshops
Profesores
Empresas
Blog
Contacto
Membresías
¿Interesado en más workshops?
Suscríbete y recibe nuestro boletín de noticias
Email
Subscríbete
Tu información nunca será compartida con terceros
Modal title
Main Content