Data Engineering with Scala and Spark

Table of Contents

1. Scala Essentials for Data Engineers
2. Environment Setup
3. An Introduction to Apache Spark and Its APIs – DataFrame, Dataset, and Spark SQL
4. Working with Databases
5. Object Stores and Data Lakes
6. Understanding Data Transformation
7. Data Profiling and Data Quality
8. Test-Driven Development, Code Health, and Maintainability
9. CI/CD with GitHub
10. Data Pipeline Orchestration
11. Performance Tuning
12. Building Batch Pipelines Using Spark and Scala
13. Building Streaming Pipelines Using Spark and Scala