Have a personal or library account? Click to login
Learning Spark SQL Cover

Learning Spark SQL

Architect streaming analytics and machine learning solutions

Publisher:Packt Publishing Limited
By: Aurobindo Sarkar  
Paid access
|Sep 2025
Product purchase options

Design, implement, and deliver successful streaming applications, machine learning pipelines and graph applications using Spark SQL API

Key Features

  • Learn about the design and implementation of streaming applications, machine learning pipelines, deep learning, and large-scale graph processing applications using Spark SQL APIs and Scala.
  • Learn data exploration, data munging, and how to process structured and semi-structured data using real-world datasets and gain hands-on exposure to the issues and challenges of working with noisy and "dirty" real-world data.
  • Understand design considerations for scalability and performance in web-scale Spark application architectures.

Book Description

In the past year, Apache Spark has been increasingly adopted for the development of distributed applications. Spark SQL APIs provide an optimized interface that helps developers build such applications quickly and easily. However, designing web-scale production applications using Spark SQL APIs can be a complex task. Hence, understanding the design and implementation best practices before you start your project will help you avoid these problems.
This book gives an insight into the engineering practices used to design and build real-world, Spark-based applications. The book's hands-on examples will give you the required confidence to work on any future projects you encounter in Spark SQL.
It starts by familiarizing you with data exploration and data munging tasks using Spark SQL and Scala. Extensive code examples will help you understand the methods used to implement typical use-cases for various types of applications. You will get a walkthrough of the key concepts and terms that are common to streaming, machine learning, and graph applications. You will also learn key performance-tuning details including Cost Based Optimization (Spark 2.2) in Spark SQL applications. Finally, you will move on to learning how such systems are architected and deployed for a successful delivery of your project.

What you will learn

  • Familiarize yourself with Spark SQL programming, including working with DataFrame/Dataset API and SQL
  • Perform a series of hands-on exercises with different types of data sources, including CSV, JSON, Avro, MySQL, and MongoDB
  • Perform data quality checks, data visualization, and basic statistical analysis tasks
  • Perform data munging tasks on publically available datasets
  • Learn how to use Spark SQL and Apache Kafka to build streaming applications
  • Learn key performance-tuning tips and tricks in Spark SQL applications
  • Learn key architectural components and patterns in large-scale Spark SQL applications

Who this book is for

If you are a developer, engineer, or an architect and want to learn how to use Apache Spark in a web-scale project, then this is the book for you. It is assumed that you have prior knowledge of SQL querying. A basic programming knowledge with Scala, Java, R, or Python is all you need to get started with this book.

Table of Contents

  1. Getting started with Spark SQL
  2. Using Spark SQL for Processing Structured and Semi-Structured Data
  3. Using Spark SQL for Data Exploration
  4. Using Spark SQL for Data Munging
  5. Using Spark SQL in Streaming Applications
  6. Using Spark SQL in Machine Learning Applications
  7. Using Spark SQL in Graph Applications
  8. Using Spark SQL with SparkR
  9. Developing Applications with Spark SQL
  10. Using Spark SQL in Deep Learning Applications
  11. Tuning Spark SQL Components for performance
  12. Spark SQL in Large-Scale Application Architectures
https://github.com/packtpublishing/learning-spark-sql
PDF ISBN: 978-1-78588-735-2
Publisher: Packt Publishing Limited
Copyright owner: © 2017 Packt Publishing Limited
Publication date: 2025
Language: English
Pages: 452
Related subjects:
Computer sciences,
Computer sciences, other

People also read

Apache Spark Quick Start Guide
Book

Apache Spark Quick Start Guide

Mehrotra, Shrey, Grade, Akash
Learning Apache Spark 2
Book

Learning Apache Spark 2

Apache Spark 2.x Cookbook
Book
New

Apache Spark 2.x Cookbook

Yadav, Rishi
Apache Spark 2.x for Java Developers
Book
New

Apache Spark 2.x for Java Developers

Kumar, Sumit, Gulati, Sourav
Mastering Spark for Data Science
Book
New

Mastering Spark for Data Science

Bifet, Albert, Hallett, Matthew, Morgan, Andrew, George, David, Amend, Antoine
Scala and Spark for Big Data Analytics
Book
New

Scala and Spark for Big Data Analytics

Alla, Sridhar, Karim, Md. Rezaul
Mastering Apache Spark 2.x
Book

Mastering Apache Spark 2.x

Spark for Data Science
Book
New

Spark for Data Science

Singhal, Bikramaditya, Duvvuri, Srinivas
Fast Data Processing with Spark 2
Book
New

Fast Data Processing with Spark 2

Sankar, Krishna, Karau, Holden
Apache Spark for Data Science Cookbook
Book
New

Apache Spark for Data Science Cookbook

Inelu, Nagamallikarjuna, Chitturi, Padma Priya