Data Engineering with Python

Table of Contents

1. What is Data Engineering?
2. Building Our Data Engineering Infrastructure
3. Reading and Writing Files
4. Working with Databases
5. Cleaning, Transforming, and Enriching Data
6. Building a 311 Data Pipeline
7. Features of a Production Pipeline
8. Version Control Using the NiFi Registry
9. Monitoring and Logging Pipelines
10. Deploying Your Pipelines
11. Building a Production Data Pipeline
12. Building a Kafka Cluster
13. Streaming Data with Apache Kafka
14. Data Processing with Apache Spark
15. Real-Time Edge Data with MiNiFi, Kafka, and Spark
16. Appendix