Название: Delta Lake: The Definitive Guide (Early Release)Автор: Denny Lee, Tathagata Das & Vini Jaiswal
Издательство: O’Reilly Media
Год: 2021-04-20
Формат: epub
Размер: 10 Mb
Язык: English
Analysis and machine learning models are only as good as the data they???re built with. Querying processed data and gaining insights from it requires a robust data pipeline--and an effective storage solution that ensures data quality, data integrity, and performance.
This guide introduces you to Delta Lake, an open-source format that enables you to build a lakehouse architecture on top of existing storage systems such as S3, ADLS, GCS, and HDFS. Delta Lake enhances Apache Spark by supporting data integrity, data quality, and performance and making it easy to store and manage massive amounts of complex data.
Data engineers, data scientists, and data practitioners will learn how to build reliable data lakes and data pipelines at scale using Delta Lake.
Understand how to tackle key data reliability challenges
Learn how to use Delta Lake to realize data reliability improvements
Concurrently run streaming and batch jobs against your data lake
Execute update, delete, and merge commands against your data lake
Use Time Travel to roll back and examine previous versions of your data
Learn best practices to build effective, high-quality end-to-end data pipelines for real world use cases
Integrate with other data technologies like Presto, Athena, Redshift and other BI tools