Apache SparkÁ÷´¦Àí(Ó°Ó¡°æ)

Foreword Preface Part ¢ñ. Fundamentals of Stream Processing with Apache Spark 1. Introducing Stream Processing What Is Stream Processing? Batch Versus Stream Processing The Notion of Time in Stream Processing The Factor of Uncertainty Some Examples of Stream Processing Scaling Up Data Processing MapReduce The Lesson Learned: Scalability and Fault Tolerance Distributed Stream Processing Stateful Stream Processing in a Distributed System Introducing Apache Spark The First Wave: Functional APIs The Second Wave: SQL A Unified Engine Spark Components Spark Streaming Structured Streaming Where Next? 2. Stream-Processing Model Sources and Sinks Immutable Streams Defined from One Another Transformations and Aggregations Window Aggregations Tumbling Windows Sliding Windows Stateless and Stateful Processing Stateful Streams An Example: Local Stateful Computation in Scala A Stateless Definition of the Fibonacci Sequence as a Stream Transformation Stateless or Stateful Streaming The Effect of Time Computing on Timestamped Events Timestamps as the Provider of the Notion of Time Event Time Versus Processing Time Computing with a Watermark Summary 3. Streaming Architectures Components of a Data Platform Architectural Models The Use of a Batch-Processing Component in a Streaming Application Referential Streaming Architectures The Lambda Architecture The Kappa Architecture Streaming Versus Batch Algorithms Streaming Algorithms Are Sometimes Completely Different in Nature Streaming Algorithms Can't Be Guaranteed to Measure Well Against Batch Algorithms Summary 4. Apache Spark as a Stream-Processing Engine The Tale of Two APIs Spark's Memory Usage Failure Recovery Lazy Evaluation Cache Hints Understanding Latency Throughput-Oriented Processing Spark's Polyglot API ¡­¡­ Part ¢ò Structured Streaming Part ¢ó Spark Streaming Part ¢ô Advanced Spark Streaming Techniques Part ¢õ Beyond Apache Spark E.References for Part ¢õ Index