spark

TimeLIne Apache HUDI 作为数据湖框架的一种开源实现，提供了事务、高效的更新和删除、高级索引、 流式集成、小文件合并、log文件合并优化和并发支持等多种能力，支持实时消费增量数据、离

In the previous blog post, we looked into how the Adaptive Query Execution (AQE) framework is implem

Cost-based optimisation (CBO) is not a new thing. It has been widely used in the RDBMS world for man

I was planning to write about the Adaptive Query Execution (AQE) in this and next few blog posts, an

In this blog post, I will explain the Dynamic Partition Pruning (DPP), which is a performance optimi

This blog post continues to discuss the partitioning and ordering in Spark. In the last blog post, I

In the last blog post, I explained the partitioning and ordering requirements for preparing a physic

In the last few blog posts, I introduced the SparkPlanner for generating physical plans from logical

This blog post looks into Spark SQL Cache Commands under the hood, walking through the execution flo

In this blog posts, I will dig into the execution internals of the runnable commands, which inherit

In this blog post, I am going to explain the Join strategies applied by the Spark Planner for genera

This blog post continues to explore the Aggregate strategy and focuses on the two hash-based aggrega

The last blog post explains the Aggregation strategy for generating physical plans for aggregate ope

In the last blog post, I gave an overview of the SparkPlanner for planning physical execution plans

After logical plans are optimised by the Catalyst Optimizer rules, SparkPlanner takes an optimized l

After two lengthy blog posts on Catalyst Optimizer rules, this blog post will close this topic and c

In the previous blog post, I covered the rules included in the “Eliminate Distinct“, “Finish Analysi