spark architecture

Spark Installation In this section, we will perform the installation of Spark. So, follow the below steps. Download the Apache Spark tar…

Spark Char Count Example In Spark char count example, we find out the frequency of each character exists in a particular file.…

RDD Persistence Spark provides a convenient way to work on the dataset by persisting it in memory across operations. While persisting an…

Spark Architecture The Spark follows the master-slave architecture. Its cluster consists of a single master and multiple slaves. The Spark architecture depends…

RDD Shared Variables In Spark, when any function passed to a transformation operation, then it is executed on a remote cluster node.…

Spark Cartesian Function In Spark, the Cartesian function generates a Cartesian product of two datasets and returns all the possible combination of…

What is RDD? The RDD (Resilient Distributed Dataset) is the Spark’s core abstraction. It is a collection of elements, partitioned across the…

Spark reduceByKey Function In Spark, the reduceByKey function is a frequently used transformation operation that performs aggregation of data. It receives key-value…

Spark cogroup Function In Spark, the cogroup function performs on different datasets, let’s say, (K, V) and (K, W) and returns a…

Spark sortByKey Function In Spark, the sortByKey function maintains the order of elements. It receives key-value pairs (K, V) as an input,…