In this article, we will discuss several ways to check the Apache Spark version installed on our system. To check the Apache Spark version, you can use one of the following methods:
1. Command-Line Interface (CLI)
Open a terminal or command prompt and run the following command:
spark-submit --version
This command will display the Spark version along with other information.
2. Spark Shell
If you have Spark installed locally, you can launch the Spark Shell by running the following command:
spark-shell
Once the shell starts, you will see the Spark version displayed in the output logs.
3. Programmatically using Scala
If you have a Spark application written in Scala, you can check the Spark version programmatically by adding the following code snippet to your application:
import org.apache.spark.SparkContext val sparkContext = new SparkContext() val sparkVersion = sparkContext.version println(s"Spark version: $sparkVersion")
This code creates a SparkContext and retrieves the version using the version method.
4. Programmatically using Python
If you have a Spark application written in Python, you can check the Spark version programmatically by adding the following code snippet to your application:
from pyspark.sql import SparkSession
spark = SparkSession.builder.getOrCreate()
sparkVersion = spark.version
print("Spark version:", sparkVersion)
This code creates a SparkSession and retrieves the version using the version attribute.
Conclusion
Any of these methods will provide you with the Apache Spark version currently installed or being used.
