Network encryption for Spark

Question

How can I add encryption for the network in Spark?

Answer 1

You can enable encryption for the Spark network as follows:

val sc = new SparkContext(new SparkConf())

./bin/spark-submit <all your existing options> --spark.network.crypto.enabled=true

This will enable AES-based RPC encryption.