How to enable dynamic resource allocation in Spark

Question

I want to enable dynamic resource allocation for my spark application and I want to do it dynamically without having to hard code it. How can I do it?

Answer 1

To dynamically enable dynamic resource allocation, you need to run this command in the Spark shell:

val sc = new SparkContext(new SparkConf())

./bin/spark-submit <all your existing options> --spark.dynamicAllocation.enabled=true