How to set maximum receiving rate for backpressure mechanism

Question

I have enabled the backpressure mechanism for my Spark application. I want to set a maximum rate of receiving the first batch data. How can I do this?

Answer 1

You can set the maximum receiving rate by using the following commands in the spark shell:

val sc = new SparkContext(new SparkConf())

./bin/spark-submit <all your existing options> --spark.streaming.backpressure.initialRate=<max receiving rate>