[Deploy] Should we make spark-daemon.sh busybox compatible?

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
Report Content as Inappropriate

[Deploy] Should we make spark-daemon.sh busybox compatible?



working on a dockerized Spark deployment I noticed the script sbin/spark-daemon.sh does not play nice with busybox.

The call "ps -p <pid>" [1] and the syntax "nohup -- <cmd>" [2] do not work with busybox.

Using an alpine based setup [1] requires installing the procps package (400KB installed size) and [2] the coreutils (5.5MB installed size). This is not much but adds to the image size and imposes more requirements for Spark to run.

These issues seem local to this script (I found no other occurences of the above syntax). Should we rewrite these lines?

Best regards,


[1] https://github.com/apache/spark/blob/master/sbin/spark-daemon.sh#L136, https://github.com/apache/spark/blob/master/sbin/spark-daemon.sh#L144, ...

[2] https://github.com/apache/spark/blob/master/sbin/spark-daemon.sh#L128