Add hive-site.xml at runtime

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Add hive-site.xml at runtime

Shivam Sharma
Hi,

I have multiple hive configurations(hive-site.xml) and because of that only I am not able to add any hive configuration in spark conf directory. I want to add this configuration file at start of any spark-submit or spark-shell. This conf file is huge so --conf is not a option for me.

Note :- I have tried [hidden email] and my mail was bouncing each time so Sean Owen suggested to mail dev.(https://issues.apache.org/jira/browse/SPARK-19546). Please give solution to above ticket also if possible.

Thanks

--
Shivam Sharma
Reply | Threaded
Open this post in threaded view
|

Re: Add hive-site.xml at runtime

Ryan Blue
Shivam,

We add hive-site.xml at runtime. We use --driver-class-path to add it to the driver and --jars to add it for the executors.

rb

On Sun, Feb 12, 2017 at 9:10 AM, Shivam Sharma <[hidden email]> wrote:
Hi,

I have multiple hive configurations(hive-site.xml) and because of that only I am not able to add any hive configuration in spark conf directory. I want to add this configuration file at start of any spark-submit or spark-shell. This conf file is huge so --conf is not a option for me.

Note :- I have tried [hidden email] and my mail was bouncing each time so Sean Owen suggested to mail dev.(https://issues.apache.org/jira/browse/SPARK-19546). Please give solution to above ticket also if possible.

Thanks

--
Shivam Sharma



--
Ryan Blue
Software Engineer
Netflix