spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amey Chaugule <ambr...@gmail.com>
Subject Re: hadoop_conf_dir when running spark on yarn
Date Mon, 03 Nov 2014 04:29:18 GMT
I thought that only applied when you're trying to run a job using
spark-submit or in the shell...

On Sun, Nov 2, 2014 at 8:47 AM, Akhil Das <akhil@sigmoidanalytics.com>
wrote:

> You can set HADOOP_CONF_DIR inside the spark-env.sh file
>
> Thanks
> Best Regards
>
> On Sat, Nov 1, 2014 at 4:14 AM, ameyc <ambrood@gmail.com> wrote:
>
>> How do i setup hadoop_conf_dir correctly when I'm running my spark job on
>> Yarn? My Yarn environment has the correct hadoop_conf_dir settings by the
>> configuration that I pull from sc.hadoopConfiguration() is incorrect.
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/hadoop-conf-dir-when-running-spark-on-yarn-tp17872.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>>
>>
>


-- 
Amey S. Chaugule

B.S Computer Engineering and Mathematics '11,
University of Illinois at Urbana-Champaign

Mime
View raw message