spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Retrieve hadoop conf object from Python API
Date Wed, 01 Jul 2015 00:52:49 GMT
Minor correction:
It should be sc._jsc

Cheers

On Tue, Jun 30, 2015 at 4:23 PM, ayan guha <guha.ayan@gmail.com> wrote:

> There is a sc._jsc_ which you can access to get/set hadoop conf.
>
> On Wed, Jul 1, 2015 at 7:41 AM, Richard Ding <piguser@gmail.com> wrote:
>
>> Hi,
>>
>> I noticed that,  in Scala API, one can call hadoopConfiguration on
>> SparkContext to retrieve the hadoop configuration object which is very
>> handy in modifying certain hadoop properties at runtime. But there is no
>> corresponding method in Python API.
>>
>> Will this method be added to Python API in a later release? In the mean
>> time, what will be the work around (of setting hadoop properties at
>> runtime) using Python API?
>>
>> Thanks,
>> Richard
>>
>
>
>
> --
> Best Regards,
> Ayan Guha
>

Mime
View raw message