spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joseph Bradley <jos...@databricks.com>
Subject Re: Random Forest driver memory
Date Thu, 18 Jun 2015 19:54:26 GMT
Hi Isca,

Could you please give more details?  Data size, model parameters, stack
traces / logs, etc. to help get a better picture?

Thanks,
Joseph

On Wed, Jun 17, 2015 at 9:56 AM, Isca Harmatz <pop1998@gmail.com> wrote:

> hello,
>
> does anyone has any help on the issue?
>
>
> Isca
>
> On Tue, Jun 16, 2015 at 7:45 AM, Isca Harmatz <pop1998@gmail.com> wrote:
>
>> hello,
>>
>> i have noticed that the random forest implementation crashes when
>> to many trees/ to big maxDepth is used.
>>
>> im guessing that this is something to do with the amount of nodes that
>> need to be
>> kept in driver's memory during the run.
>>
>> but when i examined the nodes structure is seems rather small
>>
>> does anyone now where does the memory issue come from?
>>
>> thanks,
>>   Isca
>>
>
>

Mime
View raw message