spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Baptiste Onofré ...@nanthrax.net>
Subject Re: error when running spark from oozie launcher
Date Thu, 18 Aug 2016 06:37:39 GMT
It sounds like a mismatch in the spark version ship in oozie and the runtime one.

Regards
JB



On Aug 18, 2016, 07:36, at 07:36, tkg_cangkul <yuza.rasfar@gmail.com> wrote:
>hi olivier, thx for your reply.
>
>this is the full stacktrace :
>
>Failing Oozie Launcher, Main class 
>[org.apache.oozie.action.hadoop.SparkMain], main() threw exception, 
>org.apache.spark.launcher.CommandBuilderUtils.addPermGenSizeOpt(Ljava/util/List;)V
>java.lang.NoSuchMethodError: 
>org.apache.spark.launcher.CommandBuilderUtils.addPermGenSizeOpt(Ljava/util/List;)V
>
>On 18/08/16 13:28, Olivier Girardot wrote:
>> this is not the full stacktrace, please post the full stacktrace if 
>> you want some help
>>
>>
>>
>> On Wed, Aug 17, 2016 7:24 PM, tkg_cangkul yuza.rasfar@gmail.com 
>> <mailto:yuza.rasfar@gmail.com> wrote:
>>
>>     hi i try to submit job spark with oozie. but i've got one problem
>>     here.
>>     when i submit the same job. sometimes my job succeed but
>sometimes
>>     my job was failed.
>>
>>     i've got this error message when the job was failed :
>>
>>    
>org.apache.spark.launcher.CommandBuilderUtils.addPermGenSizeOpt(Ljava/util/List;)V
>>
>>     anyone can help me to solve this? i've try to set
>>     -XX:MaxPermSize=512m  -XX:PermSize=256m in
>>     spark.driver.extraJavaOptions properties but this not help enough
>>     for me. 
>>
>>
>>
>> **Olivier Girardot****| Associé
>> o.girardot@lateral-thoughts.com
><mailto:o.girardot@lateral-thoughts.com>
>> +33 6 24 09 17 94

Mime
View raw message