spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tkg_cangkul <yuza.ras...@gmail.com>
Subject Re: error when running spark from oozie launcher
Date Thu, 18 Aug 2016 06:41:02 GMT
you're right jean,

it's mismatch library in default oozie sharelib with my spark version. 
i've replace it and it works normally now.

thx for your help Jean.

On 18/08/16 13:37, Jean-Baptiste Onofré wrote:
>
> It sounds like a mismatch in the spark version ship in oozie and the 
> runtime one.
>
> Regards
> JB
>
> On Aug 18, 2016, at 07:36, tkg_cangkul <yuza.rasfar@gmail.com 
> <mailto:yuza.rasfar@gmail.com>> wrote:
>
>     hi olivier, thx for your reply.
>
>     this is the full stacktrace :
>
>     Failing Oozie Launcher, Main class
>     [org.apache.oozie.action.hadoop.SparkMain], main() threw
>     exception,
>     org.apache.spark.launcher.CommandBuilderUtils.addPermGenSizeOpt(Ljava/util/List;)V
>     java.lang.NoSuchMethodError:
>     org.apache.spark.launcher.CommandBuilderUtils.addPermGenSizeOpt(Ljava/util/List;)V
>
>     On 18/08/16 13:28, Olivier Girardot wrote:
>>     this is not the full stacktrace, please post the full stacktrace
>>     if you want some help
>>
>>
>>
>>     On Wed, Aug 17, 2016 7:24 PM, tkg_cangkul yuza.rasfar@gmail.com
>>     <mailto:yuza.rasfar@gmail.com> wrote:
>>
>>         hi i try to submit job spark with oozie. but i've got one
>>         problem here.
>>         when i submit the same job. sometimes my job succeed but
>>         sometimes my job was failed.
>>
>>         i've got this error message when the job was failed :
>>
>>         org.apache.spark.launcher.CommandBuilderUtils.addPermGenSizeOpt(Ljava/util/List;)V
>>
>>
>>         anyone can help me to solve this? i've try to set
>>         -XX:MaxPermSize=512m  -XX:PermSize=256m in spark. driver.
>>         extraJavaOptions properties but this not help enough for me. 
>>
>>
>>
>>     **Olivier Girardot** ** | Associé
>>     o.girardot@lateral-thoughts.com
>>     <mailto:o.girardot@lateral-thoughts.com>
>>     +33 6 24 09 17 94
>


Mime
View raw message