spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Valdivia <h...@danielvaldivia.com>
Subject Re: Missing dependencies when submitting scala app
Date Thu, 24 Dec 2015 00:24:09 GMT
Hi Jeff,

The problem was I was pulling json4s 3.3.0 which seems to have some problem
with spark, I switched to 3.2.11 and everything is fine now

On Tue, Dec 22, 2015 at 5:36 PM, Jeff Zhang <zjffdu@gmail.com> wrote:

> It might be jar conflict issue. Spark has dependency org.json4s.jackson,
> do you also specify org.json4s.jackson in your sbt dependency but with a
> different version ?
>
> On Wed, Dec 23, 2015 at 6:15 AM, Daniel Valdivia <hola@danielvaldivia.com>
> wrote:
>
>> Hi,
>>
>> I'm trying to figure out how to bundle dependendies with a scala
>> application, so far my code was tested successfully on the spark-shell
>> however now that I'm trying to run it as a stand alone application which
>> I'm compilin with sbt is yielding me the error:
>>
>>
>> *java.lang.NoSuchMethodError:
>> org.json4s.jackson.JsonMethods$.parse$default$3()Z at
>> ClusterIncidents$$anonfun$1.apply(ClusterInciden*
>>
>> I'm doing "sbt clean package" and then spark-submit of the resulting jar,
>> however seems like either my driver or workers don't have the json4s
>> dependency, therefor can't find the parse method
>>
>> Any idea on how to solve this depdendency problem?
>>
>> thanks in advance
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Mime
View raw message