spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kant kodali <kanth...@gmail.com>
Subject Re: Gradle dependency problem with spark
Date Fri, 16 Dec 2016 11:04:36 GMT
I replaced *guava-14.0.1.jar*  with *guava-19.0.jar in *SPARK_HOME/jars and
seem to work ok but I am not sure if it is the right thing to do. My fear
is that if Spark uses features from Guava that are only present in 14.0.1
but not in 19.0 I guess my app will break.



On Fri, Dec 16, 2016 at 2:22 AM, kant kodali <kanth909@gmail.com> wrote:

> Hi Guys,
>
> Here is the simplified version of my problem. I have the following problem
> and I new to gradle
>
>
> dependencies {
>     compile group: 'org.apache.spark', name: 'spark-core_2.11', version: '2.0.2'
>     compile group: 'com.github.brainlag', name: 'nsq-client', version: '1.0.0.RC2'
> }
>
>
> I took out the other dependencies for simplicity. The problem here
> is spark-core_2.11 uses com.google.guava:14.0.1 and nsq-client uses
> com.google.guava:19.0 so when I submit my fat uber jar using spark-submit I
> get the following error
>
> Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.collect.Sets.newConcurrentHashSet()Ljava/util/Set;
>     at com.github.brainlag.nsq.NSQProducer.(NSQProducer.java:22)
>     at com.hello.streamprocessing.app.SparkDriver2.main(SparkDriver2.java:37)
>
>
> any help would be great. Also if you need more description you can find it
> here
> <http://stackoverflow.com/questions/41003416/how-to-solve-nomethoderror-that-arises-due-to-using-a-same-library-with-two-diff>
>
> Thanks!
>
>

Mime
View raw message