spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cheng Lian <lian.cs....@gmail.com>
Subject Re: Got error β€œ"java.lang.IllegalAccessError" when using HiveContext in Spark shell on AWS
Date Thu, 07 Aug 2014 10:56:46 GMT
Hey Zhun,

Thanks for the detailed problem description. Please see my comments inlined
below.

On Thu, Aug 7, 2014 at 6:18 PM, Zhun Shen <shenzhunallen@gmail.com> wrote:

Caused by: java.lang.IllegalAccessError: tried to access method
> com.google.common.collect.MapMaker.makeComputingMap(Lcom/google/common/base/Function;)Ljava/util/concurrent/ConcurrentMap;
> from class com.jolbox.bonecp.BoneCPDataSource
>
This line indicates that accessing MapMaker.makeComputingMap via Java
reflection fails. The version of Guava we used in Spark SQL (as a
transitive dependency) is 14.0.1. In this version, MapMaker.makeComputingMap
is still public
<https://code.google.com/p/guava-libraries/source/browse/guava/src/com/google/common/collect/MapMaker.java?name=v14.0.1#581>.
But in newer versions (say 15.0), it’s no longer public
<https://code.google.com/p/guava-libraries/source/browse/guava/src/com/google/common/collect/MapMaker.java?name=v15.0>
.

So my guess is that, a newer version of the Guava library in your classpath
shadows the version Spark SQL uses somehow. A quick and dirty fix to see
whether this is true is try putting Guava 14.0.1 jar file at the beginning
of your classpath and see whether things work.

         at
> com.jolbox.bonecp.BoneCPDataSource.<init>(BoneCPDataSource.java:64)
>         at
> org.datanucleus.store.rdbms.datasource.BoneCPDataSourceFactory.makePooledDataSource(BoneCPDataSourceFactory.java:73)
>         at
> org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:217)
>         at
> org.datanucleus.store.rdbms.ConnectionFactoryImpl.initialiseDataSources(ConnectionFactoryImpl.java:110)
>         at
> org.datanucleus.store.rdbms.ConnectionFactoryImpl.<init>(ConnectionFactoryImpl.java:82)
>         ... 119 more
>
>  ​

Mime
View raw message