spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anton Kirillov <akirillov.mail...@gmail.com>
Subject Re: Spark 2.4.0 tests fail with hadoop-3.1 profile: NoClassDefFoundError org.apache.hadoop.hive.conf.HiveConf
Date Fri, 05 Apr 2019 17:54:39 GMT
Really sorry for the formatting. Here's the original message:

Hi there! I'm trying to run Spark unit tests with the following profiles:

./build/mvn test -Pmesos "-Phadoop-3.1" -Pnetlib-lgpl -Psparkr -Phive
-Phive-thriftserver

And 'core' module fails with the following test failing with
NoClassDefFoundError:

HadoopDelegationTokenManagerSuite:
- Correctly load default credential providers
- disable hive credential provider
- using deprecated configurations
- verify no credentials are obtained
*** RUN ABORTED ***
  java.lang.NoClassDefFoundError: Could not initialize class
org.apache.hadoop.hive.conf.HiveConf
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:250)
  at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:173)
  at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  at org.apache.spark.deploy.security.HiveDelegationTokenProvider$$anonfun$obtainDelegationTokens$2.apply$mcV$sp(HiveDelegationTokenProvider.scala:114)
  at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1340)
  at org.apache.spark.deploy.security.HiveDelegationTokenProvider.obtainDelegationTokens(HiveDelegationTokenProvider.scala:113)
  at org.apache.spark.deploy.security.HadoopDelegationTokenManagerSuite$$anonfun$5.apply(HadoopDelegationTokenManagerSuite.scala:98)
  at org.apache.spark.deploy.security.HadoopDelegationTokenManagerSuite$$anonfun$5.apply(HadoopDelegationTokenManagerSuite.scala:90)
  at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)

In the meantime building a distribution works fine when running:

./dev/make-distribution.sh --tgz -Pmesos "-Phadoop-3.1" -Pnetlib-lgpl
-Psparkr -Phive -Phive-thriftserver -DskipTests

Also, there are no problems with running tests using Hadoop 2.7 profile.
Does this issue look familiar? Any help appreciated!

On Fri, Apr 5, 2019 at 10:53 AM akirillov <akirillov.mailbox@gmail.com>
wrote:

> Hi there! I'm trying to run Spark unit tests with the following profiles:
>
> And 'core' module fails with the following test failing with
> NoClassDefFoundError:
>
> In the meantime building a distribution works fine when running:
>
> Also, there are no problems with running tests using Hadoop 2.7 profile.
> Does this issue look familiar? Any help appreciated!
>
>
>
> --
> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>

Mime
View raw message