spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Qiuzhuang Lian <qiuzhuang.l...@gmail.com>
Subject Re: Run ScalaTest inside Intellij IDEA
Date Wed, 11 Jun 2014 14:16:49 GMT
I run into this issue too today via 'mvn install -DskipTests' command
today, then I issue a mvn clean and rebuild and it works.

Thanks,
Qiuzhuang


On Wed, Jun 11, 2014 at 9:51 PM, Yijie Shen <henry.yijieshen@gmail.com>
wrote:

> Thx Qiuzhuang, the problems disappeared after I add assembly jar at the
> head of list dependencies in *.iml, but while running test in Spark
> SQL(SQLQuerySuite in sql-core), another two error occurs:
>
> Error 1:
> Error:scalac:
>      while compiling:
> /Users/yijie/code/apache.spark.master/sql/core/src/main/scala/org/apache/spark/sql/test/TestSQLContext.scala
>         during phase: jvm
>      library version: version 2.10.4
>     compiler version: version 2.10.4
>   reconstructed args: -Xmax-classfile-name 120 -deprecation
> -P:genjavadoc:out=/Users/yijie/code/apache.spark.master/sql/core/target/java
> -feature -classpath
> /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/lib/javafx-doclet.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/lib/tools.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Conte…
> …
> ...
> /Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre/classes:/Users/yijie/code/apache.spark.master/lib_managed/jars/scala-library-2.10.4.jar
> -Xplugin:/Users/yijie/code/apache.spark.master/lib_managed/jars/genjavadoc-plugin_2.10.4-0.5.jar
> -Xplugin:/Users/yijie/code/apache.spark.master/lib_managed/jars/genjavadoc-plugin_2.10.4-0.5.jar
>   last tree to typer: Literal(Constant(parquet.io.api.Converter))
>               symbol: null
>    symbol definition: null
>                  tpe: Class(classOf[parquet.io.api.Converter])
>        symbol owners:
>       context owners: object TestSQLContext -> package test
> == Enclosing template or block ==
> Template( // val <local TestSQLContext>: <notype> in object
> TestSQLContext, tree.tpe=org.apache.spark.sql.test.TestSQLContext.type
>   "org.apache.spark.sql.SQLContext" // parents
>   ValDef(
>     private
>     "_"
>     <tpt>
>     <empty>
>   )
>   // 2 statements
>   DefDef( // private def readResolve(): Object in object TestSQLContext
>     <method> private <synthetic>
>     "readResolve"
>     []
>     List(Nil)
>     <tpt> // tree.tpe=Object
>     test.this."TestSQLContext" // object TestSQLContext in package test,
> tree.tpe=org.apache.spark.sql.test.TestSQLContext.type
>   )
>   DefDef( // def <init>(): org.apache.spark.sql.test.TestSQLContext.type
> in object TestSQLContext
>     <method>
>     "<init>"
>     []
>     List(Nil)
>     <tpt> // tree.tpe=org.apache.spark.sql.test.TestSQLContext.type
>     Block( // tree.tpe=Unit
>       Apply( // def <init>(sparkContext: org.apache.spark.SparkContext):
> org.apache.spark.sql.SQLContext in class SQLContext,
> tree.tpe=org.apache.spark.sql.SQLContext
>         TestSQLContext.super."<init>" // def <init>(sparkContext:
> org.apache.spark.SparkContext): org.apache.spark.sql.SQLContext in class
> SQLContext, tree.tpe=(sparkContext:
> org.apache.spark.SparkContext)org.apache.spark.sql.SQLContext
>         Apply( // def <init>(master: String,appName: String,conf:
> org.apache.spark.SparkConf): org.apache.spark.SparkContext in class
> SparkContext, tree.tpe=org.apache.spark.SparkContext
>           new org.apache.spark.SparkContext."<init>" // def <init>(master:
> String,appName: String,conf: org.apache.spark.SparkConf):
> org.apache.spark.SparkContext in class SparkContext, tree.tpe=(master:
> String, appName: String, conf:
> org.apache.spark.SparkConf)org.apache.spark.SparkContext
>           // 3 arguments
>           "local"
>           "TestSQLContext"
>           Apply( // def <init>(): org.apache.spark.SparkConf in class
> SparkConf, tree.tpe=org.apache.spark.SparkConf
>             new org.apache.spark.SparkConf."<init>" // def <init>():
> org.apache.spark.SparkConf in class SparkConf,
> tree.tpe=()org.apache.spark.SparkConf
>             Nil
>           )
>         )
>       )
>       ()
>     )
>   )
> )
> == Expanded type of tree ==
> ConstantType(value = Constant(parquet.io.api.Converter))
> uncaught exception during compilation: java.lang.AssertionError
>
> Error 2:
>
> Error:scalac: Error: assertion failed: List(object package$DebugNode,
> object package$DebugNode)
> java.lang.AssertionError: assertion failed: List(object package$DebugNode,
> object package$DebugNode)
> at scala.reflect.internal.Symbols$Symbol.suchThat(Symbols.scala:1678)
> at
> scala.reflect.internal.Symbols$ClassSymbol.companionModule0(Symbols.scala:2988)
> at
> scala.reflect.internal.Symbols$ClassSymbol.companionModule(Symbols.scala:2991)
> at
> scala.tools.nsc.backend.jvm.GenASM$JPlainBuilder.genClass(GenASM.scala:1371)
> at scala.tools.nsc.backend.jvm.GenASM$AsmPhase.run(GenASM.scala:120)
> at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1583)
> at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1557)
> at scala.tools.nsc.Global$Run.compileSources(Global.scala:1553)
> at scala.tools.nsc.Global$Run.compile(Global.scala:1662)
> at xsbt.CachedCompiler0.run(CompilerInterface.scala:126)
> at xsbt.CachedCompiler0.run(CompilerInterface.scala:102)
> at xsbt.CompilerInterface.run(CompilerInterface.scala:27)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sbt.compiler.AnalyzingCompiler.call(AnalyzingCompiler.scala:102)
> at sbt.compiler.AnalyzingCompiler.compile(AnalyzingCompiler.scala:48)
> at sbt.compiler.AnalyzingCompiler.compile(AnalyzingCompiler.scala:41)
> at
> org.jetbrains.jps.incremental.scala.local.IdeaIncrementalCompiler.compile(IdeaIncrementalCompiler.scala:28)
> at
> org.jetbrains.jps.incremental.scala.local.LocalServer.compile(LocalServer.scala:25)
> at org.jetbrains.jps.incremental.scala.remote.Main$.make(Main.scala:64)
> at org.jetbrains.jps.incremental.scala.remote.Main$.nailMain(Main.scala:22)
> at org.jetbrains.jps.incremental.scala.remote.Main.nailMain(Main.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at com.martiansoftware.nailgun.NGSession.run(NGSession.java:319)
>
>
> On Jun 11, 2014, at 11:17 AM, Qiuzhuang Lian <qiuzhuang.lian@gmail.com>
> wrote:
>
> I also run into this problem when running examples in IDEA. The issue
> looks that it uses depends on too many jars and that the classpath seems to
> have length limit. So I import the assembly jar and put the head of the
> list dependent path and it works.
>
> Thanks,
> Qiuzhuang
>
>
> On Wed, Jun 11, 2014 at 10:39 AM, 申毅杰 <henry.yijieshen@gmail.com> wrote:
>
>> Hi All,
>>
>> I want to run ScalaTest Suite in IDEA directly, but it seems didn’t pass
>> the make phase before test running.
>> The problems are as follows:
>>
>>
>> /Users/yijie/code/apache.spark.master/core/src/main/scala/org/apache/spark/executor/MesosExecutorBackend.scala
>> Error:(44, 35) type mismatch;
>>  found   : org.apache.mesos.protobuf.ByteString
>>  required: com.google.protobuf.ByteString
>>       .setData(ByteString.copyFrom(data))
>>                                   ^
>>
>> /Users/yijie/code/apache.spark.master/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala
>> Error:(119, 35) type mismatch;
>>  found   : org.apache.mesos.protobuf.ByteString
>>  required: com.google.protobuf.ByteString
>>       .setData(ByteString.copyFrom(createExecArg()))
>>                                   ^
>> Error:(257, 35) type mismatch;
>>  found   : org.apache.mesos.protobuf.ByteString
>>  required: com.google.protobuf.ByteString
>>       .setData(ByteString.copyFrom(task.serializedTask))
>>                                   ^
>>
>> Before I run test in IDEA, I build spark through ’sbt/sbt assembly’,
>> import projects into IDEA after ’sbt/sbt gen-idea’,
>> and able to run test in Terminal ’sbt/sbt test’
>>
>> Are there anything I leave out in order to run/debug testsuite inside
>> IDEA?
>>
>> Best regards,
>> Yijie
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message