spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jianshi Huang <jianshi.hu...@gmail.com>
Subject Re: Unable to find org.apache.spark.sql.catalyst.ScalaReflection class
Date Thu, 12 Mar 2015 18:23:41 GMT
Same issue here. But the classloader in my exception is somehow different.

scala.ScalaReflectionException: class
org.apache.spark.sql.catalyst.ScalaReflection in JavaMirror with
java.net.URLClassLoader@53298398 of type class java.net.URLClassLoader with
classpath


Jianshi

On Sun, Mar 1, 2015 at 9:32 AM, Michael Armbrust <michael@databricks.com>
wrote:

> I think its possible that the problem is that the scala compiler is not
> being loaded by the primordial classloader (but instead by some child
> classloader) and thus the scala reflection mirror is failing to initialize
> when it can't find it. Unfortunately, the only solution that I know of is
> to load all required jars when the JVM starts.
>
> On Sat, Feb 28, 2015 at 5:26 PM, Ashish Nigam <ashnigamtech@gmail.com>
> wrote:
>
>> Also, can scala version play any role here?
>> I am using scala 2.11.5 but all spark packages have dependency to scala
>> 2.11.2
>> Just wanted to make sure that scala version is not an issue here.
>>
>> On Sat, Feb 28, 2015 at 9:18 AM, Ashish Nigam <ashnigamtech@gmail.com>
>> wrote:
>>
>>> Hi,
>>> I wrote a very simple program in scala to convert an existing RDD to
>>> SchemaRDD.
>>> But createSchemaRDD function is throwing exception
>>>
>>> Exception in thread "main" scala.ScalaReflectionException: class
>>> org.apache.spark.sql.catalyst.ScalaReflection in JavaMirror with primordial
>>> classloader with boot classpath [.....] not found
>>>
>>>
>>> Here's more info on the versions I am using -
>>>
>>> <scala.binary.version>2.11</scala.binary.version>
>>>     <spark.version>1.2.1</spark.version>
>>>     <scala.version>2.11.5</scala.version>
>>>
>>> Please let me know how can I resolve this problem.
>>>
>>> Thanks
>>> Ashish
>>>
>>
>>
>


-- 
Jianshi Huang

LinkedIn: jianshi
Twitter: @jshuang
Github & Blog: http://huangjs.github.com/

Mime
View raw message