spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Wendell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-1403) Spark on Mesos does not set Thread's context class loader
Date Mon, 07 Apr 2014 17:26:14 GMT

    [ https://issues.apache.org/jira/browse/SPARK-1403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13962032#comment-13962032
] 

Patrick Wendell commented on SPARK-1403:
----------------------------------------

The underlying issue here is that we've made assumptions in various parts of the codebase
that the context classloader is set on a thread. In general, we should relax these assumptions
and just fallback to the classloader that loaded Spark. As a workaround this patch:

https://github.com/apache/spark/pull/322/files

just manually sets the classloader to the system class loader.

> Spark on Mesos does not set Thread's context class loader
> ---------------------------------------------------------
>
>                 Key: SPARK-1403
>                 URL: https://issues.apache.org/jira/browse/SPARK-1403
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 1.0.0
>         Environment: ubuntu 12.04 on vagrant
>            Reporter: Bharath Bhushan
>            Priority: Blocker
>
> I can run spark 0.9.0 on mesos but not spark 1.0.0. This is because the spark executor
on mesos slave throws a  java.lang.ClassNotFoundException for org.apache.spark.serializer.JavaSerializer.
> The lengthy discussion is here: http://apache-spark-user-list.1001560.n3.nabble.com/java-lang-ClassNotFoundException-spark-on-mesos-td3510.html#a3513



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message