spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Wendell <pwend...@gmail.com>
Subject Re: UnusedStubClass in 1.3.0-rc1
Date Wed, 25 Feb 2015 17:41:29 GMT
Hey Cody,

What build command are you using? In any case, we can actually comment
out the "unused" thing now in the root pom.xml. It existed just to
ensure that at least one dependency was listed in the shade plugin
configuration (otherwise, some work we do that requires the shade
plugin does not happen). However, now there are other things there. If
you just comment out the line in the root pom.xml adding this
dependency, does it work?

- Patrick

On Wed, Feb 25, 2015 at 7:53 AM, Cody Koeninger <cody@koeninger.org> wrote:
> So when building 1.3.0-rc1 I see the following warning:
>
> [WARNING] spark-streaming-kafka_2.10-1.3.0.jar, unused-1.0.0.jar define 1
> overlappping classes:
>
> [WARNING]   - org.apache.spark.unused.UnusedStubClass
>
>
> and when trying to build an assembly of a project that was previously using
> 1.3 snapshots without difficulty, I see the following errors:
>
>
> [error] (*:assembly) deduplicate: different file contents found in the
> following:
>
> [error]
> /Users/cody/.m2/repository/org/apache/spark/spark-streaming-kafka_2.10/1.3.0/spark-streaming-kafka_2.10-1.3.0.jar:org/apache/spark/unused/UnusedStubClass.class
>
> [error]
> /Users/cody/.m2/repository/org/spark-project/spark/unused/1.0.0/unused-1.0.0.jar:org/apache/spark/unused/UnusedStubClass.class
>
>
> This persists even after a clean / rebuild of both 1.3.0-rc1 and the
> project using it.
>
>
> I can just exclude that jar in the assembly definition, but is anyone else
> seeing similar issues?  If so, might be worth resolving rather than make
> users mess with assembly exclusions.
>
> I see that this class was introduced a while ago, related to SPARK-3812 but
> the jira issue doesn't have much detail.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org


Mime
View raw message