spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Wendell <pwend...@gmail.com>
Subject Re: UnusedStubClass in 1.3.0-rc1
Date Wed, 25 Feb 2015 17:42:02 GMT
This has been around for multiple versions of Spark, so I am a bit
surprised to see it not working in your build.

- Patrick

On Wed, Feb 25, 2015 at 9:41 AM, Patrick Wendell <pwendell@gmail.com> wrote:
> Hey Cody,
>
> What build command are you using? In any case, we can actually comment
> out the "unused" thing now in the root pom.xml. It existed just to
> ensure that at least one dependency was listed in the shade plugin
> configuration (otherwise, some work we do that requires the shade
> plugin does not happen). However, now there are other things there. If
> you just comment out the line in the root pom.xml adding this
> dependency, does it work?
>
> - Patrick
>
> On Wed, Feb 25, 2015 at 7:53 AM, Cody Koeninger <cody@koeninger.org> wrote:
>> So when building 1.3.0-rc1 I see the following warning:
>>
>> [WARNING] spark-streaming-kafka_2.10-1.3.0.jar, unused-1.0.0.jar define 1
>> overlappping classes:
>>
>> [WARNING]   - org.apache.spark.unused.UnusedStubClass
>>
>>
>> and when trying to build an assembly of a project that was previously using
>> 1.3 snapshots without difficulty, I see the following errors:
>>
>>
>> [error] (*:assembly) deduplicate: different file contents found in the
>> following:
>>
>> [error]
>> /Users/cody/.m2/repository/org/apache/spark/spark-streaming-kafka_2.10/1.3.0/spark-streaming-kafka_2.10-1.3.0.jar:org/apache/spark/unused/UnusedStubClass.class
>>
>> [error]
>> /Users/cody/.m2/repository/org/spark-project/spark/unused/1.0.0/unused-1.0.0.jar:org/apache/spark/unused/UnusedStubClass.class
>>
>>
>> This persists even after a clean / rebuild of both 1.3.0-rc1 and the
>> project using it.
>>
>>
>> I can just exclude that jar in the assembly definition, but is anyone else
>> seeing similar issues?  If so, might be worth resolving rather than make
>> users mess with assembly exclusions.
>>
>> I see that this class was introduced a while ago, related to SPARK-3812 but
>> the jira issue doesn't have much detail.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org


Mime
View raw message