spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dongjoon Hyun (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-26299) Shaded test jar in spark-streaming cause deploy test jar twice
Date Sat, 08 Dec 2018 20:20:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-26299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Dongjoon Hyun updated SPARK-26299:
----------------------------------
    Summary: Shaded test jar in spark-streaming cause deploy test jar twice  (was: [MAVEN]
Shaded test jar in spark-streaming cause deploy test jar twice)

> Shaded test jar in spark-streaming cause deploy test jar twice
> --------------------------------------------------------------
>
>                 Key: SPARK-26299
>                 URL: https://issues.apache.org/jira/browse/SPARK-26299
>             Project: Spark
>          Issue Type: Bug
>          Components: Deploy
>    Affects Versions: 2.3.2, 2.4.0
>            Reporter: Liu, Linhong
>            Priority: Major
>
> In spark root pom, there is a test-jar execution for all sub-projects including spark-streaming.
This will attach an artifact: org.apache.spark:spark-streaming_2.11:*{color:#ff0000}test-jar{color}*:tests:2.3.2
> Also, in streaming pom, there is a shade test configuration, it will attach an artifact: org.apache.spark:spark-streaming_2.11:{color:#ff0000}*jar*{color}:tests:2.3.2
> But two artifacts actually point to the same file: spark-streaming_2.11-2.3.2-tests.jar
> So, when deploy spark to nexus using mvn deploy, maven will upload the test.jar twice
since it belongs to 2 artifacts. Then deploy fails due to nexus don't allow overrides existing
file for non-SNAPSHOT release.
> What's more, after checking the spark-streaming, shaded test jar is exactly same to
original test jar. It seems we can just delete the related shade config.
> {code:java}
> <build>
>   <outputDirectory>target/scala-${scala.binary.version}/classes</outputDirectory>
>   <testOutputDirectory>target/scala-${scala.binary.version}/test-classes</testOutputDirectory>
>   <plugins>
>     <plugin>
>       <groupId>org.apache.maven.plugins</groupId>
>       <artifactId>maven-shade-plugin</artifactId>
>       <configuration>
>         <shadeTestJar>true</shadeTestJar>
>       </configuration>
>     </plugin>
>   </plugins>
> </build>{code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message