beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Baptiste Onofré (JIRA) <j...@apache.org>
Subject [jira] [Created] (BEAM-349) Spark runner should provide a default BoM
Date Thu, 16 Jun 2016 12:26:05 GMT
Jean-Baptiste Onofré created BEAM-349:
-----------------------------------------

             Summary: Spark runner should provide a default BoM
                 Key: BEAM-349
                 URL: https://issues.apache.org/jira/browse/BEAM-349
             Project: Beam
          Issue Type: Improvement
          Components: runner-spark
            Reporter: Jean-Baptiste Onofré
            Assignee: Jean-Baptiste Onofré
             Fix For: 0.2.0-incubating


When an user defines a pipeline, he has to define the following dependencies in order to work
with the Spark runner (in addition of the Spark runner artifact itself):

{code}
  <dependency>
                    <groupId>org.apache.spark</groupId>
                    <artifactId>spark-core_2.10</artifactId>
                    <version>${spark.version}</version>
                </dependency>
                <dependency>
                    <groupId>org.apache.spark</groupId>
                    <artifactId>spark-streaming_2.10</artifactId>
                    <version>${spark.version}</version>
                </dependency>
                <dependency>
                    <groupId>com.fasterxml.jackson.core</groupId>
                    <artifactId>jackson-core</artifactId>
                    <version>${jackson.version}</version>
                </dependency>
                <dependency>
                    <groupId>com.fasterxml.jackson.core</groupId>
                    <artifactId>jackson-annotations</artifactId>
                    <version>${jackson.version}</version>
                </dependency>
                <dependency>
                    <groupId>com.fasterxml.jackson.core</groupId>
                    <artifactId>jackson-databind</artifactId>
                    <version>${jackson.version}</version>
                </dependency>
                <dependency>
                    <groupId>com.fasterxml.jackson.module</groupId>
                    <artifactId>jackson-module-scala_2.10</artifactId>
                    <version>${jackson.version}</version>
                </dependency>
{code}

These dependencies should come as transitive dependencies from the Spark runner (as Flink
or dataflow runners do). It means we should change the scope of the dependencies in the Spark
runner pom to create a BoM (Bill of Material).

I will experiment a bit (with [~amitsela]).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message