flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [flink] bowenli86 commented on issue #8205: [FLINK-12238] [hive] Support database related operations in GenericHiveMetastoreCatalog and setup flink-connector-hive module
Date Mon, 29 Apr 2019 17:15:36 GMT
bowenli86 commented on issue #8205: [FLINK-12238] [hive] Support database related operations
in GenericHiveMetastoreCatalog and setup flink-connector-hive module
URL: https://github.com/apache/flink/pull/8205#issuecomment-487665669
 
 
   > Stephan pointed you in a good direction, we will have to check out what Hive is bundling.
Looks like only hive-exec is bundling anything, with the following include patterns:
   > 
   > ```
   > <include>org.apache.hive:hive-common</include>
   >                   <include>org.apache.hive:hive-exec</include>
   >                   <include>org.apache.hive:hive-serde</include>
   >                   <include>org.apache.hive:hive-llap-common</include>
   >                   <include>org.apache.hive:hive-llap-client</include>
   >                   <include>org.apache.hive:hive-metastore</include>
   >                   <include>org.apache.hive:hive-service-rpc</include>
   >                   <include>com.esotericsoftware:kryo-shaded</include>
   > 		  <include>com.esotericsoftware:minlog</include>
   > 		  <include>org.objenesis:objenesis</include>
   >                   <include>org.apache.parquet:parquet-hadoop-bundle</include>
   >                   <include>org.apache.thrift:libthrift</include>
   >                   <include>org.apache.thrift:libfb303</include>
   >                   <include>javax.jdo:jdo-api</include>
   >                   <include>commons-lang:commons-lang</include>
   >                   <include>org.apache.commons:commons-lang3</include>
   >                   <include>org.jodd:jodd-core</include>
   >                   <include>com.tdunning:json</include>
   >                   <include>org.apache.avro:avro</include>
   >                   <include>org.apache.avro:avro-mapred</include>
   >                   <include>org.apache.hive.shims:hive-shims-0.23</include>
   >                   <include>org.apache.hive.shims:hive-shims-0.23</include>
   >                   <include>org.apache.hive.shims:hive-shims-common</include>
   >                   <include>com.googlecode.javaewah:JavaEWAH</include>
   >                   <include>javolution:javolution</include>
   >                   <include>com.google.protobuf:protobuf-java</include>
   >                   <include>io.airlift:aircompressor</include>
   >                   <include>org.codehaus.jackson:jackson-core-asl</include>
   >                   <include>org.codehaus.jackson:jackson-mapper-asl</include>
   >                   <include>com.google.guava:guava</include>
   >                   <include>net.sf.opencsv:opencsv</include>
   >                   <include>org.apache.hive:spark-client</include>
   >                   <include>org.apache.hive:hive-storage-api</include>
   >                   <include>org.apache.orc:orc-core</include>
   >                   <include>org.apache.orc:orc-tools</include>
   >                   <include>joda-time:joda-time</include>
   > ```
   > 
   > At a quick glance I haven't seen anything problematic; nevertheless we'll have to
handle them like non-transitive dependencies and check their individual licenses, list them
and bundle the licenses.
   
   I've changed hive-exec to be a test dependency now to not block us for so long since our
work on hive table source/sink depends on this PR. I thought it is a not-so-hard problem to
solve and include hive-exec when setting up flink-connector-hive but seems I was wrong. So
let's leave it as test dependency now and change it to compile dependency whenever necessary
later. What do you think?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message