flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From fhueske <...@git.apache.org>
Subject [GitHub] flink pull request: [FLINK-2972] [JavaAPI] Remove Chill dependency...
Date Wed, 04 Nov 2015 21:07:21 GMT
GitHub user fhueske opened a pull request:


    [FLINK-2972] [JavaAPI] Remove Chill dependency from flink-java.

    This PR replaces `flink-java`'s `chill` dependency by `chill-java` to decouple `flink-java`
from Scala.
    Chill is used within the `KryoSerializer` to obtain a "Scala-preconfigured" Kryo instance.
Hence, Chill is still required for Scala DataSet or DataStream programs. Separate KryoSerializers
for Scala and Java are hard to provide, because `GenericTypeInformation` can be generated
through the `TypeExtractor`, `TypeParser`, and the Scala compiler marko. 
    In this PR, the `KryoSerializer` tries to dynamically load Chill's `ScalaKryoInstantiator`
and generate a Kryo instance through reflection to circumvent the Chill dependency. As a fallback,
a regular Kryo instance is created. Consequently, Chill's Kryo instance is used whenever Chill
is present in the classpath. To ensure that client, JM, and TM have Chill in their classpath,
we add Chill as a dependency to `flink-client`.
    **Note**: Before merging, we need to check if this works for Java and Scala programs which
are submitted to a cluster (including some collection data sources which are serialized at
the client).

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/fhueske/flink scalaFlinkJava

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1331
commit 42c44bccdbda2e6794cd1d5b8bf851860c9643ee
Author: Fabian Hueske <fhueske@apache.org>
Date:   2015-11-03T15:57:49Z

    [FLINK-2972] [JavaAPI] Remove Chill dependency from flink-java.


If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.

View raw message