flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kailashhd <...@git.apache.org>
Subject [GitHub] flink pull request #5663: [FLINK-8888] [Kinesis Connectors] Update the AWS S...
Date Fri, 09 Mar 2018 02:27:38 GMT
GitHub user kailashhd opened a pull request:


    [FLINK-8888] [Kinesis Connectors] Update the AWS SDK for flink kinesis connector

    ## What is the purpose of the change
    Updating the AWS Java SDK in order to be able to use new features like ListShards as well
as bug fixes as mentioned in [FLINK-8554](https://issues.apache.org/jira/browse/FLINK-8554)
    ## Brief change log
      - Update the AWS Java SDK from 1.1.171 to 1.11.272. 
    ## Verifying this change
    This change is already covered by existing tests, such as FlinkKinesisConsumerMigrationTest.java.
Also checked KCL and  KPL for it's dependency version on AWS SDK. KCL 1.8.1 depends on 1.11.171
and KPL  
    depends on 0.12.6 depends on 1.11.128 and it is backward compatible. Also since we shade
the dependencies in kinesis connector, we will not have dependencies conflicts with AWS EMR
which depends on 1.11.267. Also ran the mvn clean verify.
    Tested manually creating a savepoint in 1.3.2 and restarting it in the latest snapshot.

    ## Does this pull request potentially affect one of the following parts:
      - Dependencies (does it add or upgrade a dependency): (**yes** / no)
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes
/ **no**)
      - The serializers: (yes / **no** / don't know)
      - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing,
Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
      - The S3 file system connector: (yes / **no** / don't know) - AWS Java SDK changed only
for Kinesis
    ## Documentation
      - Does this pull request introduce a new feature? (yes / **no**)
      - If yes, how is the feature documented? (not **applicable** / docs / JavaDocs / not

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/kailashhd/flink UpdateAWSSDK

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5663
commit 47d0e5523ca908bc5b11c21902ea1a5385914644
Author: Kailash HD <kdayanand@...>
Date:   2018-03-08T18:32:23Z

    [FLINK-8888] [Kinesis Connectors] Update the AWS SDK for flink kinesis connector



View raw message