flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dyana Rose (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-8267) Kinesis Producer example setting Region key
Date Fri, 15 Dec 2017 10:22:00 GMT
Dyana Rose created FLINK-8267:

             Summary: Kinesis Producer example setting Region key
                 Key: FLINK-8267
                 URL: https://issues.apache.org/jira/browse/FLINK-8267
             Project: Flink
          Issue Type: Bug
            Reporter: Dyana Rose
            Priority: Minor

In the example code for the kinesis producer the region key is set like:

producerConfig.put(AWSConfigConstants.AWS_REGION, "us-east-1");

However, the AWS Kinesis Producer Library requires that the region key be Region (https://github.com/awslabs/amazon-kinesis-producer/blob/94789ff4bb2f5dfa05aafe2d8437d3889293f264/java/amazon-kinesis-producer-sample/default_config.properties#L269)
so the setting at this point should be:

producerConfig.put(AWSConfigConstants.AWS_REGION, "us-east-1");
producerConfig.put("Region", "us-east-1");

When you run the Kinesis Producer you can see the effect of not setting the Region key by
a log line

org.apache.flink.streaming.connectors.kinesis.FlinkKinesisProducer  - Started Kinesis producer
instance for region ''

The KPL also then assumes it's running on EC2 and attempts to determine it's own region, which

(EC2MetadataClient)Http request to Ec2MetadataService failed.
[error] [main.cc:266] Could not configure the region. It was not given in the config and we
were unable to retrieve it from EC2 metadata

At the least I'd say the documentation should mention the difference between these two keys
and when you are required to also set the Region key.

On the other hand, is this even the intended behaviour of this connector? Was it intended
that the AWSConfigConstants.AWS_REGION key also set the region of the of the kinesis stream?
The documentation for the example states 

The example demonstrates producing a single Kinesis stream in the AWS region “us-east-1”.

This message was sent by Atlassian JIRA

View raw message