flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-3923) Unify configuration conventions of the Kinesis producer to the same as the consumer
Date Tue, 07 Jun 2016 05:07:20 GMT

    [ https://issues.apache.org/jira/browse/FLINK-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317860#comment-15317860
] 

Tzu-Li (Gordon) Tai commented on FLINK-3923:
--------------------------------------------

I'm wondering whether or not it might be a better idea to use dedicated configuration classes.
i.e.,
FlinkKinesisConsumerConfiguration and FlinkKinesisProducerConfiguration.
Both takes the required AWS connection info (region, credential) as constructor args, then
use cascading set methods for additional settings.

For example for the consumer,
.setInitialPosition()
.setDescribeStreamBackfireMillis()
.setWatermarkAssigner()
... (any other config we may add in the future)

The configuration classes will be responsible for setting the default values of this optional
settings (behaviour of reading default values when not set is kind of floppy right now).

What do you think?

> Unify configuration conventions of the Kinesis producer to the same as the consumer
> -----------------------------------------------------------------------------------
>
>                 Key: FLINK-3923
>                 URL: https://issues.apache.org/jira/browse/FLINK-3923
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Kinesis Connector, Streaming Connectors
>    Affects Versions: 1.1.0
>            Reporter: Robert Metzger
>            Assignee: Abdullah Ozturk
>             Fix For: 1.1.0
>
>
> Currently, the Kinesis consumer and producer are configured differently.
> The producer expects a list of arguments for the access key, secret, region, stream.
The consumer is accepting properties (similar to the Kafka connector).
> The objective of this issue is to change the producer so that it is also using a properties-based
configuration (including an input validation step)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message