flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Pompermaier <pomperma...@okkam.it>
Subject Retrieve written records of a sink after job
Date Wed, 14 Feb 2018 11:22:21 GMT
Hi to all,
I have a (batch) job that writes to 1 or more sinks.
Is there a way to retrieve, once the job has terminated, the number of
records written to each sink?
Is there any better way than than using an accumulator for each sink?
If that is the only way to do that, the Sink API could be enriched in order
to automatically create an accumulator when required. E.g.

dataset.output(JDBCOutputFormat.buildJDBCOutputFormat()
            .setDrivername(...)
            .setDBUrl(...)
            .setQuery(...)
            *.addRecordsCountAccumulator("some-name")*
            .finish())

Best,
Flavio

Mime
View raw message