flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ufuk Celebi <...@apache.org>
Subject Re: HBase Input Format for streaming
Date Mon, 06 Jun 2016 14:07:09 GMT
>From the code it looks like the open method of the TableInputFormat is
never called. What are you doing differently in the
StreamingTableInputFormat?

– Ufuk


On Mon, Jun 6, 2016 at 1:49 PM, Christophe Salperwyck
<christophe.salperwyck@gmail.com> wrote:
> Hi all,
>
> I am trying to read data from HBase and use the windows functions of Flink
> streaming. I can read my data using the ExecutionEnvironment but not from
> the StreamExecutionEnvironment.
>
> Is that a known issue?
>
> Are the inputsplits used in the streaming environment?
>
> Here a sample of my code:
>
> final StreamExecutionEnvironment env =
> StreamExecutionEnvironment.getExecutionEnvironment();
> env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
>
> @SuppressWarnings("serial")
> final DataStreamSource<ANA> anaDS = env.createInput(new
> TableInputFormat<ANA>() {
> ...
> }
>
> final WindowedStream<ANA, Tuple, TimeWindow> ws = anaDS.
> assignTimestampsAndWatermarks(new xxxxAssignerWithPunctuatedWatermarks()).
> keyBy(0).
> timeWindow(Time.days(30), Time.days(30));
>
> ws.sum(2).printToErr();
> env.execute();
>
> The error I get is:
> Caused by: java.io.IOException: No table result scanner provided!
> at
> org.apache.flink.addons.hbase.TableInputFormat.nextRecord(TableInputFormat.java:103)
>
> It seems the "Result" is not read for a first time before calling this
> function.
>
> I built a "StreamingTableInputFormat" as a temporary work around but let me
> know if there is something I did wrong.
>
> Thanks for everything, Flink is great!
>
> Cheers,
> Christophe

Mime
View raw message