flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: Problem: Please check that all IDs specified via `uid(String)` are unique.
Date Wed, 03 May 2017 18:59:18 GMT
Is the method called multiple times with different input sets?

Does the issue occur regardless of which uid you set?

On 03.05.2017 20:21, Rami Al-Isawi wrote:
> Hi,
>
> Nope, that is the only uid in the whole project. The 
> “userRawDataStream" comes from a splitter:
> public static class Splitterimplements OutputSelector<EventRaw>{
>      @Override public Iterable<String> select(EventRaw value) {
>          List<String> output =new ArrayList<>();
>          output.add(value.type);
>          return output;
>      }
> }
> Any ideas how to get better debug messages?
> Regards,
> -Rami
>
>> On 3 May 2017, at 14:12, Chesnay Schepler <chesnay@apache.org 
>> <mailto:chesnay@apache.org>> wrote:
>>
>> Hello,
>>
>> was a uid set on "userRawDataStream", or any of it's parent 
>> transformations?
>>
>> On 03.05.2017 12:59, Rami Al-Isawi wrote:
>>> Hi,
>>>
>>> I am trying to set uids. I keep getting this (Flink.1.2):
>>>
>>> Exception in thread "main" java.lang.IllegalArgumentException: Hash 
>>> collision on user-specified ID. Most likely cause is a non-unique 
>>> ID. Please check that all IDs specified via `uid(String)` are unique.
>>>
>>> Here is the code snippet.
>>> public DataStream<UserAction> processUserActions(DataStream<UserRaw>
userRawDataStream, Time duration, Sonar.ReportSpan span) {
>>>      return userRawDataStream.map(userRaw->new UserActionValue(userRaw)).startNewChain().uid("111111")
>>>               .keyBy("userRaw.eventRaw.env_key","userRaw.eventRaw.tag","userRaw.eventRaw."+span.name(),"userRaw.type")
>>>              .window(GlobalWindows.create())
>>>              .trigger(new TimedTrigger(duration))
>>>              .sum("count")//.startNewChain().uid("22222222") .map(userActionsCount->new
UserAction(userActionsCount,span));//.startNewChain().uid("33333333"); }
>>> I made sure that they are unique and even with one uid, the error is there. I
guess there is an easy fix, but I cannot see it.
>>> Regards,
>>> -Rami
>>> Disclaimer: This message and any attachments thereto are intended 
>>> solely for the addressed recipient(s) and may contain confidential 
>>> information. If you are not the intended recipient, please notify 
>>> the sender by reply e-mail and delete the e-mail (including any 
>>> attachments thereto) without producing, distributing or retaining 
>>> any copies thereof. Any review, dissemination or other use of, or 
>>> taking of any action in reliance upon, this information by persons 
>>> or entities other than the intended recipient(s) is prohibited. 
>>> Thank you. 
>>
>>
>
> Disclaimer: This message and any attachments thereto are intended 
> solely for the addressed recipient(s) and may contain confidential 
> information. If you are not the intended recipient, please notify the 
> sender by reply e-mail and delete the e-mail (including any 
> attachments thereto) without producing, distributing or retaining any 
> copies thereof. Any review, dissemination or other use of, or taking 
> of any action in reliance upon, this information by persons or 
> entities other than the intended recipient(s) is prohibited. Thank you. 



Mime
View raw message