flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Pompermaier <pomperma...@okkam.it>
Subject Re: Long ids from String
Date Tue, 03 Nov 2015 09:10:43 GMT
Hi Martin,
thanks for the suggestion but unfortunately in my use case I have another
problem: I have to join triplets when f2==f0..is there any way to translate
also references? i.e. if I have 2 tuples <a,b,c> <c,d,e>, when I apply that
function I obtain something like <1,<a,b,c>>,<2,<c,d,e>>.
I'd like to be able to join the 1st tuple with the 2nd (so I should know
that c=>2).
Which strategy do you think it could be the best option to achieve that?
At the moment I was thinking to persist the ids and use a temporary table
with an autoincrement long id but maybe there's a simpler solution..


On Tue, Nov 3, 2015 at 9:59 AM, Martin Junghanns <m.junghanns@mailbox.org>

> Hi Flavio,
> If you just want to assign a unique Long identifier to each element in
> your dataset, you can use the DataSetUtils.zipWithUniqueId() method [1].
> Best,
> Martin
> [1]
> https://github.com/apache/flink/blob/master/flink-java/src/main/java/org/apache/flink/api/java/utils/DataSetUtils.java#L131
> On 03.11.2015 09:42, Flavio Pompermaier wrote:
> > Hi to all,
> >
> > I was reading the thread about the Neo4j connector and an old question
> came
> > to my mind.
> >
> > In my Flink job I have Tuples with String ids that I use to join on that
> > I'd like to convert to long (because Flink should improve quite a lot the
> > memory usage and the processing time if I'm not wrong).
> > Is there any recommended way to do that conversion in Flink?
> >
> > Best,
> > Flavio
> >


Flavio Pompermaier

*Development Department*_______________________________________________
*OKKAM**Srl **- www.okkam.it <http://www.okkam.it/>*

*Phone:* +(39) 0461 283 702
*Fax:* + (39) 0461 186 6433
*Email:* pompermaier@okkam.it
*Headquarters:* Trento (Italy), via G.B. Trener 8
*Registered office:* Trento (Italy), via Segantini 23

Confidentially notice. This e-mail transmission may contain legally
privileged and/or confidential information. Please do not read it if you
are not the intended recipient(S). Any use, distribution, reproduction or
disclosure by any other person is strictly prohibited. If you have received
this e-mail in error, please notify the sender and destroy the original
transmission and its attachments without reading or saving it in any manner.

View raw message