flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Hogan <c...@greghogan.com>
Subject Re: Looping over a DataSet and accesing another DataSet
Date Mon, 31 Oct 2016 12:28:22 GMT
The DataSet API only supports binary joins but one can simulate an n-ary
join by chaining successive join operations.

Your algorithm requires a global ordering on edges, requiring a parallelism
of 1, and will not scale in a distributed processing system. Flink excels
at processing bulk (larger than memory) data in serial.


On Mon, Oct 31, 2016 at 5:54 AM, otherwise777 <wouter@onzichtbaar.net>

> Thank you for your reply and explanation, I think there is one issue with
> your method though, you said that i should make a join with the the key
> value pair A on v and  the Edge set (u,v), this would work, however i not
> only need to access A[v] in one iteration but also A[u], so if i join on v
> that won't be possible
> Did i understand it correctly?
> --
> View this message in context: http://apache-flink-user-
> mailing-list-archive.2336050.n4.nabble.com/Looping-over-a-
> DataSet-and-accesing-another-DataSet-tp9778p9782.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.

View raw message