spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jun <>
Subject Re:spark vs flink low memory available
Date Tue, 11 Aug 2015 07:49:39 GMT
your detail of log file?

At 2015-08-10 22:02:16, "Pa Rö" <> wrote:

hi community,

i have build a spark and flink k-means application.

my test case is a clustering on 1 million points on 3node cluster.

inmemorybottlenecksbegins flink tooutsourceto disk andworkslowly butworks.

however spark lose executers if thememory is fullandstarts again (infinety loop?).

i try to customize the memory setting with the help from the mailing list here, thanks.

but spark not work.

is it necessary tohave anyconfigurationsto be set? i mean flink work with low memory, spark
must also be able to or not?

best regards,

View raw message