spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jun <kit...@126.com>
Subject Re:spark vs flink low memory available
Date Tue, 11 Aug 2015 07:49:39 GMT
your detail of log file?


At 2015-08-10 22:02:16, "Pa Rö" <paul.roewer1990@googlemail.com> wrote:

hi community,


i have build a spark and flink k-means application.

my test case is a clustering on 1 million points on 3node cluster.

inmemorybottlenecksbegins flink tooutsourceto disk andworkslowly butworks.

however spark lose executers if thememory is fullandstarts again (infinety loop?).


i try to customize the memory setting with the help from the mailing list here, thanks.

but spark not work.

is it necessary tohave anyconfigurationsto be set? i mean flink work with low memory, spark
must also be able to or not?


best regards,

paul
Mime
View raw message