spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: spark vs flink low memory available
Date Tue, 11 Aug 2015 08:03:57 GMT
Pa:
Can you try 1.5.0 SNAPSHOT ?

See SPARK-7075 Project Tungsten (Spark 1.5 Phase 1)

Cheers

On Tue, Aug 11, 2015 at 12:49 AM, jun <kitaev@126.com> wrote:

> your detail of log file?
>
> At 2015-08-10 22:02:16, "Pa Rö" <paul.roewer1990@googlemail.com> wrote:
>
> hi community,
>
> i have build a spark and flink k-means application.
> my test case is a clustering on 1 million points on 3node cluster.
>
> in memory bottlenecks begins flink to outsource to disk and work slowly
> but works.
> however spark lose executers if the memory is full and starts again
> (infinety loop?).
>
> i try to customize the memory setting with the help from the mailing list
> here
> <http://mail-archives.us.apache.org/mod_mbox/spark-user/201508.mbox/%3CCAH2_pyKQhFR4tBvPbt2TDHGm+ZRkCBZfNK7ueDkJPdhe472fVQ@mail.gmail.com%3E>,
> thanks.
> but spark not work.
>
> is it necessary to have any configurations to be set? i mean flink work
> with low memory, spark must also be able to or not?
>
> best regards,
> paul
>
>
>
>

Mime
View raw message