flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gábor Gévay <gga...@gmail.com>
Subject Re: Deep Copy in FLINK, Kryo Copy is used in the different operator
Date Wed, 14 Feb 2018 16:07:00 GMT
Hello,

You might also be able to make Flink use a better serializer than
Kryo. Flink falls back to Kryo when it can't use its own serializers,
see here:
https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/types_serialization.html
For example, it might help to make your type a POJO.

Best,
Gábor




On Wed, Feb 14, 2018 at 3:38 PM, Aljoscha Krettek <aljoscha@apache.org> wrote:
> Hi,
>
> You can disable those copies via ExecutionConfig.enableObjectReuse(), which you can get
from the StreamExecutionEnvironment via getConfig().
>
> Best,
> Aljoscha
>
>> On 12. Feb 2018, at 04:00, chen <eric__715@126.com> wrote:
>>
>> Actually our team have our own Stream Engine, we tested our engine and flink,
>> find out when we aggregate the stream data, the throughput is decreasing
>> very fast.
>>
>> So we catch the stack and find out a deep copy in flink.
>>
>> In different operator, there will be
>> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.copy between
>> in different operator.
>>
>>
>>
>>
>> --
>> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>

Mime
View raw message