flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Yao <g...@data-artisans.com>
Subject Re: State management and heap usage
Date Thu, 12 Apr 2018 08:45:38 GMT
Hi Michael,

You can configure the default state backend by setting state.backend in
flink-conf.yaml, or you can configure it per job [1]. The default state
is "jobmanager" (MemoryStateBackend), which stores state and checkpoints on
Java heap. RocksDB must be explicitly enabled, e.g., by setting
state.backend to



On Wed, Apr 11, 2018 at 11:04 PM, TechnoMage <mlatta@technomage.com> wrote:

> I am pretty new to flink and have an initial streaming job working both
> locally and remotely.  But, both ways if the data volume is too high it
> runs out of heap.  I am using RichMapFunction to process multiple streams
> of data.  I assumed Flink would manage keeping state in ram when possible,
> and spill to RocksDB when it exceeded heap.
> Is this correct?  If so are there configs I need to set to enable or tune
> this so it can run within a fixed memory size?
> Michael

View raw message