spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anty Rao <ant....@gmail.com>
Subject Re: Not per-key state in spark streaming
Date Thu, 08 Dec 2016 02:49:38 GMT
On Wed, Dec 7, 2016 at 7:42 PM, Anty Rao <ant.rao@gmail.com> wrote:

> Hi
> I'm new to Spark. I'm doing some research to see if spark streaming can
> solve my problem. I don't want to keep per-key state,b/c my data set is
> very huge and keep a little longer time, it not viable to keep all per key
> state in memory.Instead, i want to have a bloom filter based state. Does it
> possible to achieve this in Spark streaming.
>
> Is it possible to achieve this by extending Spark API?

> --
> Anty Rao
>



-- 
Anty Rao

Mime
View raw message