samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Riccomini <criccom...@apache.org>
Subject Re: Truncating rocks db
Date Sun, 15 Feb 2015 23:43:26 GMT
Hey Benjamin,

You're right. Currently you have to call .all(), and delete everything.

RocksDB just committed TTL support for their Java library. This feature
allows data to automatically be expired out. Once RocksDB releases their
TTL patch (I believe in a few weeks, according to Igor), we'll update Samza
0.9.0. Our tracker patch is here:

  https://issues.apache.org/jira/browse/SAMZA-537

> Is there no way to just say I don't care about the old data, gimme a new
store?

We don't have this feature right now, but we could add it. This feature is
a bit more complicated when a changelog is attached, since we will have to
execute deletes for every key (we still need to call .all()). Are you
running with a changelog?

Cheers,
Chris

On Sun, Feb 15, 2015 at 10:41 AM, Benjamin Edwards <edwards.benj@gmail.com>
wrote:

> Hi,
>
> I am trialling samza for some windowed stream processing. Typically I want
> to aggregate a bunch of state over some window of messages, process the
> data, then drop the current state. The only way that I can see to do that
> at the moment is to delete every key. This seems expensive. Is there no way
> to just say I don't care about the old data, gimme a new store?
>
> Ben
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message