nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Burgess <>
Subject Re: New Distributed Map Cache Impementations
Date Thu, 14 Nov 2019 14:16:44 GMT

There are also Redis and Couchbase distributed map cache clients
already in NiFi. I don't see any Jiras or PRs related to a DynamoDB or
JDBC ones. I thought about making ones for JDBC, Hazelcast and/or
Nitrite [1] (with or without a DataGate server), but never got around
to it. I think DynamoDB and JDBC implementations would be helpful, the
latter could support DynamoDB in the meantime using the Simba JDBC
driver [2].

While thinking about the JDBC one, I figured it might be nice to be
able to cache the table locally for X amount of time or N number of
entries, in case you pre-populate the cache and are just reading it
with the client. Any write (from the NiFi client) would invalidate the
cache and the table would be re-fetched on the next read operation. I
did something similar for the DatabaseRecordLookupService, but that's
a read-only service so I didn't have to worry about writes, I was just
trying to improve performance where possible.



On Thu, Nov 14, 2019 at 8:31 AM Shawn Weeks <> wrote:
> Has anyone already done some work on adding new services for distribute map cache? I’m
looking at moving to aws and I really don’t want to have to run emr just for hbase. I’ve
been thinking about starting on either a DynamoDB or simple jdbc implementation.
> Thanks
> Shawn
> Sent from my iPhone

View raw message