storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nikos R. Katsipoulakis" <nick.kat...@gmail.com>
Subject Re: Storm transaction boundary
Date Thu, 14 Apr 2016 16:00:46 GMT
Hello Sreekumar,

Have you thought of using Storm's distributed cache? If not, that might a
way to cache messages before you push them to the target DB. Another way to
do so, is if you can create your own Bolt to periodically push messages in
the database.

I hope I helped.

Cheers,
Nikos

On Thu, Apr 14, 2016 at 12:54 AM, pradeep s <sreekumar.pradeep@gmail.com>
wrote:

> Hi,
> We are using Storm for processing CDC messages from Oracle Golden Gate .
> Pipeline is as below
> Oracle GoldenGate-->Queue-->Storm-->Relational DB
>  We have a requirement to hold the messages for a transaction Id till all
> the messages for that transaction is available in Storm. There can be
> scenarios like 1 million updates happening in onme transaction source
> oracle system.
> Can you please suggest a best approach for holding the messages and then
> pushing to target db only when all messages for tran id is available in
> storm.
>
> Regards
> Pradeep S
>



-- 
Nikos R. Katsipoulakis,
Department of Computer Science
University of Pittsburgh

Mime
View raw message