samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom Dearman <>
Subject Samza commit guarantees
Date Thu, 19 Feb 2015 16:23:49 GMT
Can someone help with the following questions please:

 a) If using RockDB kv implementation, is there a way to guarantee that a put is committed
(at least on that instance disc), I notice that RockDB implementation does nothing for kv.flush().

b) When is it guaranteed that the kv put is in the change log (I am using kafka implementation).

c) When using messageCollector.send and systems.kafka.producer.producer.type=sync does that
guarantee that the message is in kafka log when the send returns.  I am new to kafka, but
it seems to me that if you have type=sync set, you still need to wait for the future objects
get to return,  is this what Samza does?

d) If my Samza job fails while processing a message, I fix it and deploy again, will the message
offset still point to a value <= the message I failed on.  ie I understand it can be earlier,
but is it possible the offset will now point to one higher.
View raw message