kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <matth...@confluent.io>
Subject Re: update of GlobalKTable
Date Fri, 28 Jun 2019 07:04:02 GMT
Your expectation and understanding how GlobalKTables work is correct.

If you can reproduce the issue in an IDE, set a breakpoint in
`GlobalStreamThread#pollAndUpdate()` to see if data is fetched from the
brokers.


-Matthias


On 6/26/19 7:09 AM, Imre Gábor wrote:
> Hi,
> 
> I am new to Kafka and observed a strange behavior. (I am using Kafka through the spring-kafka
library)
> 
> I think I misunderstand something about the "liveness" of the data in a global table.
> I thought if I define and materialize a GlobalKTable at the startup of my application,
and messages arrive later to the underlying topic, the data in the store are automatically
updated.
> i.e. if I have a method that queries the store of the table, it will see the most up-to-date
data. (With a bit of latency of course.)
> However, it seems not to be the case:
> 
> 
> 1.       My application defines a GlobalKTable over a topic, materialized with a name
and serializers passed.
> 
> 2.       I have a business logic method that calls store() method with the name of this
global table, and tries to get a value by key.
> 
> 3.       Next, I wrote a test case, that starts the application, and I send a message
with a producer to the appropriate topic, and I expect I can get it by its key from the global
table's store.
> 
> However, it returns null. I can see the message in the topic so it is surely delivered.
> 
> 4.       If I iterate through the store via store.all() before calling store.get(key),
the KeyValue is there with the appropriate key! However, store.get(key) still returns null.
> 
> 5.       I implemented hashCode and equals for my key class appropriately, so it cannot
be the problem
> 
> 6.       Also, it is a not about a small latency, because if I wait about a minute, still
cannot get the data.
> 
> 7.       If I modify my test case so that I do not send a test message (because there
is one in the topic already), and rerun the test (which starts the application and defined
the global table),
> 
> the value is correctly found by key. (I guess the message from the topic is read when
the table is created.)
> 
> Do you have any idea what am I missing?
> 
> Best regards
> 
> Gabor
> 


Mime
View raw message