kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Parthasarathy, Mohan" <mpart...@hpe.com>
Subject Re: Is this a valid use case for reading local store ?
Date Thu, 01 Oct 2020 17:39:42 GMT
Hi Guozhang,

The async event trigger process is not running as a kafka streams application. It offers REST
interface where other applications post events which in turn needs to go through App1's state
and send requests to App2 via Kafka. Here is the diagram:

   KafkaTopics--->  App1 ---> App2 
                                   |
                                   V
                REST ---->App3

REST API to App3 and read the local store of App1 (IQ) and send requests to App2 (through
kafka topic, not shown above).  Conceptually it looks same as your use case. What do people
do if a kafka streams application (App1) has to offer REST interface also ?

-thanks
Mohan

´╗┐On 9/30/20, 5:01 PM, "Guozhang Wang" <wangguoz@gmail.com> wrote:

    Hello Mohan,
    
    If I understand correctly, your async event trigger process runs out of the
    streams application, that reads the state stores of app2 through the
    interactive query interface, right? This is actually a pretty common use
    case pattern for IQ :)
    
    
    Guozhang
    
    On Wed, Sep 30, 2020 at 1:22 PM Parthasarathy, Mohan <mparthas@hpe.com>
    wrote:
    
    > Hi,
    >
    > A traditional kafka streams application (App1)  reading data from a kafka
    > topic, doing aggregations resulting in some local state. The output of this
    > application is consumed by a different application(App2) for doing a
    > different task. Under some conditions, there is an external trigger (async
    > event) which needs to trigger requests for all the keys in the local store
    > to App2. To achieve this, we can read the local stores from all the
    > replicas and send the request to App2.
    >
    > This async event happens less frequently compared to the normal case that
    > leads to the state creation in the first place. Are there any caveats doing
    > it this way ? If not, any other suggestions ?
    >
    > Thanks
    > Mohan
    >
    >
    
    -- 
    -- Guozhang
    

Mime
View raw message