kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guozhang Wang <wangg...@gmail.com>
Subject Re: [DISCUSS] KIP-247: Add public test utils for Kafka Streams
Date Tue, 16 Jan 2018 18:01:40 GMT
Thanks Matthias, I made a pass over the wiki and left some comments; I see
you have addressed most of them. Here are a few more:

1. "TopologyTestDriver#process()": how about rename it to "pipeInput" or
"sendInput"?

2. For "ConsumerRecordFactory" constructor where "startTimestampMs" is not
specified, what would be the default value?


This is not a comment but just reminder:

3. ConsumerRecordFactory: I have to admit that putting this class in
o.a.k.streams.test may not be ideal, and if we are going to have an
o.a.k.clients.test in the future testing artifact this may better be moved
there.


Guozhang


On Mon, Jan 15, 2018 at 2:59 AM, Saïd Bouras <said.bouras@gmail.com> wrote:

> Hi Matthias,
>
> I read the KIP and it will be very helpful thanks to the changes, I don't
> see though a part that handle topologies that use avro schemas, is it in
> the scope of the KIP ?
>
> I open an issue two month ago in the schema-registry repo :
> https://github.com/confluentinc/schema-registry/issues/651 that explain
> that when testing topologies using schema registry, the schema registry
> client mock is not thread safe and thus in the different processors nodes
> when deserializing it will not work...
>
> In my unit tests I wrapped the mock schema registry client into a singleton
> but this solution is not enough satisfying.
>
> Thanks in advance, regards :-)
>
>
> On Fri, Jan 12, 2018 at 3:06 AM Matthias J. Sax <matthias@confluent.io>
> wrote:
>
> > Dear Kafka community,
> >
> > I want to propose KIP-247 to add public test utils to the Streams API.
> > The goal is to simplify testing of Kafka Streams applications.
> >
> > Please find details in the wiki:
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 247%3A+Add+public+test+utils+for+Kafka+Streams
> >
> > This is an initial KIP, and we hope to add more utility functions later.
> > Thus, this KIP is not comprehensive but a first step. Of course, we can
> > enrich this initial KIP if we think it falls too short. But we should
> > not aim to be comprehensive to keep the scope manageable.
> >
> > In fact, I think we should add some more helpers to simplify result
> > verification. I will update the KIP with this asap. Just wanted to start
> > the discussion early on.
> >
> > An initial WIP PR can be found here:
> > https://github.com/apache/kafka/pull/4402
> >
> > I also included the user-list (please hit "reply-all" to include both
> > lists in this KIP discussion).
> >
> > Thanks a lot.
> >
> >
> > -Matthias
> >
> >
> >
>
> --
>
> Saïd BOURAS
>
> Consultant Big Data
> Mobile: 0662988731
> Zenika Paris
> 10 rue de Milan 75009 Paris
> Standard : +33(0)1 45 26 19 15 <+33(0)145261915> - Fax : +33(0)1 72 70 45
> 10
> <+33(0)172704510>
>



-- 
-- Guozhang

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message