kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <matth...@confluent.io>
Subject Re: [DISCUSS] KIP-247: Add public test utils for Kafka Streams
Date Tue, 16 Jan 2018 21:56:38 GMT
Thanks a lot for the comments.

@Guozhang: I updated the KIP accordingly. With regard to potential
client test-utils, I agree, but not sure how to resolve it. I guess, we
just need to move the class for this case later on. (One reason to
annotate all classes with @Evolving)

@Bill: The new artifact will be included without the "classifier:test"
tag, because it's a regular dependency (the published artifact is not a
test artifact). For existing code, we don't remove any existing internal
test class in 1.1.0 so the code should still work -- but as internal
test classes are internal, we don't provide any guarantee about
compatibility in the first place.

About `ConsumerRecordFactory`: I think all overloads are useful -- if
you remove the overload taking a topicName as input, you cannot
overwrite the defaultTopicName and thus need to create a factory for
each input topic. On the other hand, if you remove the overloads without
talking a topicName, you force people to define a defaultTopicName, and
thus they need to create a factory for each topic, too. The goal is to
allow the usage of a single factory even if there are multiple topics.
The defaultName is useful, if you want to create a lot to record for a
single topic, but not a good fit if you create just a few record for
each topic. (At least, that is my thinking.).

The `null` approach might work, too, but I think this result in ugly
boiler plate code and thus I personally prefer the add the overloads.
Let me know if you have a strong opinion for the `null` approach with
reduced number of overloads.

Hope to add the helpers for result verification this week...


@Jeff: the available method include global stores -- I added a comment
to the KIP


-Matthias


On 1/16/18 11:28 AM, Jeff Klukas wrote:
> From what I can tell, global state stores are managed separately from other
> state stores and are accessed via different methods.
> 
> Do the proposed methods on TopologyTestDriver (such as getStateStore) cover
> global stores? If not, can we add an interface for accessing and testing
> global stores in the scope of this KIP?
> 
> On Thu, Jan 11, 2018 at 9:06 PM, Matthias J. Sax <matthias@confluent.io>
> wrote:
> 
>> Dear Kafka community,
>>
>> I want to propose KIP-247 to add public test utils to the Streams API.
>> The goal is to simplify testing of Kafka Streams applications.
>>
>> Please find details in the wiki:
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> 247%3A+Add+public+test+utils+for+Kafka+Streams
>>
>> This is an initial KIP, and we hope to add more utility functions later.
>> Thus, this KIP is not comprehensive but a first step. Of course, we can
>> enrich this initial KIP if we think it falls too short. But we should
>> not aim to be comprehensive to keep the scope manageable.
>>
>> In fact, I think we should add some more helpers to simplify result
>> verification. I will update the KIP with this asap. Just wanted to start
>> the discussion early on.
>>
>> An initial WIP PR can be found here:
>> https://github.com/apache/kafka/pull/4402
>>
>> I also included the user-list (please hit "reply-all" to include both
>> lists in this KIP discussion).
>>
>> Thanks a lot.
>>
>>
>> -Matthias
>>
>>
>>
> 


Mime
View raw message