kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harald Kirsch <harald.kir...@raytion.com>
Subject Re: Some general questions...
Date Tue, 06 Dec 2016 15:07:53 GMT
This sounds like you might want to run the Kafka broker on Windows. Have 
a look at https://issues.apache.org/jira/browse/KAFKA-1194 for possible 
issues with regard to log cleaning.


On 06.12.2016 00:50, Doyle, Keith wrote:
> We’re beginning to make use of Kafka, and it is encouraging.  But there
> are a couple of questions I’ve had a hard time finding answers for.
> We’re using the rdkafka-dotnet client on the consumer side and it’s
> straightforward as far as it goes.  However, documentation seems to be
> scant—the Wiki points to a FAQ which has, like, two questions neither of
> which are the questions we have.   And I can’t find a mailing list,
> forum, blog, or other community where questions can be asked.  I found
> some indication in the Git repository that there may be some API docs,
> but it’s not at all clear exactly where those are.
> So I’m posting that question here because I can’t find anywhere else
> that might be even remotely relevant to post it—where can I find out
> more info about rdkafka and particularly rdkafka-dotnet, and some way to
> ask questions that aren’t answered in the documentation?
> And second, my current question about rdkafka-dotnet, is the example
> consumers both seem to read an entire message into memory.   We don’t
> want to presume any particular message size, and may not want to cache
> the entire message in memory while processing it.   Is there an
> interface where we can consume messages via a stream, so that we can
> read chunks of a message and process them based on some kind of batch
> size that will allow us better control over memory usage?

View raw message