kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Taylor <gary.tay...@hismessages.com>
Subject Re: Developing with kafka and other non jvm languages
Date Mon, 26 Mar 2018 07:20:45 GMT
Hi Matthias,

  Thanks for the reply.  Ill start hunting around to see what I can find with regards to clients.
 I had just looked at the 2 most popular ones but you never know, the features I want may
be in another library



> On 25 Mar 2018, at 20:24, Matthias J. Sax <matthias@confluent.io> wrote:
> Gary,
> The Apache Kafka project itself, only maintains Java clients -- thus,
> your are right that those are the primary and best supported clients.
> (Kafka used to have Scala clients, but those are all deprecated now and
> will be removed eventually.)
> Clients in other languages are not part of Apache Kafka project itself
> but developed by third parties. Thus, the quality and feature set of
> those clients can vary. A list of known third party clients is
> maintained in the Kafka Wiki:
> https://cwiki.apache.org/confluence/display/KAFKA/Clients
> If a client does not support a feature, it's a client limitation only.
> The Kafka protocol is a binary protocol and thus it's up to the client
> to (fully) implement the protocol -- the broker does not even know in
> which language a client is written. (cf
> https://kafka.apache.org/protocol) Thus, you should open feature
> requests for clients to support pattern subscription if it's missing and
> you need it.
> If you need help to get librdkafka to compile on arm, you might want to
> reach out to on Github (https://github.com/edenhill/librdkafka) or
> Confluent mailing list
> (https://groups.google.com/forum/#!forum/confluent-platform) or
> Confluent Slack (https://launchpass.com/confluentcommunity).
> Hope this helps!
> -Matthias
> On 3/25/18 12:46 AM, Gary Taylor wrote:
>> Hi,
>>  This is a fairly generic question but has some specifics too
>> Ill ask the specific first - I am trying to use golang to talk to kafka and it works,
but a fairly important part of my application is to subscribe to information in many topics
where the topic is matched server side and will include new topics added since the subscribe.
 I have read that kafka can do this, but I cannot see any way of asking it to do it via golang
apart from with the library https://github.com/confluentinc/confluent-kafka-go <https://github.com/confluentinc/confluent-kafka-go>
- which uses a library called librdkafka which I would have to compile and I am struggling
as I am using an arm processor (long story behind that one - but I don’t think this solution
will ever work with an arm processor).  All other libraries seem to not offer this.  Is this
because of some underlying restriction in kafka or do you think it is going to be a case of
keep hunting until I find one that does ?
>> Then the more generic question - I get the feeling that Scala and Java are first
class citizens when it comes to using kafka.  Whilst I love Scala - I am learning golang and
am more generally a ruby developer.  Am I likely go get a ‘second class’ service with
these other languages - with restrictions around zookeeper for example (I think I read something
about having to have knowledge of which partition to subscribe to - where zookeeper would
normally track this but there is no API for it) ?
>> Many Thanks
>> Gary Taylor

View raw message