spot-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nathanael Smith <nathan...@apache.org>
Subject Re: Kerberos support
Date Thu, 07 Sep 2017 19:53:57 GMT
Hi Vladimir,

1. Apache Spot does not support kerberos as of the current release (1.0).

2. My first test with python-kafka  using “SASL_PLAINTEXT” and “GSSAPI” is resulting
in no brokers found.
I’m not able to say at this moment that this was completely a code issue not something with
infrastructure.

That being said,
____
Pro’s for librdkafka:

Kerberos has been supported for quite some time 
Used by several large projects including Confluent's own library

Cons for librdkafka:

requires more setup, This is not a show stopper by any means but should be considered as it’s
not a simple pip install.
will require more development time
____

I think if librdkafka can be implemented quickly and the usage of the KafkaTopic and KafkaConsumer
in spot-ingest/common/kafka_client.py remains the same it will add some stability.

- Nathanael



> On Sep 7, 2017, at 7:17 AM, Vladimir Shlyakhtin <vladimir.shlyakhtin@sstech.us>
wrote:
> 
> Hello,
> 
> I'm just wondering if Apache Spot fully supports Kerberos. As far as I understand not
yet. (According to jira cases in "Spot Kerberos enablement" Epic).
> Currently I'm working on installing Apache Spot on Kerberized environment. So was created
spot principal, granted access to hive, kafka and hdfs. Now I'm trying to run ingest component.
> But in ingest's code I see only "kinit" call. For python-kafka it is not enough. As it
is required security_protocol and sasl_mechanism for creation KafkaConsumer.
> I did such changes but still get exception "kafka.errors.NoBrokersAvailable: NoBrokersAvailable"
> It is the same as was reported in: https://github.com/dpkp/kafka-python/issues/1189
> Nathanael mentioned in SPOT-77 about librdkafka.
> What do you think? Is it worth to switch to librdkafka or continue use python-kafka.
> 
> Thanks
> 
> Regards,
> - Vladimir


Mime
View raw message