From user-return-9155-apmail-drill-user-archive=drill.apache.org@drill.apache.org Thu Apr 19 08:34:44 2018 Return-Path: X-Original-To: apmail-drill-user-archive@www.apache.org Delivered-To: apmail-drill-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D5B29185A2 for ; Thu, 19 Apr 2018 08:34:44 +0000 (UTC) Received: (qmail 37888 invoked by uid 500); 19 Apr 2018 08:34:44 -0000 Delivered-To: apmail-drill-user-archive@drill.apache.org Received: (qmail 37825 invoked by uid 500); 19 Apr 2018 08:34:44 -0000 Mailing-List: contact user-help@drill.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@drill.apache.org Delivered-To: mailing list user@drill.apache.org Received: (qmail 37802 invoked by uid 99); 19 Apr 2018 08:34:43 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 19 Apr 2018 08:34:43 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 1ABC3C219A for ; Thu, 19 Apr 2018 08:34:43 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.889 X-Spam-Level: * X-Spam-Status: No, score=1.889 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=yahoo.com.au Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id QWd9of1iV-nW for ; Thu, 19 Apr 2018 08:34:40 +0000 (UTC) Received: from sonic315-49.consmr.mail.ne1.yahoo.com (sonic315-49.consmr.mail.ne1.yahoo.com [66.163.190.175]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 4C1AB5F11F for ; Thu, 19 Apr 2018 08:34:40 +0000 (UTC) X-YMail-OSG: l429NZoVM1mOVLzSkPiiN9CWPn6SCvfpR_thBUP5zcMRCrn7fw1GJG_Q_3NbeAE 4ONuXxb3uGGRsmTbrBSsZ4_Eyx_jYDyW76EzOP5sF608pZYmvmbNKv74v2HwLaXwj24QMlKDPOXq JfCfBN2Rakz_ujFB7DnT3pJFUHedmSav1.eaRhDsiaZME.fAQSPS4..7jqsU5ah6XHfr.WjXBkBI zToOm7jqz5dCqYCKKBYdjNfge3zMC3dT_Gv.x5Z9VeX3NON9.1hVL1YG6Vi73H2nFRTfx.gH5FeM 6fEgKnHhv0Vn5mDyEoqzztYvR20ygYKHRWyjR24S6PDlvXOZcGd0MKQy5WkFKvQLH5oX9C0dgIbZ xDqBX8gi7SHbV87IjzdO4aswN1sXCc7MtS94KVoLHiluVNVMDE.vXHiNF8iZmNaU5RXpAc7gDgfN d0mQOujXIuatzQD.FK7qfLcx1VWMApEd4Z0k.TKlwMYoQ4Jyq_bxM8GyJ7UQYZ.3PGiJ5uhLP0uL LaxgPp9DjEl2NhcwPO4oiFazkg2h8ah8rdrw.ncIcru9xVvm6OOJQTUKx3KbwfB_vlM0- Received: from sonic.gate.mail.ne1.yahoo.com by sonic315.consmr.mail.ne1.yahoo.com with HTTP; Thu, 19 Apr 2018 08:34:32 +0000 Date: Thu, 19 Apr 2018 08:30:32 +0000 (UTC) From: Spiro Ketal Reply-To: Spiro Ketal To: Sorabh Hamirwasia Cc: "user@drill.apache.org\"" Message-ID: <1383209295.2494824.1524126632272@mail.yahoo.com> Subject: =?UTF-8?Q?Re:_ldap,_kerberos_zookeeper_and_drill_in?= =?UTF-8?Q?tegration_-_Drill_failing_to=C2=A0_authenticate?= MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_2494823_2080089041.1524126632267" References: <1383209295.2494824.1524126632272.ref@mail.yahoo.com> X-Mailer: WebService/1.1.11782 YMailNorrin Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36 ------=_Part_2494823_2080089041.1524126632267 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi Sorabh,Thankyou for the quick response (and apologies for the second pos= t - I didn't think it had gone through).=C2=A0 I'm effectively trying to model a large machine and wanted to keep my initi= al post short. I'll give a more detailed description. The system layout is = as follows:node000.local ldap/kerberos/NTP/CA/DNS node.node001.local zookee= per/kafka/drillnode002.local zookeeper/kafka/drillnode003.local=C2=A0zookee= per/kafka/drillnode004.local namenode/datanodenode005.local secondary namen= ode/datanodenode006.local datanodenode007.local datanode clusterid, local, LOCAL and node00N represent the real hostnames, cluster, = REALM and FQDNs. I've set up principles as follows:zookeeper/clusterid@LOCALzclient/clusteri= d@LOCAL I could not get zk to work with a single clusterid principle across the 3-m= ember zk cluster so resorted to:zookeeper/node001.local -> zookeeper.keytab= (on node001)zookeeper/node002.local ->=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0"=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0(on node002) zookeeper/node002.local ->=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0"=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(on no= de003) zclient/node001.local -> zclient.keytab (on node001)zclient/node002.local -= >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 (on node002) zclient/node002.local ->=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (on node003) Zookeeper works with this config. The relevant drill config is as follows:=C2=A0impersonation: {=C2=A0 =C2=A0= enabled: true,=C2=A0 =C2=A0 max_chained_user_hops: 3=C2=A0 },=C2=A0securit= y.auth: {=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0=C2=A0 =C2=A0 mechanisms:["KERBEROS","PLAIN"],=C2= =A0=C2=A0=C2=A0 =C2=A0 principal:"drill/clusterid@LOCAL",=C2=A0=C2=A0=C2=A0= =C2=A0 keytab:"/etc/security/keytabs/drill.keytab"=C2=A0=C2=A0=C2=A0 }=C2= =A0=C2=A0security.user: {=C2=A0 =C2=A0 auth.enabled: true,=C2=A0 =C2=A0 aut= h.packages +=3D "org.apache.drill.exec.rpc.user.security",=C2=A0 =C2=A0 aut= h.impl: "pam4j",=C2=A0 =C2=A0 auth.pam_profiles: ["sudo", "login"],=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0=C2=A0 } The permissions on the keytab are as follows (I have not tried to integrate= HDFS, yet):-rw------- 1 drill=C2=A0 =C2=A0 =C2=A0drill=C2=A0 =C2=A0 =C2=A0= 558 Apr 18 20:11 drill.keytab-rw------- 1 drill=C2=A0 =C2=A0 =C2=A0drill=C2= =A0 =C2=A0 =C2=A0614 Apr 18 20:11 drill_client.keytab-rw------- 1 zookeeper= zookeeper 302 Apr 18 18:27 zclient.keytab-rw------- 1 zookeeper zookeeper = 310 Apr 18 18:27 zookeeper.keytab drill@node000:~$ id drilluid=3D649(drill) gid=3D1002(drill) groups=3D1002(d= rill),994(zookeeper),993(hadoop)drill@node000:~$ id zookeeperuid=3D996(zook= eeper) gid=3D994(zookeeper) groups=3D994(zookeeper) I'll re-read your posts and take a good look when I'm less tired.Thanks aga= in.Cheers, Chris. From: Sorabh Hamirwasia To: "user@drill.apache.org" <= user@drill.apache.org>,=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Spiro Ketal=C2=A0 = Subject: Re: ldap, kerberos zookeeper and drill in= tegration - Drill failing to=C2=A0 authenticateDate: Thu, 19 Apr 2018 03:45= :40 +0000 Hi Spiro,For the error while connecting using sqlline:- Do you have TGT gen= erated for your client user which you are using to connect to Drill ? If ye= s can you check if sqlline process user has access to that ticket file or n= ot ?=C2=A0Can you please share your sqlline command? Also can you share you= r drill-override.conf config ? Instruction on how to configure for Kerberos on server and client side are = here[1]. Not sure if you got chance to look at it. As far as enabling Drill authentication to zookeeper is concerned I have no= t tried it but Drill internally uses curator framework to connect to Zookee= per. So it will depend upon how Curator supports authenticating using Kerbe= ros. From quick search it looks like just providing a JAAS conf file to bel= ow system property should be good enough. -Djava.security.auth.login.config I would say first let's try to make DrillClient to Drillbit path working wi= th Kerberos. [1]: https://drill.apache.org/docs/configuring-kerberos-security/ Thanks,Sorabh ________________________________From: Spiro Ketal Sent: Wednesday, April 18, 2018 7:41:55 PMTo: user@drill.apache.org= Subject: ldap, kerberos zookeeper and drill integration - Drill failing to = authenticate Dear Apache List Members,I have a test system that comprises of several VMs= One provides (integrated) OpenLDAP/Kerberos5 services (LDAP backend) and I= 've integrated zookeeper with this (via JAAS). The authentication works=C2= =A0but I had to use a per node config (with FQDNs) to achieve this. (ie. zk= /node0001.my.domain; zk/node0002.my.domain, ..., instead of zk/nodes@MY.DOM= AIN)My goal is to have drill authenticate and to be able to use the underly= ing java DoAs() functionality to interact with HDFS and zookeeper. I don't = quite understand how the kerberos authentication works in the case of drill= . Does the drill software provide a kerberos authenticated client connectio= n to zookeeper? The drill cluster appears to start OK (without client-side = authentication to zookeeper - which I'd like to remedy) but I seem to be re= ceiving errors relating to GSSAPI when I attempt to connect to the drill vi= a sqlline:=C2=A0Error: Failure in connecting to Drill: org.apache.drill.exe= c.rpc.RpcException: javax.security.sasl.SaslException: Failed to login. [Ca= used by javax.security.auth.login.LoginException: Unable to obtain password= from user]=C2=A0(state=3D,code=3D0)I've tried various combinations but can= 't seem to get drill to authenticate.Any assistance or pointers would be gr= eatly appreciated.Thanks.Cheers, Spiro. ------=_Part_2494823_2080089041.1524126632267--