sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suhas Satish <suhas.sat...@gmail.com>
Subject Re: sqoop import into secure Hbase with kerberos
Date Tue, 06 Aug 2013 17:31:17 GMT
Ataching the logs here at the time of authentication, I do not see any
error msges here.

/var/log/kadmind.log
/var/log/krb5kdc.log

Please let me know if there is any other places I can find other log files

Cheers,
Suhas.


On Mon, Aug 5, 2013 at 4:48 PM, Abraham Elmahrek <abe@cloudera.com> wrote:

> User,
>
> Could you please provide your KDC logs around the time you tried to
> authenticate?
>
> Note: A kerberos client will negotiate the encryption algorithm it
> can/will use with the KDC. It may choose AES-256.
>
> -Abe
>
>
> On Mon, Aug 5, 2013 at 3:55 PM, Suhas Satish <suhas.satish@gmail.com>wrote:
>
>> I generated a keytab with the following cmd and it supports multiple
>> encryption types other than aes256 as listed below.
>> But I still get the same error from sqoop import tool because the
>> sqoop.keytab is not being read (sqoop being the hbase client in this case).
>>
>> kadmin:  ktadd -k sqoop.keytab kuser1
>> Entry for principal kuser1 with kvno 2, encryption type
>> aes256-cts-hmac-sha1-96 added to keytab WRFILE:sqoop.keytab.
>> Entry for principal kuser1 with kvno 2, encryption type
>> aes128-cts-hmac-sha1-96 added to keytab WRFILE:sqoop.keytab.
>> Entry for principal kuser1 with kvno 2, encryption type des3-cbc-sha1
>> added to keytab WRFILE:sqoop.keytab.
>> Entry for principal kuser1 with kvno 2, encryption type arcfour-hmac
>> added to keytab WRFILE:sqoop.keytab.
>> Entry for principal kuser1 with kvno 2, encryption type des-hmac-sha1
>> added to keytab WRFILE:sqoop.keytab.
>> Entry for principal kuser1 with kvno 2, encryption type des-cbc-md5 added
>> to keytab WRFILE:sqoop.keytab.
>>
>> Here are some more debug logs I obtained from kerberos -
>>
>> *kadmin:  getprinc kuser1*
>> Principal: kuser1@QA.LAB
>> Expiration date: [never]
>> Last password change: Mon Aug 05 15:40:30 PDT 2013
>> Password expiration date: [none]
>> Maximum ticket life: 1 day 00:00:00
>> Maximum renewable life: 0 days 00:00:00
>> Last modified: Mon Aug 05 15:40:30 PDT 2013 (mapr/admin@QA.LAB)
>> Last successful authentication: [never]
>> Last failed authentication: [never]
>> Failed password attempts: 0
>> Number of keys: 6
>> Key: vno 2, aes256-cts-hmac-sha1-96, no salt
>> Key: vno 2, aes128-cts-hmac-sha1-96, no salt
>> Key: vno 2, des3-cbc-sha1, no salt
>> Key: vno 2, arcfour-hmac, no salt
>> Key: vno 2, des-hmac-sha1, no salt
>> Key: vno 2, des-cbc-md5, no salt
>> MKey: vno 1
>> Attributes:
>> Policy: [none]
>>
>> *getprinc hbase/qa-node133.qa.lab*
>> Principal: hbase/qa-node133.qa.lab@QA.LAB
>> Expiration date: [never]
>> Last password change: Mon Jul 29 19:17:46 PDT 2013
>> Password expiration date: [none]
>> Maximum ticket life: 0 days 10:00:00
>> Maximum renewable life: 7 days 00:00:00
>> Last modified: Mon Jul 29 19:17:46 PDT 2013 (kuser1/admin@QA.LAB)
>> Last successful authentication: [never]
>> Last failed authentication: [never]
>> Failed password attempts: 0
>> Number of keys: 6
>> Key: vno 2, aes256-cts-hmac-sha1-96, no salt
>> Key: vno 2, aes128-cts-hmac-sha1-96, no salt
>> Key: vno 2, des3-cbc-sha1, no salt
>> Key: vno 2, arcfour-hmac, no salt
>> Key: vno 2, des-hmac-sha1, no salt
>> Key: vno 2, des-cbc-md5, no salt
>> MKey: vno 1
>> Attributes:
>> Policy: [none]
>>
>>
>> Thanks,
>> Suhas.
>>
>>
>> On Mon, Aug 5, 2013 at 2:29 PM, Abraham Elmahrek <abe@cloudera.com>wrote:
>>
>>> There should be a password. You should have a keytab associated with
>>> that principal, which would allow you to authenticate as that principal.
>>> See
>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Security-Guide/CDH4-Security-Guide.htmlfor
more details on how that works.
>>>
>>> A couple of things...
>>> 1. You need to make your kerberos credentials renewable. Right now it
>>> seems like you cannot renew. See
>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Security-Guide/cdh4sg_topic_17.html
>>> .
>>> 2. AES256 encryption is not inherently supported. Did you install
>>> support for AES256?
>>>
>>> -Abe
>>>
>>>
>>> On Mon, Aug 5, 2013 at 1:53 PM, Suhas Satish <suhas.satish@gmail.com>wrote:
>>>
>>>> klist -e -v
>>>>
>>>> Ticket cache: FILE:/tmp/krb5cc_0
>>>> Default principal: kuser1@QA.LAB
>>>>
>>>> Valid starting     Expires            Service principal
>>>> 08/05/13 12:34:42  08/05/13 22:34:42  krbtgt/QA.LAB@QA.LAB
>>>> renew until 08/05/13 12:34:42, Etype (skey, tkt):
>>>> aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96
>>>>
>>>> Kerberos 5 version 1.10.3
>>>>
>>>> The principal in hbase-site.xml is
>>>> hbase/qa-node133.qa.lab@QA.LAB
>>>>
>>>> How do I create a credential using kinit matching that in
>>>> hbase-site.xml?  kinit  hbase/qa-node133.qa.lab   throws an error msg
>>>> *kinit: Password incorrect while getting initial credentials*
>>>> *although I know that there is no password for that principal. *
>>>> *
>>>> *
>>>> *
>>>> *
>>>>
>>>> Cheers,
>>>> Suhas.
>>>>
>>>>
>>>> On Mon, Aug 5, 2013 at 12:52 PM, Abraham Elmahrek <abe@cloudera.com>wrote:
>>>>
>>>>> Hi there,
>>>>>
>>>>> It seems like your client isn't authenticated in both cases. You seem
>>>>> to be receiving errors from HBase and Sqoop. Sqoop 1.4.3 should simply
work
>>>>> if your user is already authenticated. Internally, Sqoop is generating
>>>>> delegation tokens to communicate with HBase. It cannot do that without
>>>>> being properly authenticated first though.
>>>>>
>>>>> Could you provide the output of the following command:
>>>>> "klist -e -v"
>>>>>
>>>>> -Abe
>>>>>
>>>>>
>>>>> On Mon, Aug 5, 2013 at 12:15 PM, Suhas Satish <suhas.satish@gmail.com>wrote:
>>>>>
>>>>>> I have configured hbase 94.9  with kerberos successfully for
>>>>>> authentication and authorization as mentioned in the CDH security
docs. I
>>>>>> am using sqoop 1.4.3. Is there any configuration required from the
sqoop
>>>>>> client side for kerberos?
>>>>>>
>>>>>> I have the following permissions on hbase tables -
>>>>>> hbase(main):003:0> grant 'kuser1', 'RWXCA', 'demo'
>>>>>> ERROR: org.apache.hadoop.hbase.security.AccessDeniedException:
>>>>>> org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient
>>>>>> permissions (user=kuser1, scope=demo, family=, qualifer=, action=ADMIN)
>>>>>>
>>>>>>
>>>>>> bin/sqoop import --connect jdbc:mysql://10.10.1.10/TestDB    --table
>>>>>> t1  --hbase-table  t1  --column-family world
>>>>>>
>>>>>>
>>>>>> When I try to import into it using sqoop with the above cmd, I get
>>>>>> the following error -
>>>>>>
>>>>>>
>>>>>> 2013-08-05 11:59:33,121 ERROR
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer:
>>>>>> org.apache.hadoop.hbase.security.AccessDeniedException: Token
>>>>>> generation only allowed for Kerberos authenticated clients
>>>>>> at
>>>>>> org.apache.hadoop.hbase.security.token.TokenProvider.getAuthenticationToken(TokenProvider.java:87)
>>>>>>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>> at
>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>>  at
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>  at
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion.exec(HRegion.java:5576)
>>>>>> at
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.execCoprocessor(HRegionServer.java:3868)
>>>>>>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>> at
>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>>  at
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>  at
>>>>>> org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:308)
>>>>>> at
>>>>>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
>>>>>>
>>>>>>
>>>>>> Cheers,
>>>>>> Suhas.
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message