sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suhas Satish <suhas.sat...@gmail.com>
Subject Re: sqoop import into secure Hbase with kerberos
Date Tue, 06 Aug 2013 18:09:42 GMT
I was able to isolate this problem to the Sqoop side not picking up correct
kerberos credentials. Hbase is picking up the correct kerberos credentials
when Hbase put and scan are done in isolation without using Sqoop.

A direct map-reduce put into HBase uses the following 2 methods -
HBaseConfiguration.merge(conf, HBaseConfiguration.create(conf));
TableMapReduceUtil.initCredentials(job);

I was looking at how sqoop 1.4.3 does HBase puts to see if it converts
sqoop import arguments into map-reduce jobs and uses the above methods
somewhere. This is what I found -
HBasePutProcessor.java  - SqoopRecordProcessor that performs a HBase "put"
operation - has a method to get hadoop configuration, but none to merge any
kerberos specific configurations specified  in sqoop-site.xml-

  public Configuration getConf() {
    return this.conf;



HBaseUtil.java   - makes sure hbase jars are present on class path
PutTransformer.java  - converts jdbc statements in the form of K-V map into
hbase put commands and returns a list
ToStringPutTransformer.java - extends the above class

Does anyone know sqoop internals of how to specify kerberos configurations
and get sqoop to read them?

Cheers,
Suhas.


On Tue, Aug 6, 2013 at 10:31 AM, Suhas Satish <suhas.satish@gmail.com>wrote:

> Ataching the logs here at the time of authentication, I do not see any
> error msges here.
>
> /var/log/kadmind.log
> /var/log/krb5kdc.log
>
> Please let me know if there is any other places I can find other log files
>
> Cheers,
> Suhas.
>
>
> On Mon, Aug 5, 2013 at 4:48 PM, Abraham Elmahrek <abe@cloudera.com> wrote:
>
>> User,
>>
>> Could you please provide your KDC logs around the time you tried to
>> authenticate?
>>
>> Note: A kerberos client will negotiate the encryption algorithm it
>> can/will use with the KDC. It may choose AES-256.
>>
>> -Abe
>>
>>
>> On Mon, Aug 5, 2013 at 3:55 PM, Suhas Satish <suhas.satish@gmail.com>wrote:
>>
>>> I generated a keytab with the following cmd and it supports multiple
>>> encryption types other than aes256 as listed below.
>>> But I still get the same error from sqoop import tool because the
>>> sqoop.keytab is not being read (sqoop being the hbase client in this case).
>>>
>>> kadmin:  ktadd -k sqoop.keytab kuser1
>>> Entry for principal kuser1 with kvno 2, encryption type
>>> aes256-cts-hmac-sha1-96 added to keytab WRFILE:sqoop.keytab.
>>> Entry for principal kuser1 with kvno 2, encryption type
>>> aes128-cts-hmac-sha1-96 added to keytab WRFILE:sqoop.keytab.
>>> Entry for principal kuser1 with kvno 2, encryption type des3-cbc-sha1
>>> added to keytab WRFILE:sqoop.keytab.
>>> Entry for principal kuser1 with kvno 2, encryption type arcfour-hmac
>>> added to keytab WRFILE:sqoop.keytab.
>>> Entry for principal kuser1 with kvno 2, encryption type des-hmac-sha1
>>> added to keytab WRFILE:sqoop.keytab.
>>> Entry for principal kuser1 with kvno 2, encryption type des-cbc-md5
>>> added to keytab WRFILE:sqoop.keytab.
>>>
>>> Here are some more debug logs I obtained from kerberos -
>>>
>>> *kadmin:  getprinc kuser1*
>>> Principal: kuser1@QA.LAB
>>> Expiration date: [never]
>>> Last password change: Mon Aug 05 15:40:30 PDT 2013
>>> Password expiration date: [none]
>>> Maximum ticket life: 1 day 00:00:00
>>> Maximum renewable life: 0 days 00:00:00
>>> Last modified: Mon Aug 05 15:40:30 PDT 2013 (mapr/admin@QA.LAB)
>>> Last successful authentication: [never]
>>> Last failed authentication: [never]
>>> Failed password attempts: 0
>>> Number of keys: 6
>>> Key: vno 2, aes256-cts-hmac-sha1-96, no salt
>>> Key: vno 2, aes128-cts-hmac-sha1-96, no salt
>>> Key: vno 2, des3-cbc-sha1, no salt
>>> Key: vno 2, arcfour-hmac, no salt
>>> Key: vno 2, des-hmac-sha1, no salt
>>> Key: vno 2, des-cbc-md5, no salt
>>> MKey: vno 1
>>> Attributes:
>>> Policy: [none]
>>>
>>> *getprinc hbase/qa-node133.qa.lab*
>>> Principal: hbase/qa-node133.qa.lab@QA.LAB
>>> Expiration date: [never]
>>> Last password change: Mon Jul 29 19:17:46 PDT 2013
>>> Password expiration date: [none]
>>> Maximum ticket life: 0 days 10:00:00
>>> Maximum renewable life: 7 days 00:00:00
>>> Last modified: Mon Jul 29 19:17:46 PDT 2013 (kuser1/admin@QA.LAB)
>>> Last successful authentication: [never]
>>> Last failed authentication: [never]
>>> Failed password attempts: 0
>>> Number of keys: 6
>>> Key: vno 2, aes256-cts-hmac-sha1-96, no salt
>>> Key: vno 2, aes128-cts-hmac-sha1-96, no salt
>>> Key: vno 2, des3-cbc-sha1, no salt
>>> Key: vno 2, arcfour-hmac, no salt
>>> Key: vno 2, des-hmac-sha1, no salt
>>> Key: vno 2, des-cbc-md5, no salt
>>> MKey: vno 1
>>> Attributes:
>>> Policy: [none]
>>>
>>>
>>> Thanks,
>>> Suhas.
>>>
>>>
>>> On Mon, Aug 5, 2013 at 2:29 PM, Abraham Elmahrek <abe@cloudera.com>wrote:
>>>
>>>> There should be a password. You should have a keytab associated with
>>>> that principal, which would allow you to authenticate as that principal.
>>>> See
>>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Security-Guide/CDH4-Security-Guide.htmlfor
more details on how that works.
>>>>
>>>> A couple of things...
>>>> 1. You need to make your kerberos credentials renewable. Right now it
>>>> seems like you cannot renew. See
>>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Security-Guide/cdh4sg_topic_17.html
>>>> .
>>>> 2. AES256 encryption is not inherently supported. Did you install
>>>> support for AES256?
>>>>
>>>> -Abe
>>>>
>>>>
>>>> On Mon, Aug 5, 2013 at 1:53 PM, Suhas Satish <suhas.satish@gmail.com>wrote:
>>>>
>>>>> klist -e -v
>>>>>
>>>>> Ticket cache: FILE:/tmp/krb5cc_0
>>>>> Default principal: kuser1@QA.LAB
>>>>>
>>>>> Valid starting     Expires            Service principal
>>>>> 08/05/13 12:34:42  08/05/13 22:34:42  krbtgt/QA.LAB@QA.LAB
>>>>> renew until 08/05/13 12:34:42, Etype (skey, tkt):
>>>>> aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96
>>>>>
>>>>> Kerberos 5 version 1.10.3
>>>>>
>>>>> The principal in hbase-site.xml is
>>>>> hbase/qa-node133.qa.lab@QA.LAB
>>>>>
>>>>> How do I create a credential using kinit matching that in
>>>>> hbase-site.xml?  kinit  hbase/qa-node133.qa.lab   throws an error msg
>>>>> *kinit: Password incorrect while getting initial credentials*
>>>>> *although I know that there is no password for that principal. *
>>>>> *
>>>>> *
>>>>> *
>>>>> *
>>>>>
>>>>> Cheers,
>>>>> Suhas.
>>>>>
>>>>>
>>>>> On Mon, Aug 5, 2013 at 12:52 PM, Abraham Elmahrek <abe@cloudera.com>wrote:
>>>>>
>>>>>> Hi there,
>>>>>>
>>>>>> It seems like your client isn't authenticated in both cases. You
seem
>>>>>> to be receiving errors from HBase and Sqoop. Sqoop 1.4.3 should simply
work
>>>>>> if your user is already authenticated. Internally, Sqoop is generating
>>>>>> delegation tokens to communicate with HBase. It cannot do that without
>>>>>> being properly authenticated first though.
>>>>>>
>>>>>> Could you provide the output of the following command:
>>>>>> "klist -e -v"
>>>>>>
>>>>>> -Abe
>>>>>>
>>>>>>
>>>>>> On Mon, Aug 5, 2013 at 12:15 PM, Suhas Satish <suhas.satish@gmail.com
>>>>>> > wrote:
>>>>>>
>>>>>>> I have configured hbase 94.9  with kerberos successfully for
>>>>>>> authentication and authorization as mentioned in the CDH security
docs. I
>>>>>>> am using sqoop 1.4.3. Is there any configuration required from
the sqoop
>>>>>>> client side for kerberos?
>>>>>>>
>>>>>>> I have the following permissions on hbase tables -
>>>>>>> hbase(main):003:0> grant 'kuser1', 'RWXCA', 'demo'
>>>>>>> ERROR: org.apache.hadoop.hbase.security.AccessDeniedException:
>>>>>>> org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient
>>>>>>> permissions (user=kuser1, scope=demo, family=, qualifer=, action=ADMIN)
>>>>>>>
>>>>>>>
>>>>>>> bin/sqoop import --connect jdbc:mysql://10.10.1.10/TestDB
>>>>>>>  --table t1  --hbase-table  t1  --column-family world
>>>>>>>
>>>>>>>
>>>>>>> When I try to import into it using sqoop with the above cmd,
I get
>>>>>>> the following error -
>>>>>>>
>>>>>>>
>>>>>>> 2013-08-05 11:59:33,121 ERROR
>>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer:
>>>>>>> org.apache.hadoop.hbase.security.AccessDeniedException: Token
>>>>>>> generation only allowed for Kerberos authenticated clients
>>>>>>> at
>>>>>>> org.apache.hadoop.hbase.security.token.TokenProvider.getAuthenticationToken(TokenProvider.java:87)
>>>>>>>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>> at
>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>>>  at
>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>>  at
>>>>>>> org.apache.hadoop.hbase.regionserver.HRegion.exec(HRegion.java:5576)
>>>>>>> at
>>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.execCoprocessor(HRegionServer.java:3868)
>>>>>>>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>> at
>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>>>  at
>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>>  at
>>>>>>> org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:308)
>>>>>>> at
>>>>>>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
>>>>>>>
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Suhas.
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message