sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Lyle <dlyle65...@gmail.com>
Subject Re: Sqoop Import with Postgresql and Kerberos
Date Thu, 20 Nov 2014 23:56:33 GMT
Hi Abe,

Thanks for the quick reply. Currently, the postgres instance in all
that is kerberized. When I add the kerberosServerName I get:

java.sql.SQLException: No suitable driver found for
jdbc:postgresql://postgres.lyleco.com:5432;kerberosServerName=postgres/postgres.lyleco.com@LYLECO.COM

or

java.sql.SQLException: No suitable driver found for
jdbc:postgresql://postgres.lyleco.com:5432&kerberosServerName=postgres/postgres.lyleco.com@LYLECO.COM

(I tried both). I'm also troubled by the repeated kinit: Generic
preauthentication failure while getting initial credentials at the
very top. I don't understand how sqoop is delegating, but the kinit is
surprising.

-D...


On Thu, Nov 20, 2014 at 6:15 PM, Abraham Elmahrek <abe@cloudera.com> wrote:
> This is a long shot... but I thought I'd say something.
>
> If you have Hadoop configured for Kerberos, then Sqoop should automatically
> use your kerberos credentials to run a MapReduce job. That MapReduce job
> will run with its own Kerberos principal (normally yarn or mapred). This
> means that Hadoop has access to postgresql as long as it requests a session
> from the KDC, assuming postgresql and Hadoop are part of the same realm. My
> understanding is the JDBC driver will handle this for you if you provide the
> postgresql service name via "kerberosServerName" in the JDBC connection
> string:
> "jdbc:postgresql://postgres.lyleco.com:5432;kerberosServerName=postgres/<host>@<REALM>".
>
> Ref: http://jdbc.postgresql.org/documentation/84/connect.html
>
> -Abe
>
> On Thu, Nov 20, 2014 at 2:53 PM, David Lyle <dlyle65535@gmail.com> wrote:
>>
>> Hi,
>>
>> I'm trying to use Sqoop to connect to a kerberized Postgres instance.
>> At this time, I can connect from psql, but my sqoop session gives me
>> the log below. Any idea where I'm going wrong?
>>
>> Thanks!
>>
>> -David...
>>
>>
>> sqoop list-databases  -Djavax.security.auth.debug=true --connect
>> "jdbc:postgresql://postgres.lyleco.com:5432" -verbose
>>
>> kinit: Generic preauthentication failure while getting initial credentials
>> kinit: Generic preauthentication failure while getting initial credentials
>>
>> 14/11/20 17:26:37 INFO sqoop.Sqoop: Running Sqoop version:
>> 1.4.4.2.1.7.0-784
>> 14/11/20 17:26:37 DEBUG tool.BaseSqoopTool: Enabled debug logging.
>> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Loaded manager factory:
>> com.cloudera.sqoop.manager.DefaultManagerFactory
>> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Trying ManagerFactory:
>> com.cloudera.sqoop.manager.DefaultManagerFactory
>> 14/11/20 17:26:37 DEBUG manager.DefaultManagerFactory: Trying with
>> scheme: jdbc:postgresql:
>> 14/11/20 17:26:37 INFO manager.SqlManager: Using default fetchSize of 1000
>> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Instantiated ConnManager
>> org.apache.sqoop.manager.PostgresqlManager@1663108
>> 14/11/20 17:26:37 DEBUG manager.SqlManager: No connection paramenters
>> specified. Using regular API for making connection.
>> 14/11/20 17:26:37 ERROR manager.CatalogQueryManager: Failed to list
>> databases
>> org.postgresql.util.PSQLException: GSS Authentication failed
>> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49)
>> at
>> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380)
>> at
>> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108)
>> at
>> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
>> at
>> org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)
>> at
>> org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)
>> at
>> org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)
>> at
>> org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32)
>> at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
>> at org.postgresql.Driver.makeConnection(Driver.java:393)
>> at org.postgresql.Driver.connect(Driver.java:267)
>> at java.sql.DriverManager.getConnection(DriverManager.java:571)
>> at java.sql.DriverManager.getConnection(DriverManager.java:233)
>> at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824)
>> at
>> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
>> at
>> org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57)
>> at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49)
>> at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222)
>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231)
>> at org.apache.sqoop.Sqoop.main(Sqoop.java:240)
>> Caused by: javax.security.auth.login.LoginException: No LoginModules
>> configured for pgjdbc
>> at javax.security.auth.login.LoginContext.init(LoginContext.java:273)
>> at javax.security.auth.login.LoginContext.<init>(LoginContext.java:418)
>> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42)
>> ... 22 more
>> 14/11/20 17:26:37 ERROR sqoop.Sqoop: Got exception running Sqoop:
>> java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS
>> Authentication failed
>> java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS
>> Authentication failed
>> at
>> org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:73)
>> at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49)
>> at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222)
>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231)
>> at org.apache.sqoop.Sqoop.main(Sqoop.java:240)
>> Caused by: org.postgresql.util.PSQLException: GSS Authentication failed
>> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49)
>> at
>> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380)
>> at
>> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108)
>> at
>> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
>> at
>> org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)
>> at
>> org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)
>> at
>> org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)
>> at
>> org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32)
>> at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
>> at org.postgresql.Driver.makeConnection(Driver.java:393)
>> at org.postgresql.Driver.connect(Driver.java:267)
>> at java.sql.DriverManager.getConnection(DriverManager.java:571)
>> at java.sql.DriverManager.getConnection(DriverManager.java:233)
>> at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824)
>> at
>> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
>> at
>> org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57)
>> ... 7 more
>> Caused by: javax.security.auth.login.LoginException: No LoginModules
>> configured for pgjdbc
>> at javax.security.auth.login.LoginContext.init(LoginContext.java:273)
>> at javax.security.auth.login.LoginContext.<init>(LoginContext.java:418)
>> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42)
>> ... 22 more
>
>

Mime
View raw message