sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abraham Elmahrek <...@cloudera.com>
Subject Re: Sqoop Import with Postgresql and Kerberos
Date Fri, 21 Nov 2014 00:24:07 GMT
Hey there,

My mistake. Apparently the it's a bit more like an HTTP string:
jdbc:postgresql://postgres.lyleco.com:5432?kerberosServerName=postgres/
postgres.lyleco.com@LYLECO.COM. Also, are the JDBC jars in /var/lib/sqoop?
Or are they part of Sqoop's class path somehow?

The "Generic preauthentication" errors are discomforting. Are your
credentials renewable? I believe hadoop-auth starts a renewer thread that
runs "kinit -R". That could be causing that error.

-Abe

On Thu, Nov 20, 2014 at 3:56 PM, David Lyle <dlyle65535@gmail.com> wrote:

> Hi Abe,
>
> Thanks for the quick reply. Currently, the postgres instance in all
> that is kerberized. When I add the kerberosServerName I get:
>
> java.sql.SQLException: No suitable driver found for
> jdbc:postgresql://postgres.lyleco.com:5432;kerberosServerName=postgres/
> postgres.lyleco.com@LYLECO.COM
>
> or
>
> java.sql.SQLException: No suitable driver found for
> jdbc:postgresql://postgres.lyleco.com:5432&kerberosServerName=postgres/
> postgres.lyleco.com@LYLECO.COM
>
> (I tried both). I'm also troubled by the repeated kinit: Generic
> preauthentication failure while getting initial credentials at the
> very top. I don't understand how sqoop is delegating, but the kinit is
> surprising.
>
> -D...
>
>
> On Thu, Nov 20, 2014 at 6:15 PM, Abraham Elmahrek <abe@cloudera.com>
> wrote:
> > This is a long shot... but I thought I'd say something.
> >
> > If you have Hadoop configured for Kerberos, then Sqoop should
> automatically
> > use your kerberos credentials to run a MapReduce job. That MapReduce job
> > will run with its own Kerberos principal (normally yarn or mapred). This
> > means that Hadoop has access to postgresql as long as it requests a
> session
> > from the KDC, assuming postgresql and Hadoop are part of the same realm.
> My
> > understanding is the JDBC driver will handle this for you if you provide
> the
> > postgresql service name via "kerberosServerName" in the JDBC connection
> > string:
> > "jdbc:postgresql://postgres.lyleco.com:5432
> ;kerberosServerName=postgres/<host>@<REALM>".
> >
> > Ref: http://jdbc.postgresql.org/documentation/84/connect.html
> >
> > -Abe
> >
> > On Thu, Nov 20, 2014 at 2:53 PM, David Lyle <dlyle65535@gmail.com>
> wrote:
> >>
> >> Hi,
> >>
> >> I'm trying to use Sqoop to connect to a kerberized Postgres instance.
> >> At this time, I can connect from psql, but my sqoop session gives me
> >> the log below. Any idea where I'm going wrong?
> >>
> >> Thanks!
> >>
> >> -David...
> >>
> >>
> >> sqoop list-databases  -Djavax.security.auth.debug=true --connect
> >> "jdbc:postgresql://postgres.lyleco.com:5432" -verbose
> >>
> >> kinit: Generic preauthentication failure while getting initial
> credentials
> >> kinit: Generic preauthentication failure while getting initial
> credentials
> >>
> >> 14/11/20 17:26:37 INFO sqoop.Sqoop: Running Sqoop version:
> >> 1.4.4.2.1.7.0-784
> >> 14/11/20 17:26:37 DEBUG tool.BaseSqoopTool: Enabled debug logging.
> >> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Loaded manager factory:
> >> com.cloudera.sqoop.manager.DefaultManagerFactory
> >> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Trying ManagerFactory:
> >> com.cloudera.sqoop.manager.DefaultManagerFactory
> >> 14/11/20 17:26:37 DEBUG manager.DefaultManagerFactory: Trying with
> >> scheme: jdbc:postgresql:
> >> 14/11/20 17:26:37 INFO manager.SqlManager: Using default fetchSize of
> 1000
> >> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Instantiated ConnManager
> >> org.apache.sqoop.manager.PostgresqlManager@1663108
> >> 14/11/20 17:26:37 DEBUG manager.SqlManager: No connection paramenters
> >> specified. Using regular API for making connection.
> >> 14/11/20 17:26:37 ERROR manager.CatalogQueryManager: Failed to list
> >> databases
> >> org.postgresql.util.PSQLException: GSS Authentication failed
> >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49)
> >> at
> >>
> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380)
> >> at
> >>
> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108)
> >> at
> >>
> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
> >> at
> >>
> org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)
> >> at
> >>
> org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)
> >> at
> >>
> org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)
> >> at
> >>
> org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32)
> >> at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
> >> at org.postgresql.Driver.makeConnection(Driver.java:393)
> >> at org.postgresql.Driver.connect(Driver.java:267)
> >> at java.sql.DriverManager.getConnection(DriverManager.java:571)
> >> at java.sql.DriverManager.getConnection(DriverManager.java:233)
> >> at
> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824)
> >> at
> >>
> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
> >> at
> >>
> org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57)
> >> at
> org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49)
> >> at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> >> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
> >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222)
> >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231)
> >> at org.apache.sqoop.Sqoop.main(Sqoop.java:240)
> >> Caused by: javax.security.auth.login.LoginException: No LoginModules
> >> configured for pgjdbc
> >> at javax.security.auth.login.LoginContext.init(LoginContext.java:273)
> >> at javax.security.auth.login.LoginContext.<init>(LoginContext.java:418)
> >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42)
> >> ... 22 more
> >> 14/11/20 17:26:37 ERROR sqoop.Sqoop: Got exception running Sqoop:
> >> java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS
> >> Authentication failed
> >> java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS
> >> Authentication failed
> >> at
> >>
> org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:73)
> >> at
> org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49)
> >> at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> >> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
> >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222)
> >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231)
> >> at org.apache.sqoop.Sqoop.main(Sqoop.java:240)
> >> Caused by: org.postgresql.util.PSQLException: GSS Authentication failed
> >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49)
> >> at
> >>
> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380)
> >> at
> >>
> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108)
> >> at
> >>
> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
> >> at
> >>
> org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)
> >> at
> >>
> org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)
> >> at
> >>
> org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)
> >> at
> >>
> org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32)
> >> at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
> >> at org.postgresql.Driver.makeConnection(Driver.java:393)
> >> at org.postgresql.Driver.connect(Driver.java:267)
> >> at java.sql.DriverManager.getConnection(DriverManager.java:571)
> >> at java.sql.DriverManager.getConnection(DriverManager.java:233)
> >> at
> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824)
> >> at
> >>
> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
> >> at
> >>
> org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57)
> >> ... 7 more
> >> Caused by: javax.security.auth.login.LoginException: No LoginModules
> >> configured for pgjdbc
> >> at javax.security.auth.login.LoginContext.init(LoginContext.java:273)
> >> at javax.security.auth.login.LoginContext.<init>(LoginContext.java:418)
> >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42)
> >> ... 22 more
> >
> >
>

Mime
View raw message