drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ron Cecchini <roncecch...@comcast.net>
Subject Re: Problem running Drill in a Docker container in OpenShift
Date Fri, 31 Jan 2020 06:13:51 GMT
So, apparently I spoke too soon, as my image-from-an-image in OpenShift actually *didn't* start
up successfully.

I'll take Ted's and Charles' comments in the current "[DISCUSS]" thread regarding attracting
users who may never contribute a line of code but nevertheless report on their use of Drill
and problems they may encounter as license to ask a follow up question...  Besides, with the
growing usage of OpenShift, I'm sure I won't be the last one trying to do this!  

You guys were talking about attracting enthusiastic users who want to spread the word to their
friends.  Well, as a bit of a backstory, it was an enthusiastic analyst here who floated the
idea of replacing our MongoDB + JSON with Drill + Parquet.  Eventually our tech lead signed
on to the idea and I got tasked to help out.  If this all works out, I'm sure word will spread
to other analysts.  So here I am...  (I don't mind being the "canary in the coalmine", so
to speak.  I've often had to take on that role.  And I get to learn new things...)

Ok, enough blah-blah-blah.  The problem I'm having, as far as I can tell, stems from the fact
that OpenShift doesn't set a user name when running a container.  In particular, the Java
System.getProperty("user.name") and "user.home" calls return "?" as reported in the Zookeeper
(I'm in embedded mode but the Zookeeper config is still being filled in) section of the sqlline.log:

 [main] INFO   o.apache.drill.exec.server.Drillbit - Drillbit environment: user.name=?
 [main] INFO   o.apache.drill.exec.server.Drillbit - Drillbit environment: user.home=?

As a quick point of comparison, when I run a Drill Docker on my desktop (not in OpenShift),
with zero config changes, everything of course works fine, and "user.name" is "root" and "user.home"
is "/root".  (Probably because I installed Docker with a "sudo yum install".)  Similarly,
when I run an embedded-Drill on my desktop installed from the source *tar.gz - i.e. not in
a Docker, and not in OpenShift - "user.name" is my $USER and "user.home" is my $HOME, and
everything again runs fine.

I'll include the big stack trace at the bottom, but really the only question I have at the
moment is:

Given that I don't think I can get OpenShift to set or pass in a "user.name" property for
the JDK to get, is there an environment variable (OpenShift *can* pass those in) or a magical
setting in drill-override.conf that will let me get past this "invalid null input: name" problem
I'm running into during the "login" phase of the startup?

And if so, could the "user.name" be set to anything or would it have to be root?  (I don't
quite understand the relationship between the "user.name" and the Hadoop login, etc.)

Thank you so much for any help!

Ron

--------------------------------------------------------------------------------

Error: Failure in starting embedded Drillbit: org.apache.drill.exec.exception.DrillbitStartupException:
Failed to login. (state=,code=0)
java.sql.SQLException: Failure in starting embedded Drillbit: org.apache.drill.exec.exception.DrillbitStartupException:
Failed to login.
	at org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:143)
	at org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:67)
	at org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:67)
	at org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:138)
	at org.apache.drill.jdbc.Driver.connect(Driver.java:75)
	at sqlline.DatabaseConnection.connect(DatabaseConnection.java:135)
	at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:192)
	at sqlline.Commands.connect(Commands.java:1364)
	at sqlline.Commands.connect(Commands.java:1244)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
	at sqlline.SqlLine.dispatch(SqlLine.java:730)
	at sqlline.SqlLine.initArgs(SqlLine.java:410)
	at sqlline.SqlLine.begin(SqlLine.java:515)
	at sqlline.SqlLine.start(SqlLine.java:267)
	at sqlline.SqlLine.main(SqlLine.java:206)
Caused by: org.apache.drill.exec.exception.DrillbitStartupException: Failed to login.
	at org.apache.drill.exec.server.BootStrapContext.login(BootStrapContext.java:161)
	at org.apache.drill.exec.server.BootStrapContext.<init>(BootStrapContext.java:82)
	at org.apache.drill.exec.server.Drillbit.<init>(Drillbit.java:171)
	at org.apache.drill.exec.server.Drillbit.<init>(Drillbit.java:135)
	at org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:133)
	... 18 more
Caused by: org.apache.hadoop.security.KerberosAuthException: failure to login: javax.security.auth.login.LoginException:
java.lang.NullPointerException: invalid null input: name
	at com.sun.security.auth.UnixPrincipal.<init>(UnixPrincipal.java:71)
	at com.sun.security.auth.module.UnixLoginModule.login(UnixLoginModule.java:133)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
	at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
	at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
	at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
	at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
	at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:1926)
	at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1837)
	at org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:710)
	at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:660)
	at org.apache.drill.exec.server.BootStrapContext.login(BootStrapContext.java:156)
	at org.apache.drill.exec.server.BootStrapContext.<init>(BootStrapContext.java:82)
	at org.apache.drill.exec.server.Drillbit.<init>(Drillbit.java:171)
	at org.apache.drill.exec.server.Drillbit.<init>(Drillbit.java:135)
	at org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:133)
	at org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:67)
	at org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:67)
	at org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:138)
	at org.apache.drill.jdbc.Driver.connect(Driver.java:75)
	at sqlline.DatabaseConnection.connect(DatabaseConnection.java:135)
	at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:192)
	at sqlline.Commands.connect(Commands.java:1364)
	at sqlline.Commands.connect(Commands.java:1244)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
	at sqlline.SqlLine.dispatch(SqlLine.java:730)
	at sqlline.SqlLine.initArgs(SqlLine.java:410)
	at sqlline.SqlLine.begin(SqlLine.java:515)
	at sqlline.SqlLine.start(SqlLine.java:267)
	at sqlline.SqlLine.main(SqlLine.java:206)

	at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1847)
	at org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:710)
	at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:660)
	at org.apache.drill.exec.server.BootStrapContext.login(BootStrapContext.java:156)
	... 22 more
Caused by: javax.security.auth.login.LoginException: java.lang.NullPointerException: invalid
null input: name
	at com.sun.security.auth.UnixPrincipal.<init>(UnixPrincipal.java:71)
	at com.sun.security.auth.module.UnixLoginModule.login(UnixLoginModule.java:133)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
	at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
	at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
	at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
	at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
	at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:1926)
	at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1837)
	at org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:710)
	at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:660)
	at org.apache.drill.exec.server.BootStrapContext.login(BootStrapContext.java:156)
	at org.apache.drill.exec.server.BootStrapContext.<init>(BootStrapContext.java:82)
	at org.apache.drill.exec.server.Drillbit.<init>(Drillbit.java:171)
	at org.apache.drill.exec.server.Drillbit.<init>(Drillbit.java:135)
	at org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:133)
	at org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:67)
	at org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:67)
	at org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:138)
	at org.apache.drill.jdbc.Driver.connect(Driver.java:75)
	at sqlline.DatabaseConnection.connect(DatabaseConnection.java:135)
	at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:192)
	at sqlline.Commands.connect(Commands.java:1364)
	at sqlline.Commands.connect(Commands.java:1244)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
	at sqlline.SqlLine.dispatch(SqlLine.java:730)
	at sqlline.SqlLine.initArgs(SqlLine.java:410)
	at sqlline.SqlLine.begin(SqlLine.java:515)
	at sqlline.SqlLine.start(SqlLine.java:267)
	at sqlline.SqlLine.main(SqlLine.java:206)

	at javax.security.auth.login.LoginContext.invoke(LoginContext.java:856)
	at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
	at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
	at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
	at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
	at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:1926)
	at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1837)
	... 25 more
Apache Drill 1.17.0
"You told me to, Drill Sergeant!"

> On January 29, 2020 at 5:47 PM Ron Cecchini <roncecchini@comcast.net> wrote:
> 
> Sorry for the spam, but I think I figured it out.  
> 
> Thank you so much for your suggestions to build an image from an image.  I finally put
2 & 2 together and realized what you were saying and created the following Dockerfile.
 I then built and pushed the image into OpenShift - and it started up nicely.  
> 
> I haven't had a chance to test it yet, but I'm optimistic.
> 
> Thank you again.
> 
> ---
> 
> Dockerfile:
> 
> # Use the latest official release of Apache Drill
> FROM apache/drill:1.17.0
> 
> # Make /opt/drill readable by everyone due to OpenShift's security
> RUN chgrp -R 0 /opt/drill && chmod -R g=u /opt/drill
> 
> # Start Drill in embedded mode and connects to Sqlline
> ENTRYPOINT /opt/drill/bin/drill-embedded

Mime
View raw message