drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charles Givre <cgi...@gmail.com>
Subject Re: Initial Oracle Setup
Date Tue, 23 Jun 2020 19:21:58 GMT
Hi Chris, 
Not exactly... Drill has plugins which allow it to connect to external source system. They
are bundled with Drill so no need to install anything, but the code is structured that way.
 That particular code got some significant improvements so I'm wondering if simply updating
will resolve the issue.  The config looks correct to me. 



> On Jun 23, 2020, at 3:13 PM, Christopher Kane <christopher.kane@cardconnect.com.INVALID>
wrote:
> 
> I'm all for it honestly, I've never used Apache Drill before and am trying it out as
a POC for something I'm trying to do at work. So you're saying in addition to the driver required
by whichever RDBMS you're setting up, there is an Apache Drill JDBC plugin that works in combination
with the driver?
> 
> -----Original Message-----
> From: Charles Givre <cgivre@gmail.com>
> Sent: Tuesday, June 23, 2020 3:11 PM
> To: user@drill.apache.org
> Subject: Re: Initial Oracle Setup
> 
> Another qq:  Would you be willing to try Drill 1.18?  There were some significant changes
made to the JDBC plugin in 1.18 and I wonder if that might solve the issue.
> 
>> On Jun 23, 2020, at 3:05 PM, Christopher Kane <christopher.kane@cardconnect.com.INVALID>
wrote:
>> 
>> On both the directory of drill and on startup I am running Apache
>> Drill 1.17.0
>> 
>> -----Original Message-----
>> From: Charles Givre <cgivre@gmail.com <mailto:cgivre@gmail.com>>
>> Sent: Tuesday, June 23, 2020 3:03 PM
>> To: user <user@drill.apache.org <mailto:user@drill.apache.org>>
>> Subject: Re: Initial Oracle Setup
>> 
>> One more question.. what version of Drill are you using?
>> 
>> 
>>> On Jun 23, 2020, at 3:01 PM, Christopher Kane <christopher.kane@cardconnect.com.INVALID>
wrote:
>>> 
>>> Rafael
>>> -I did place it in the jars/3rdparty folder.
>>> 
>>> Storage Plugin Config:
>>> {
>>> "type": "jdbc",
>>> "enabled": true,
>>> "driver": "oracle.jdbc.driver.OracleDriver",
>>> "url": "jdbc:oracle:thin:username/password@chris-ora-scan:1521/chris_app"
>>> }
>>> (it's been modified to remove sensitive information)
>>> 
>>> Charles
>>> -I did turn Verbose on as you instructed, below is the full stacktrace. Now I
will say that the one DBA I was conversing with mentioned something about JRE, and I find
interesting in this stack trace that it seems to be a RunTime exception.
>>> -I'm not sure if I was aware they were at a version 10? This is the link I was
using to download the JDBC driver: https://urldefense.proofpoint.com/v2/url?u=https-3A__www.oracle.com_database_technologies_jdbc-2Ddrivers-2D12c-2Ddownloads.html&d=DwIFAg&c=rE3mhBYFJfJGqQ7WI0-DPw&r=BYRbCBlPlEiTy68EGn8lfllmVqBXbraSyDLC7XDswJU&m=8cPWOW4PY_sKPTUxECbL9XacM9DQCSnBbAJL9oQht0s&s=zddp1Im2FovgkwAQ3SyRaJzBS1FxeOgiwYdViTSn4H4&e=
<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.oracle.com_database_technologies_jdbc-2Ddrivers-2D12c-2Ddownloads.html&d=DwIFAg&c=rE3mhBYFJfJGqQ7WI0-DPw&r=BYRbCBlPlEiTy68EGn8lfllmVqBXbraSyDLC7XDswJU&m=8cPWOW4PY_sKPTUxECbL9XacM9DQCSnBbAJL9oQht0s&s=zddp1Im2FovgkwAQ3SyRaJzBS1FxeOgiwYdViTSn4H4&e=>
 and the only options I see available and relevant are either ojdbc7.jar or ojdbc6.jar. At
this point I've tried both but am willing to change my configuration while troubleshooting.
In addition the recommended driver for Oracle by Apache seems to be ojdbc7.12.1.0.2.jar referenced
from this page here: https://urldefense.proofpoint.com/v2/url?u=https-3A__drill.apache.org_docs_rdbms-2Dstorage-2Dplugin_&d=DwIFAg&c=rE3mhBYFJfJGqQ7WI0-DPw&r=BYRbCBlPlEiTy68EGn8lfllmVqBXbraSyDLC7XDswJU&m=8cPWOW4PY_sKPTUxECbL9XacM9DQCSnBbAJL9oQht0s&s=rzaAsfW2VGICMfzMAhjoT2vZqBRkti6X56uH4Uv1HU0&e=
<https://urldefense.proofpoint.com/v2/url?u=https-3A__drill.apache.org_docs_rdbms-2Dstorage-2Dplugin_&d=DwIFAg&c=rE3mhBYFJfJGqQ7WI0-DPw&r=BYRbCBlPlEiTy68EGn8lfllmVqBXbraSyDLC7XDswJU&m=8cPWOW4PY_sKPTUxECbL9XacM9DQCSnBbAJL9oQht0s&s=rzaAsfW2VGICMfzMAhjoT2vZqBRkti6X56uH4Uv1HU0&e=>
 and that is the one I am currently using, ojdbc7.jar from the 12.1.0.2 release of Oracle
Database.
>>> 
>>> 2020-06-23 14:52:37,017 [qtp377017996-50] ERROR
>>> o.a.d.e.server.rest.StorageResources - Unable to create/ update
>>> plugin: QA2 Oracle
>>> org.apache.drill.common.exceptions.ExecutionSetupException: Failure
>>> setting up new storage plugin configuration for config
>>> org.apache.drill.exec.store.jdbc.JdbcStorageConfig@7d9c9e53
>>> at
>>> org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StorageP
>>> l
>>> uginRegistryImpl.java:536) at
>>> org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(
>>> S
>>> toragePluginRegistryImpl.java:132)
>>> at
>>> org.apache.drill.exec.server.rest.PluginConfigWrapper.createOrUpdateI
>>> n
>>> Storage(PluginConfigWrapper.java:56)
>>> at
>>> org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePlug
>>> i
>>> nJSON(StorageResources.java:212) at
>>> org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePlug
>>> i
>>> n(StorageResources.java:233) at
>>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Nativ
>>> e
>>> Method) at
>>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Native
>>> M
>>> ethodAccessorImpl.java:62) at
>>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(De
>>> l
>>> egatingMethodAccessorImpl.java:43)
>>> at java.base/java.lang.reflect.Method.invoke(Method.java:564)
>>> at
>>> org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHa
>>> n
>>> dlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
>>> at
>>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethod
>>> D
>>> ispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
>>> at
>>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethod
>>> D
>>> ispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
>>> at
>>> org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatch
>>> e
>>> rProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvi
>>> d
>>> er.java:205) at
>>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethod
>>> D
>>> ispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
>>> at
>>> org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(Resour
>>> c
>>> eMethodInvoker.java:389) at
>>> org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(Resourc
>>> e
>>> MethodInvoker.java:347) at
>>> org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(Resourc
>>> e
>>> MethodInvoker.java:102) at
>>> org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:32
>>> 6
>>> ) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
>>> at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
>>> at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
>>> at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
>>> at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
>>> at
>>> org.glassfish.jersey.process.internal.RequestScope.runInScope(Request
>>> S
>>> cope.java:317) at
>>> org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:
>>> 3
>>> 05) at
>>> org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHand
>>> l
>>> er.java:1154) at
>>> org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.ja
>>> v
>>> a:473) at
>>> org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:4
>>> 2
>>> 7) at
>>> org.glassfish.jersey.servlet.ServletContainer.service(ServletContaine
>>> r
>>> .java:388) at
>>> org.glassfish.jersey.servlet.ServletContainer.service(ServletContaine
>>> r
>>> .java:341) at
>>> org.glassfish.jersey.servlet.ServletContainer.service(ServletContaine
>>> r
>>> .java:228) at
>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848
>>> )
>>> at
>>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
>>> H
>>> andler.java:1780) at
>>> org.apache.drill.exec.server.rest.CsrfTokenValidateFilter.doFilter(Cs
>>> r
>>> fTokenValidateFilter.java:55) at
>>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
>>> H
>>> andler.java:1767) at
>>> org.apache.drill.exec.server.rest.CsrfTokenInjectFilter.doFilter(Csrf
>>> T
>>> okenInjectFilter.java:54) at
>>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
>>> H
>>> andler.java:1767) at
>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:
>>> 583) at
>>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandl
>>> e
>>> r.java:224) at
>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandl
>>> e
>>> r.java:1180) at
>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:
>>> 5
>>> 13) at
>>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandle
>>> r
>>> .java:185) at
>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandle
>>> r
>>> .java:1112) at
>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j
>>> a
>>> va:141) at
>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.
>>> java:134) at org.eclipse.jetty.server.Server.handle(Server.java:539)
>>> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
>>> at
>>> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.jav
>>> a
>>> :251) at
>>> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(Abstra
>>> c
>>> tConnection.java:283) at
>>> org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
>>> at
>>> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoin
>>> t
>>> .java:93) at
>>> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeP
>>> r
>>> oduceConsume(ExecuteProduceConsume.java:303)
>>> at
>>> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceC
>>> o
>>> nsume(ExecuteProduceConsume.java:148)
>>> at
>>> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(Exec
>>> u
>>> teProduceConsume.java:136) at
>>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPoo
>>> l
>>> .java:671) at
>>> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.
>>> java:589) at java.base/java.lang.Thread.run(Thread.java:832)
>>> Caused by: java.lang.RuntimeException: Rule description
>>> 'JDBC_DREL_ConverterJDBC.QA2 Oracle' is not valid at
>>> org.apache.calcite.plan.RelOptRule.<init>(RelOptRule.java:104)
>>> at
>>> org.apache.calcite.rel.convert.ConverterRule.<init>(ConverterRule.jav
>>> a
>>> :78) at
>>> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin$JdbcDrelConverterR
>>> u
>>> le.<init>(JdbcStoragePlugin.java:224)
>>> at
>>> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin$DrillJdbcConventio
>>> n
>>> .<init>(JdbcStoragePlugin.java:119)
>>> at
>>> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.<init>(JdbcStorage
>>> P
>>> lugin.java:103) at
>>> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInsta
>>> n
>>> ce0(Native Method) at
>>> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInsta
>>> n
>>> ce(NativeConstructorAccessorImpl.java:62)
>>> at
>>> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newI
>>> n
>>> stance(DelegatingConstructorAccessorImpl.java:45)
>>> at
>>> java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constru
>>> c
>>> tor.java:500) at
>>> java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:
>>> 4
>>> 81) at
>>> org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StorageP
>>> l
>>> uginRegistryImpl.java:528)
>>> ... 56 common frames omitted
>>> 
>>> -----Original Message-----
>>> From: Charles Givre <cgivre@gmail.com>
>>> Sent: Tuesday, June 23, 2020 2:51 PM
>>> To: user@drill.apache.org
>>> Subject: Re: Initial Oracle Setup
>>> 
>>> One more thing...
>>> Did you try with the current Oracle JDBC driver?  I believe they are at version
10 now.
>>> Thanks,
>>> -- C
>>> 
>>>> On Jun 23, 2020, at 1:59 PM, Rafael Jaimes III <rafjaimes@gmail.com>
wrote:
>>>> 
>>>> Hi Chris,
>>>> 
>>>> You put the JDBC driver in the 3rd party folder?
>>>> 
>>>> You're right you might be onto something if you get the password
>>>> failed message. Can you post the exact text (JSON) of the storage
>>>> plugin config?
>>>> 
>>>> -Rafael
>>>> 
>>>> On Tue, Jun 23, 2020 at 1:56 PM Christopher Kane
>>>> <christopher.kane@cardconnect.com.invalid> wrote:
>>>>> 
>>>>> Hey Everyone,
>>>>> 
>>>>> I'm looking to setup and create a new Oracle Storage Plugin for my local
Apache Drill instance. I have determined that a correct oracle connection string with an incorrect
password results in the following message: Please retry: Error while creating / updating storage
: java.sql.SQLException: Cannot create PoolableConnectionFactory (ORA-01017: invalid username/password;
logon denied ). Now updating the connection string to use the correct password results in
the following message: Please retry: Error while creating / updating storage : Rule description
'JDBC_DREL_ConverterJDBC.QA2 Oracle' is not valid.
>>>>> 
>>>>> I have confirmed with our DBAs and through my own testing that my connection
string is correct. When attempting to google anything related to the message 'JDBC_DREL_Converter'
unfortunately I am not able to find anything useful. At this point I'm kind of grasping at
straws as to what my issue is at this point. In addition I also have the correct oracle thin
driver installed, which was the recommend one from the RDBMS page within the Apache Drill
documentation and I've also added this line to my drill-override.conf, drill.exec.sys.store.provider.local.path
= "ojdbc7.jar". But I'm assuming if any of that wasn't setup correctly, I wouldn't be able
to get an error message back from Oracle saying that my password was incorrect. Any help in
regards to this would be greatly appreciated.
>>>>> 
>>>>> Christopher Kane
>>>>> Software Test Engineer | Clover, CardConnect, FiServ
>>>>> ckane@clover.com<mailto:ckane@clover.com>,
>>>>> Christopher.Kane@cardconnect.com<mailto:Christopher.Kane@cardconnect.
>>>>> com>,
>>>>> Christopher.Kane2@firstdata.com<mailto:Christopher.Kane2@firstdata.
>>>>> c
>>>>> o
>>>>> m>,
>>>>> m>Christopher.Kane1@fiserv.com<mailto:Christopher.Kane1@fiserv.com>
>>>>> 
>>>>> THIS MESSAGE IS CONFIDENTIAL. This e-mail message and any attachments
are proprietary and confidential information protected from disclosure and intended only for
the use of the recipient(s) named above. If the reader of this message is not the intended
recipient, or an employee or agent responsible for delivering this message to the intended
recipient, you are hereby notified that any dissemination, distribution or copying of this
message or any attachments is strictly prohibited. If you have received this communication
in error, please notify CardConnect immediately by replying to this message and then delete
this message and any attachments from your computer.
>>> 
>>> THIS MESSAGE IS CONFIDENTIAL. This e-mail message and any attachments are proprietary
and confidential information protected from disclosure and intended only for the use of the
recipient(s) named above. If the reader of this message is not the intended recipient, or
an employee or agent responsible for delivering this message to the intended recipient, you
are hereby notified that any dissemination, distribution or copying of this message or any
attachments is strictly prohibited. If you have received this communication in error, please
notify CardConnect immediately by replying to this message and then delete this message and
any attachments from your computer.
>> 
>> THIS MESSAGE IS CONFIDENTIAL. This e-mail message and any attachments are proprietary
and confidential information protected from disclosure and intended only for the use of the
recipient(s) named above. If the reader of this message is not the intended recipient, or
an employee or agent responsible for delivering this message to the intended recipient, you
are hereby notified that any dissemination, distribution or copying of this message or any
attachments is strictly prohibited. If you have received this communication in error, please
notify CardConnect immediately by replying to this message and then delete this message and
any attachments from your computer.
> 
> THIS MESSAGE IS CONFIDENTIAL. This e-mail message and any attachments are proprietary
and confidential information protected from disclosure and intended only for the use of the
recipient(s) named above. If the reader of this message is not the intended recipient, or
an employee or agent responsible for delivering this message to the intended recipient, you
are hereby notified that any dissemination, distribution or copying of this message or any
attachments is strictly prohibited. If you have received this communication in error, please
notify CardConnect immediately by replying to this message and then delete this message and
any attachments from your computer.


Mime
View raw message