drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charles Givre <cgi...@gmail.com>
Subject Re: Initial Oracle Setup
Date Tue, 23 Jun 2020 20:03:44 GMT
Christopher, Rafael, 
Just for your guys knowledge... Drill uses a JDBC connection pool to manage JDBC connections
to external sources.  As part of Drill 1.18 [1], we updated the connection pool with HikariCP
from DBCP2 which apparently is faster and more reliable. [2]. However, it seems to be a known
issue that HikariCP doesn't work with Presto. [3]. Not sure why, but when I researched it,
it did appear that the HikariCP community was aware of this but wasn't in a big hurry to fix...
;-/
Glad we got it working!
-- C

[1]: https://github.com/apache/drill/pull/2027 <https://github.com/apache/drill/pull/2027>
[2]: https://beansroasted.wordpress.com/2017/07/29/connection-pool-analysis/ <https://beansroasted.wordpress.com/2017/07/29/connection-pool-analysis/>
[3]: https://github.com/brettwooldridge/HikariCP/issues/1424 <https://github.com/brettwooldridge/HikariCP/issues/1424>

> On Jun 23, 2020, at 3:43 PM, Rafael Jaimes III <rafjaimes@gmail.com> wrote:
> 
> Glad you got it working with the new version regardless!
> 
> On Tue, Jun 23, 2020 at 3:38 PM Christopher Kane
> <christopher.kane@cardconnect.com.invalid> wrote:
>> 
>> Wow, I gotta hand it to you, I don't know what I was expecting to happen but I got
a Success! So that's awesome, thank you so much for your help! I am going to quickly run a
test query on this data source now and see what happens.
>> 
>> -----Original Message-----
>> From: Christopher Kane <christopher.kane@cardconnect.com.INVALID>
>> Sent: Tuesday, June 23, 2020 3:31 PM
>> To: user@drill.apache.org
>> Subject: RE: Initial Oracle Setup
>> 
>> Yeah I am not the most familiar with building from source, so I'll check out your
pre-release, install it like I did 1.17.0 and see what happens!
>> 
>> -----Original Message-----
>> From: Charles Givre <cgivre@gmail.com>
>> Sent: Tuesday, June 23, 2020 3:23 PM
>> To: user <user@drill.apache.org>
>> Subject: Re: Initial Oracle Setup
>> 
>> It hasn't been released yet, so you'd have to d/l from github (https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_drill&d=DwIFAg&c=rE3mhBYFJfJGqQ7WI0-DPw&r=BYRbCBlPlEiTy68EGn8lfllmVqBXbraSyDLC7XDswJU&m=b6gHt-qP_soEwZSNloM5zPbmSepKW4FaAKoOwoxYlUk&s=C8v5-xTogFlPrAmfl-0zcAfmZZUP5ZIkcddMxV8aGcg&e=
 <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_drill&d=DwIFAg&c=rE3mhBYFJfJGqQ7WI0-DPw&r=BYRbCBlPlEiTy68EGn8lfllmVqBXbraSyDLC7XDswJU&m=b6gHt-qP_soEwZSNloM5zPbmSepKW4FaAKoOwoxYlUk&s=C8v5-xTogFlPrAmfl-0zcAfmZZUP5ZIkcddMxV8aGcg&e=
>) and build from source.  If you can't do that, I have a pre-release here: https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cgivre_drill_releases_tag_1.18.0-2DSNAPSHOT&d=DwIFAg&c=rE3mhBYFJfJGqQ7WI0-DPw&r=BYRbCBlPlEiTy68EGn8lfllmVqBXbraSyDLC7XDswJU&m=b6gHt-qP_soEwZSNloM5zPbmSepKW4FaAKoOwoxYlUk&s=3IE3kzUlrMTuhmPxkBggBYoK9fszW_xrb--95H-qXY0&e=
 <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cgivre_drill_releases_tag_1.18.0-2DSNAPSHOT&d=DwIFAg&c=rE3mhBYFJfJGqQ7WI0-DPw&r=BYRbCBlPlEiTy68EGn8lfllmVqBXbraSyDLC7XDswJU&m=b6gHt-qP_soEwZSNloM5zPbmSepKW4FaAKoOwoxYlUk&s=3IE3kzUlrMTuhmPxkBggBYoK9fszW_xrb--95H-qXY0&e=
>
>> 
>> 
>>> On Jun 23, 2020, at 3:21 PM, Christopher Kane <christopher.kane@cardconnect.com.INVALID>
wrote:
>>> 
>>> You might have to provide a link for Drill 1.18, I'm not seeing that available
on the downloads section of the site.
>>> 
>>> -----Original Message-----
>>> From: Christopher Kane <christopher.kane@cardconnect.com.INVALID>
>>> Sent: Tuesday, June 23, 2020 3:14 PM
>>> To: user@drill.apache.org
>>> Subject: RE: Initial Oracle Setup
>>> 
>>> I'm all for it honestly, I've never used Apache Drill before and am trying it
out as a POC for something I'm trying to do at work. So you're saying in addition to the driver
required by whichever RDBMS you're setting up, there is an Apache Drill JDBC plugin that works
in combination with the driver?
>>> 
>>> -----Original Message-----
>>> From: Charles Givre <cgivre@gmail.com>
>>> Sent: Tuesday, June 23, 2020 3:11 PM
>>> To: user@drill.apache.org
>>> Subject: Re: Initial Oracle Setup
>>> 
>>> Another qq:  Would you be willing to try Drill 1.18?  There were some significant
changes made to the JDBC plugin in 1.18 and I wonder if that might solve the issue.
>>> 
>>>> On Jun 23, 2020, at 3:05 PM, Christopher Kane <christopher.kane@cardconnect.com.INVALID>
wrote:
>>>> 
>>>> On both the directory of drill and on startup I am running Apache
>>>> Drill 1.17.0
>>>> 
>>>> -----Original Message-----
>>>> From: Charles Givre <cgivre@gmail.com <mailto:cgivre@gmail.com>>
>>>> Sent: Tuesday, June 23, 2020 3:03 PM
>>>> To: user <user@drill.apache.org <mailto:user@drill.apache.org>>
>>>> Subject: Re: Initial Oracle Setup
>>>> 
>>>> One more question.. what version of Drill are you using?
>>>> 
>>>> 
>>>>> On Jun 23, 2020, at 3:01 PM, Christopher Kane <christopher.kane@cardconnect.com.INVALID>
wrote:
>>>>> 
>>>>> Rafael
>>>>> -I did place it in the jars/3rdparty folder.
>>>>> 
>>>>> Storage Plugin Config:
>>>>> {
>>>>> "type": "jdbc",
>>>>> "enabled": true,
>>>>> "driver": "oracle.jdbc.driver.OracleDriver",
>>>>> "url": "jdbc:oracle:thin:username/password@chris-ora-scan:1521/chris_app"
>>>>> }
>>>>> (it's been modified to remove sensitive information)
>>>>> 
>>>>> Charles
>>>>> -I did turn Verbose on as you instructed, below is the full stacktrace.
Now I will say that the one DBA I was conversing with mentioned something about JRE, and I
find interesting in this stack trace that it seems to be a RunTime exception.
>>>>> -I'm not sure if I was aware they were at a version 10? This is the link
I was using to download the JDBC driver: https://urldefense.proofpoint.com/v2/url?u=https-3A__www.oracle.com_database_technologies_jdbc-2Ddrivers-2D12c-2Ddownloads.html&d=DwIFAg&c=rE3mhBYFJfJGqQ7WI0-DPw&r=BYRbCBlPlEiTy68EGn8lfllmVqBXbraSyDLC7XDswJU&m=8cPWOW4PY_sKPTUxECbL9XacM9DQCSnBbAJL9oQht0s&s=zddp1Im2FovgkwAQ3SyRaJzBS1FxeOgiwYdViTSn4H4&e=
<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.oracle.com_database_technologies_jdbc-2Ddrivers-2D12c-2Ddownloads.html&d=DwIFAg&c=rE3mhBYFJfJGqQ7WI0-DPw&r=BYRbCBlPlEiTy68EGn8lfllmVqBXbraSyDLC7XDswJU&m=8cPWOW4PY_sKPTUxECbL9XacM9DQCSnBbAJL9oQht0s&s=zddp1Im2FovgkwAQ3SyRaJzBS1FxeOgiwYdViTSn4H4&e=>
 and the only options I see available and relevant are either ojdbc7.jar or ojdbc6.jar. At
this point I've tried both but am willing to change my configuration while troubleshooting.
In addition the recommended driver for Oracle by Apache seems to be ojdbc7.12.1.0.2.jar referenced
from this page here: https://urldefense.proofpoint.com/v2/url?u=https-3A__drill.apache.org_docs_rdbms-2Dstorage-2Dplugin_&d=DwIFAg&c=rE3mhBYFJfJGqQ7WI0-DPw&r=BYRbCBlPlEiTy68EGn8lfllmVqBXbraSyDLC7XDswJU&m=8cPWOW4PY_sKPTUxECbL9XacM9DQCSnBbAJL9oQht0s&s=rzaAsfW2VGICMfzMAhjoT2vZqBRkti6X56uH4Uv1HU0&e=
<https://urldefense.proofpoint.com/v2/url?u=https-3A__drill.apache.org_docs_rdbms-2Dstorage-2Dplugin_&d=DwIFAg&c=rE3mhBYFJfJGqQ7WI0-DPw&r=BYRbCBlPlEiTy68EGn8lfllmVqBXbraSyDLC7XDswJU&m=8cPWOW4PY_sKPTUxECbL9XacM9DQCSnBbAJL9oQht0s&s=rzaAsfW2VGICMfzMAhjoT2vZqBRkti6X56uH4Uv1HU0&e=>
 and that is the one I am currently using, ojdbc7.jar from the 12.1.0.2 release of Oracle
Database.
>>>>> 
>>>>> 2020-06-23 14:52:37,017 [qtp377017996-50] ERROR
>>>>> o.a.d.e.server.rest.StorageResources - Unable to create/ update
>>>>> plugin: QA2 Oracle
>>>>> org.apache.drill.common.exceptions.ExecutionSetupException: Failure
>>>>> setting up new storage plugin configuration for config
>>>>> org.apache.drill.exec.store.jdbc.JdbcStorageConfig@7d9c9e53
>>>>> at
>>>>> org.apache.drill.exec.store.StoragePluginRegistryImpl.create(Storage
>>>>> P
>>>>> l
>>>>> uginRegistryImpl.java:536) at
>>>>> org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate
>>>>> (
>>>>> S
>>>>> toragePluginRegistryImpl.java:132)
>>>>> at
>>>>> org.apache.drill.exec.server.rest.PluginConfigWrapper.createOrUpdate
>>>>> I
>>>>> n
>>>>> Storage(PluginConfigWrapper.java:56)
>>>>> at
>>>>> org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePlu
>>>>> g
>>>>> i
>>>>> nJSON(StorageResources.java:212) at
>>>>> org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePlu
>>>>> g
>>>>> i
>>>>> n(StorageResources.java:233) at
>>>>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Nati
>>>>> v
>>>>> e
>>>>> Method) at
>>>>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Nativ
>>>>> e
>>>>> M
>>>>> ethodAccessorImpl.java:62) at
>>>>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(D
>>>>> e
>>>>> l
>>>>> egatingMethodAccessorImpl.java:43)
>>>>> at java.base/java.lang.reflect.Method.invoke(Method.java:564)
>>>>> at
>>>>> org.glassfish.jersey.server.model.internal.ResourceMethodInvocationH
>>>>> a
>>>>> n
>>>>> dlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
>>>>> at
>>>>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMetho
>>>>> d
>>>>> D
>>>>> ispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
>>>>> at
>>>>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMetho
>>>>> d
>>>>> D
>>>>> ispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
>>>>> at
>>>>> org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatc
>>>>> h
>>>>> e
>>>>> rProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProv
>>>>> i
>>>>> d
>>>>> er.java:205) at
>>>>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMetho
>>>>> d
>>>>> D
>>>>> ispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
>>>>> at
>>>>> org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(Resou
>>>>> r
>>>>> c
>>>>> eMethodInvoker.java:389) at
>>>>> org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(Resour
>>>>> c
>>>>> e
>>>>> MethodInvoker.java:347) at
>>>>> org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(Resour
>>>>> c
>>>>> e
>>>>> MethodInvoker.java:102) at
>>>>> org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:3
>>>>> 2
>>>>> 6
>>>>> ) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
>>>>> at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
>>>>> at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
>>>>> at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
>>>>> at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
>>>>> at
>>>>> org.glassfish.jersey.process.internal.RequestScope.runInScope(Reques
>>>>> t
>>>>> S
>>>>> cope.java:317) at
>>>>> org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:
>>>>> 3
>>>>> 05) at
>>>>> org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHan
>>>>> d
>>>>> l
>>>>> er.java:1154) at
>>>>> org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.j
>>>>> a
>>>>> v
>>>>> a:473) at
>>>>> org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:
>>>>> 4
>>>>> 2
>>>>> 7) at
>>>>> org.glassfish.jersey.servlet.ServletContainer.service(ServletContain
>>>>> e
>>>>> r
>>>>> .java:388) at
>>>>> org.glassfish.jersey.servlet.ServletContainer.service(ServletContain
>>>>> e
>>>>> r
>>>>> .java:341) at
>>>>> org.glassfish.jersey.servlet.ServletContainer.service(ServletContain
>>>>> e
>>>>> r
>>>>> .java:228) at
>>>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:84
>>>>> 8
>>>>> )
>>>>> at
>>>>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servle
>>>>> t
>>>>> H
>>>>> andler.java:1780) at
>>>>> org.apache.drill.exec.server.rest.CsrfTokenValidateFilter.doFilter(C
>>>>> s
>>>>> r
>>>>> fTokenValidateFilter.java:55) at
>>>>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servle
>>>>> t
>>>>> H
>>>>> andler.java:1767) at
>>>>> org.apache.drill.exec.server.rest.CsrfTokenInjectFilter.doFilter(Csr
>>>>> f
>>>>> T
>>>>> okenInjectFilter.java:54) at
>>>>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servle
>>>>> t
>>>>> H
>>>>> andler.java:1767) at
>>>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:
>>>>> 583) at
>>>>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHand
>>>>> l
>>>>> e
>>>>> r.java:224) at
>>>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHand
>>>>> l
>>>>> e
>>>>> r.java:1180) at
>>>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:
>>>>> 5
>>>>> 13) at
>>>>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandl
>>>>> e
>>>>> r
>>>>> .java:185) at
>>>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandl
>>>>> e
>>>>> r
>>>>> .java:1112) at
>>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.
>>>>> j
>>>>> a
>>>>> va:141) at
>>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.
>>>>> java:134) at org.eclipse.jetty.server.Server.handle(Server.java:539)
>>>>> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
>>>>> at
>>>>> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.ja
>>>>> v
>>>>> a
>>>>> :251) at
>>>>> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(Abstr
>>>>> a
>>>>> c
>>>>> tConnection.java:283) at
>>>>> org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
>>>>> at
>>>>> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoi
>>>>> n
>>>>> t
>>>>> .java:93) at
>>>>> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute
>>>>> P
>>>>> r
>>>>> oduceConsume(ExecuteProduceConsume.java:303)
>>>>> at
>>>>> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produce
>>>>> C
>>>>> o
>>>>> nsume(ExecuteProduceConsume.java:148)
>>>>> at
>>>>> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(Exe
>>>>> c
>>>>> u
>>>>> teProduceConsume.java:136) at
>>>>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPo
>>>>> o
>>>>> l
>>>>> .java:671) at
>>>>> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.
>>>>> java:589) at java.base/java.lang.Thread.run(Thread.java:832)
>>>>> Caused by: java.lang.RuntimeException: Rule description
>>>>> 'JDBC_DREL_ConverterJDBC.QA2 Oracle' is not valid at
>>>>> org.apache.calcite.plan.RelOptRule.<init>(RelOptRule.java:104)
>>>>> at
>>>>> org.apache.calcite.rel.convert.ConverterRule.<init>(ConverterRule.ja
>>>>> v
>>>>> a
>>>>> :78) at
>>>>> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin$JdbcDrelConverter
>>>>> R
>>>>> u
>>>>> le.<init>(JdbcStoragePlugin.java:224)
>>>>> at
>>>>> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin$DrillJdbcConventi
>>>>> o
>>>>> n
>>>>> .<init>(JdbcStoragePlugin.java:119)
>>>>> at
>>>>> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.<init>(JdbcStorag
>>>>> e
>>>>> P
>>>>> lugin.java:103) at
>>>>> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInst
>>>>> a
>>>>> n
>>>>> ce0(Native Method) at
>>>>> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInst
>>>>> a
>>>>> n
>>>>> ce(NativeConstructorAccessorImpl.java:62)
>>>>> at
>>>>> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.new
>>>>> I
>>>>> n
>>>>> stance(DelegatingConstructorAccessorImpl.java:45)
>>>>> at
>>>>> java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constr
>>>>> u
>>>>> c
>>>>> tor.java:500) at
>>>>> java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:
>>>>> 4
>>>>> 81) at
>>>>> org.apache.drill.exec.store.StoragePluginRegistryImpl.create(Storage
>>>>> P
>>>>> l
>>>>> uginRegistryImpl.java:528)
>>>>> ... 56 common frames omitted
>>>>> 
>>>>> -----Original Message-----
>>>>> From: Charles Givre <cgivre@gmail.com>
>>>>> Sent: Tuesday, June 23, 2020 2:51 PM
>>>>> To: user@drill.apache.org
>>>>> Subject: Re: Initial Oracle Setup
>>>>> 
>>>>> One more thing...
>>>>> Did you try with the current Oracle JDBC driver?  I believe they are
at version 10 now.
>>>>> Thanks,
>>>>> -- C
>>>>> 
>>>>>> On Jun 23, 2020, at 1:59 PM, Rafael Jaimes III <rafjaimes@gmail.com>
wrote:
>>>>>> 
>>>>>> Hi Chris,
>>>>>> 
>>>>>> You put the JDBC driver in the 3rd party folder?
>>>>>> 
>>>>>> You're right you might be onto something if you get the password
>>>>>> failed message. Can you post the exact text (JSON) of the storage
>>>>>> plugin config?
>>>>>> 
>>>>>> -Rafael
>>>>>> 
>>>>>> On Tue, Jun 23, 2020 at 1:56 PM Christopher Kane
>>>>>> <christopher.kane@cardconnect.com.invalid> wrote:
>>>>>>> 
>>>>>>> Hey Everyone,
>>>>>>> 
>>>>>>> I'm looking to setup and create a new Oracle Storage Plugin for
my local Apache Drill instance. I have determined that a correct oracle connection string
with an incorrect password results in the following message: Please retry: Error while creating
/ updating storage : java.sql.SQLException: Cannot create PoolableConnectionFactory (ORA-01017:
invalid username/password; logon denied ). Now updating the connection string to use the correct
password results in the following message: Please retry: Error while creating / updating storage
: Rule description 'JDBC_DREL_ConverterJDBC.QA2 Oracle' is not valid.
>>>>>>> 
>>>>>>> I have confirmed with our DBAs and through my own testing that
my connection string is correct. When attempting to google anything related to the message
'JDBC_DREL_Converter' unfortunately I am not able to find anything useful. At this point I'm
kind of grasping at straws as to what my issue is at this point. In addition I also have the
correct oracle thin driver installed, which was the recommend one from the RDBMS page within
the Apache Drill documentation and I've also added this line to my drill-override.conf, drill.exec.sys.store.provider.local.path
= "ojdbc7.jar". But I'm assuming if any of that wasn't setup correctly, I wouldn't be able
to get an error message back from Oracle saying that my password was incorrect. Any help in
regards to this would be greatly appreciated.
>>>>>>> 
>>>>>>> Christopher Kane
>>>>>>> Software Test Engineer | Clover, CardConnect, FiServ
>>>>>>> ckane@clover.com<mailto:ckane@clover.com>,
>>>>>>> Christopher.Kane@cardconnect.com<mailto:Christopher.Kane@cardconnect.
>>>>>>> com>,
>>>>>>> Christopher.Kane2@firstdata.com<mailto:Christopher.Kane2@firstdata.
>>>>>>> c
>>>>>>> o
>>>>>>> m>,
>>>>>>> m>Christopher.Kane1@fiserv.com<mailto:Christopher.Kane1@fiserv.com
>>>>>>> m>>
>>>>>>> 
>>>>>>> THIS MESSAGE IS CONFIDENTIAL. This e-mail message and any attachments
are proprietary and confidential information protected from disclosure and intended only for
the use of the recipient(s) named above. If the reader of this message is not the intended
recipient, or an employee or agent responsible for delivering this message to the intended
recipient, you are hereby notified that any dissemination, distribution or copying of this
message or any attachments is strictly prohibited. If you have received this communication
in error, please notify CardConnect immediately by replying to this message and then delete
this message and any attachments from your computer.
>>>>> 
>>>>> THIS MESSAGE IS CONFIDENTIAL. This e-mail message and any attachments
are proprietary and confidential information protected from disclosure and intended only for
the use of the recipient(s) named above. If the reader of this message is not the intended
recipient, or an employee or agent responsible for delivering this message to the intended
recipient, you are hereby notified that any dissemination, distribution or copying of this
message or any attachments is strictly prohibited. If you have received this communication
in error, please notify CardConnect immediately by replying to this message and then delete
this message and any attachments from your computer.
>>>> 
>>>> THIS MESSAGE IS CONFIDENTIAL. This e-mail message and any attachments are
proprietary and confidential information protected from disclosure and intended only for the
use of the recipient(s) named above. If the reader of this message is not the intended recipient,
or an employee or agent responsible for delivering this message to the intended recipient,
you are hereby notified that any dissemination, distribution or copying of this message or
any attachments is strictly prohibited. If you have received this communication in error,
please notify CardConnect immediately by replying to this message and then delete this message
and any attachments from your computer.
>>> 
>>> THIS MESSAGE IS CONFIDENTIAL. This e-mail message and any attachments are proprietary
and confidential information protected from disclosure and intended only for the use of the
recipient(s) named above. If the reader of this message is not the intended recipient, or
an employee or agent responsible for delivering this message to the intended recipient, you
are hereby notified that any dissemination, distribution or copying of this message or any
attachments is strictly prohibited. If you have received this communication in error, please
notify CardConnect immediately by replying to this message and then delete this message and
any attachments from your computer.
>>> THIS MESSAGE IS CONFIDENTIAL. This e-mail message and any attachments are proprietary
and confidential information protected from disclosure and intended only for the use of the
recipient(s) named above. If the reader of this message is not the intended recipient, or
an employee or agent responsible for delivering this message to the intended recipient, you
are hereby notified that any dissemination, distribution or copying of this message or any
attachments is strictly prohibited. If you have received this communication in error, please
notify CardConnect immediately by replying to this message and then delete this message and
any attachments from your computer.
>> 
>> THIS MESSAGE IS CONFIDENTIAL. This e-mail message and any attachments are proprietary
and confidential information protected from disclosure and intended only for the use of the
recipient(s) named above. If the reader of this message is not the intended recipient, or
an employee or agent responsible for delivering this message to the intended recipient, you
are hereby notified that any dissemination, distribution or copying of this message or any
attachments is strictly prohibited. If you have received this communication in error, please
notify CardConnect immediately by replying to this message and then delete this message and
any attachments from your computer.
>> THIS MESSAGE IS CONFIDENTIAL. This e-mail message and any attachments are proprietary
and confidential information protected from disclosure and intended only for the use of the
recipient(s) named above. If the reader of this message is not the intended recipient, or
an employee or agent responsible for delivering this message to the intended recipient, you
are hereby notified that any dissemination, distribution or copying of this message or any
attachments is strictly prohibited. If you have received this communication in error, please
notify CardConnect immediately by replying to this message and then delete this message and
any attachments from your computer.


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message