sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From corbacho anthony <acorba...@nflabs.com>
Subject Re: Sqoop and OSGI
Date Fri, 21 Jun 2013 05:53:05 GMT
Abe,

I see your point.
I tried with this config, but I get still the same error.
[com.cloudera.sqoop.manager.ConnMananger (not found)].

Anthony


On Fri, Jun 21, 2013 at 2:41 PM, Abraham Elmahrek <abe@cloudera.com> wrote:

> Anthony,
>
> You've set that configuration incorrectly. It should be conf.set(
> "sqoop.connection.factories", "com.cloudera.sqoop.manager.ConnMananger");.
> Sqoop instantiates a Configuration object and adds resources to it.
>
> -Abe
>
>
> On Thu, Jun 20, 2013 at 6:44 PM, corbacho anthony <acorbacho@nflabs.com>wrote:
>
>> Hello Abe.
>>
>> The think is, I didn`t install sqoop, I use it via pom on my java code.
>> I added on my code this:
>>
>> Configuration conf = new HdfsConfiguration();
>> conf.set("sqoop.connection.factories",
>> "com.cloudera.sqoop.apache.ConnMananger");
>> [....]
>>
>> return Sqoop.runTool(str, conf);
>> But still the same error.
>>
>>
>> On Thu, Jun 20, 2013 at 12:50 PM, Abraham Elmahrek <abe@cloudera.com>wrote:
>>
>>> Anthony,
>>>
>>> Weird, that should be com.cloudera.sqoop.manager.ConnMananger. I
>>> suspect there's a problem with your sqoop-site.xml (if its even being
>>> parsed at all) or classpath. The sqoop startup script sets
>>> SQOOP_HOME_DIR, SQOOP_CONF_DIR, HADOOP_CLASSPATH, and a few other
>>> environment variables. I suspect there's something required there.
>>>
>>> -Abe
>>>
>>>
>>> On Wed, Jun 19, 2013 at 6:53 PM, corbacho anthony <acorbacho@nflabs.com>wrote:
>>>
>>>> I tried an same issues.
>>>> The problem is Sqoop cannot
>>>> find com.cloudera.sqoop.apache.ConnMananger...
>>>>
>>>>
>>>> On Thu, Jun 20, 2013 at 10:33 AM, corbacho anthony <
>>>> acorbacho@nflabs.com> wrote:
>>>>
>>>>> I used
>>>>> https://repository.cloudera.com/artifactory/cloudera-repos/com/cloudera/sqoop/sqoop/
>>>>>
>>>>> I will try with sqoop 1.4.3.
>>>>> Thank you Jarec.
>>>>>
>>>>> Anthony
>>>>>
>>>>>
>>>>> On Thu, Jun 20, 2013 at 10:28 AM, Jarek Jarcec Cecho <
>>>>> jarcec@apache.org> wrote:
>>>>>
>>>>>> Hi Anthony,
>>>>>> what maven repository are you using?
>>>>>>
>>>>>> Cloudera maven repository [1] seems to be containing all released
>>>>>> versions (instructions how to use this repository are in installation
guide
>>>>>> [3]). Also thanks to SQOOP-1007 we do have vanila Sqoop artifacts
available
>>>>>> in main maven repository [2].
>>>>>>
>>>>>> Jarcec
>>>>>>
>>>>>> Links:
>>>>>> 1:
>>>>>> https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/sqoop/sqoop/
>>>>>> 2: http://search.maven.org/#browse%7C585117616
>>>>>> 3:
>>>>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/cdh4ig_topic_31.html
>>>>>>
>>>>>> On Thu, Jun 20, 2013 at 10:17:35AM +0900, corbacho anthony wrote:
>>>>>> > hello Jarec and Abe.
>>>>>> >
>>>>>> > Abe: the debugging mode show nothing :(
>>>>>> > Jarec: In cloudera repo I can only find Sqoop 1.4.0.
>>>>>> >
>>>>>> > There is the full stack (in case):
>>>>>> > 2013-06-20 10:10:29,022 | ERROR | qtp1742391956-59 | ConnFactory
|
>>>>>> Could
>>>>>> > not load ManagerFactory com.cloudera.sqoop.apache.ConnMananger
(not
>>>>>> found)
>>>>>> > 2013-06-20 10:10:29,023 | ERROR | qtp1742391956-59 | BaseSqoopTool
>>>>>> | Got
>>>>>> > error creating database manager: java.io.IOException: No manager
for
>>>>>> > connect string: jdbc:mysql://xxx:0000/x
>>>>>> > at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:119)
>>>>>> > at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:202)
>>>>>> > at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:83)
>>>>>> > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:441)
>>>>>> > at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>>>>>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>>>> > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>>>>>> > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>>>>>> > at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:49)
>>>>>> > at
>>>>>> com.nflabs.com.peloton.imprt.db.SqoopImport.tested(SqoopImport.java:131)
>>>>>> > at
>>>>>> com.nflabs.peloton.imprt.impl.ImprtSvcImpl.test(ImprtSvcImpl.java:191)
>>>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>> > at
>>>>>> >
>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>> > at
>>>>>> >
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> > at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> > at
>>>>>> org.apache.aries.proxy.impl.ProxyHandler$1.invoke(ProxyHandler.java:50)
>>>>>> > at
>>>>>> org.apache.aries.proxy.impl.DefaultWrapper.invoke(DefaultWrapper.java:31)
>>>>>> > at
>>>>>> org.apache.aries.proxy.impl.ProxyHandler.invoke(ProxyHandler.java:78)
>>>>>> > at com.sun.proxy.$Proxy76.test(Unknown Source)
>>>>>> > at
>>>>>> >
>>>>>> com.nflabs.peloton.rs.imprt.ImprtService.BrowseDirectory1(ImprtService.java:478)
>>>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>> > at
>>>>>> >
>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>> > at
>>>>>> >
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> > at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:173)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:89)
>>>>>> > at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:165)
>>>>>> > at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:92)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58)
>>>>>> > at
>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>>>> > at
>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>> > at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.workqueue.SynchronousExecutor.execute(SynchronousExecutor.java:37)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:106)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:207)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:209)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:152)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:114)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:185)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:113)
>>>>>> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:693)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:164)
>>>>>> > at
>>>>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:538)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:478)
>>>>>> > at
>>>>>> >
>>>>>> org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:70)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:480)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:937)
>>>>>> > at
>>>>>> >
>>>>>> org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:116)
>>>>>> > at
>>>>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:871)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>>>>>> > at
>>>>>> >
>>>>>> org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:72)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
>>>>>> > at org.eclipse.jetty.server.Server.handle(Server.java:346)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:589)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1048)
>>>>>> > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:601)
>>>>>> > at
>>>>>> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:214)
>>>>>> > at
>>>>>> org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:411)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:535)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)
>>>>>> > at java.lang.Thread.run(Thread.java:680)
>>>>>> >
>>>>>> > Thanks
>>>>>> > Anthony
>>>>>> >
>>>>>> >
>>>>>> > On Thu, Jun 20, 2013 at 2:52 AM, Abraham Elmahrek <abe@cloudera.com>
>>>>>> wrote:
>>>>>> >
>>>>>> > > Anthony,
>>>>>> > >
>>>>>> > > You could also try enabling debug logging to see what managers
>>>>>> are being
>>>>>> > > loaded (if any). This can normally be done by adding "--verbose"
>>>>>> to your
>>>>>> > > list of arguments. I'm not sure how it will work with your
>>>>>> intended use
>>>>>> > > case.
>>>>>> > >
>>>>>> > > -Abe
>>>>>> > >
>>>>>> > >
>>>>>> > > On Wed, Jun 19, 2013 at 7:57 AM, Jarek Jarcec Cecho <
>>>>>> jarcec@apache.org>wrote:
>>>>>> > >
>>>>>> > >> Hi Anthony,
>>>>>> > >> I'm glad to see your interested in Sqoop project! I
have to
>>>>>> mention that
>>>>>> > >> whereas the direct usage of the Sqoop classes might
work, it's
>>>>>> not
>>>>>> > >> officially supported API and you might hit some issues
along the
>>>>>> way.
>>>>>> > >>
>>>>>> > >> I would strongly suggest to upgrade to latest Sqoop
version as
>>>>>> the 1.4.0
>>>>>> > >> is quite old. In most cases using the --driver parameter
is not
>>>>>> needed as
>>>>>> > >> it will instruct Sqoop to use the Generic JDBC Driver
instead of
>>>>>> > >> specialized MySQL connector, I would suggest to drop
this
>>>>>> parameter
>>>>>> > >> entirely.
>>>>>> > >>
>>>>>> > >> Jarcec
>>>>>> > >>
>>>>>> > >> On Wed, Jun 19, 2013 at 03:25:51PM +0900, corbacho
anthony wrote:
>>>>>> > >> > Hello,
>>>>>> > >> >
>>>>>> > >> > I am trying to use Sqoop in an osgi pluging but
it wont work
>>>>>> and I dont
>>>>>> > >> > know why.
>>>>>> > >> > I keep getting the following error:
>>>>>> > >> > "2013-06-19 13:53:27,556 | ERROR | qtp122715895-276
|
>>>>>> BaseSqoopTool  |
>>>>>> > >> ?  ?
>>>>>> > >> > |  -  -  | Got error creating database manager:
>>>>>> java.io.IOException: No
>>>>>> > >> > manager for connect string: jdbc:mysql://...."
>>>>>> > >> >
>>>>>> > >> > I have include sqoop in my pom (1.4.0-cdh4b1)
and I also have
>>>>>> my mysql
>>>>>> > >> > driver loader (i can use in the code).
>>>>>> > >> >
>>>>>> > >> > the part who throw an exception is:
>>>>>> > >> >
>>>>>> > >> > String[] str = {
>>>>>> > >> > "import",
>>>>>> > >> > "--driver", "com.mysql.jdbc.Driver",
>>>>>> > >> > "--connect", "jdbc:mysql://___MY_DB___",
>>>>>> > >> > "--hadoop-home", __HOME__,
>>>>>> > >> > "--query", "Select [.....] AND $CONDITIONS",
>>>>>> > >> > "--split-by", "id",
>>>>>> > >> > "--target-dir", "hdfs://__SERVER__/__FOLDER__",
>>>>>> > >> > "--username", "foo",
>>>>>> > >> > "--password", "barou"};
>>>>>> > >> >
>>>>>> > >> > return Sqoop.runTool(str);
>>>>>> > >> >
>>>>>> > >> > Thanks,
>>>>>> > >> > Anthony
>>>>>> > >>
>>>>>> > >
>>>>>> > >
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message