sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Robson <David.Rob...@software.dell.com>
Subject RE: Sqoop with Oozie
Date Thu, 18 Sep 2014 23:50:11 GMT
Are you able to upgrade to Sqoop 1.4.5? As Sqoop 1.4.5 now includes OraOop – so you don’t
need to manually install it – would make it a lot easier I believe.

The error you are getting is because you are trying to use the connection factory as the manager
factory – probably somewhere in the config it has the wrong class name. Anyway if you can
upgrade (or are already using 1.4.5) then you can un-install OraOop and just add the “—direct”
option to your Sqoop job and it will use OraOop.

From: Chalcy [mailto:chalcy@gmail.com]
Sent: Friday, 19 September 2014 7:30 AM
To: user@sqoop.apache.org
Subject: Re: Sqoop with Oozie

The oraoop jar needs to be in hdfs for oozie to run I believe.  Please copy the oraoop jar
to hdfs and give that path to oozie conf.


On Thu, Sep 18, 2014 at 2:32 PM, Felipe Naberezny Lopes <felipe.naberezny@b2wdigital.com<mailto:felipe.naberezny@b2wdigital.com>>
wrote:
Without Oozie the plugin works and the Oraoop jar is in the Oozie share lib.

I'm not using a Cloudera distribution for Hadoop.

Maybe that is the problem. I don't know anything else to check.

-----Mensagem original-----
De: Gwen Shapira [mailto:gshapira@cloudera.com<mailto:gshapira@cloudera.com>]
Enviada em: quinta-feira, 18 de setembro de 2014 15:25
Para: user@sqoop.apache.org<mailto:user@sqoop.apache.org>
Assunto: Re: Sqoop with Oozie

Also, does everything work without Oozie?

On Thu, Sep 18, 2014 at 11:24 AM, Gwen Shapira <gshapira@cloudera.com<mailto:gshapira@cloudera.com>>
wrote:
> Do you have the Oraoop jar in Oozie's share lib path?
>
> On Thu, Sep 18, 2014 at 10:46 AM, Felipe Naberezny Lopes
> <felipe.naberezny@b2wdigital.com<mailto:felipe.naberezny@b2wdigital.com>>
wrote:
>> Gwen,
>>
>> It's throwing the same exception:
>>
>> 4569 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool  - Got error
>> creating database manager: java.lang.ClassCastException:
>> com.quest.oraoop.OracleConnectionFactory cannot be cast to
>> com.cloudera.sqoop.manager.ManagerFactory
>>
>> -----Mensagem original-----
>> De: Gwen Shapira [mailto:gshapira@cloudera.com<mailto:gshapira@cloudera.com>]
Enviada em:
>> quinta-feira, 18 de setembro de 2014 14:32
>> Para: user@sqoop.apache.org<mailto:user@sqoop.apache.org>
>> Assunto: Re: Sqoop with Oozie
>>
>> I believe you need to add this:
>>
>>  import -D
>> sqoop.connection.factories=com.cloudera.sqoop.manager.DefaultManagerF
>> actory,com.quest.oraoop.OraOopManagerFactory
>>
>> On Thu, Sep 18, 2014 at 10:30 AM, Felipe Naberezny Lopes <felipe.naberezny@b2wdigital.com<mailto:felipe.naberezny@b2wdigital.com>>
wrote:
>>> The plugin is QuestDataConnector for Oracle and Hadoop. The plugin
>>> is a connection manager between Oracle and Hadoop.
>>>
>>>
>>>
>>> When I install it in my Sqoop Client Machine, it stays in
>>> SQOOP_HOME/conf/managers.d
>>>
>>>
>>>
>>> I’m trying to make Oozie use the plugin.
>>>
>>>
>>>
>>> The jars are in my Hadoop cluster.
>>>
>>>
>>>
>>> I set the property “sqoop.connection.factories “ with the path to
>>> the jars of plugin.
>>>
>>>
>>>
>>> Apparently, Oozie is using that configuration, but it’s throwing the
>>> given
>>> error:
>>>
>>>
>>>
>>> “ 5542 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool  - Got error
>>> creating database manager: java.lang.ClassCastException:
>>> com.quest.oraoop.OracleConnectionFactory cannot be cast to
>>> com.cloudera.sqoop.manager.ManagerFactory “
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> De: pratik khadloya [mailto:tispratik@gmail.com<mailto:tispratik@gmail.com>]
Enviada em:
>>> quinta-feira, 18 de setembro de 2014 14:15
>>>
>>>
>>> Para: user@sqoop.apache.org<mailto:user@sqoop.apache.org>
>>> Assunto: Re: Sqoop with Oozie
>>>
>>>
>>>
>>> Can you share the plugin name and what error (with stack trace) you
>>> are getting?
>>>
>>>
>>>
>>> ~Pratik
>>>
>>>
>>>
>>> On Thu, Sep 18, 2014 at 10:04 AM, Felipe Naberezny Lopes
>>> <felipe.naberezny@b2wdigital.com<mailto:felipe.naberezny@b2wdigital.com>>
wrote:
>>>
>>> I'm connecting Sqoop with Oracle.
>>>
>>> There is a plugin made by Toad that improve the performance of data import.
>>>
>>> I'm trying to use it, but I don't know how to make Oozie use that plugin.
>>>
>>> You know something that solves this?
>>>
>>> -----Mensagem original-----
>>> De: Gwen Shapira [mailto:gshapira@cloudera.com<mailto:gshapira@cloudera.com>]
Enviada em:
>>> quinta-feira, 18 de setembro de 2014 13:55
>>> Para: user@sqoop.apache.org<mailto:user@sqoop.apache.org>
>>> Assunto: Re: Sqoop with Oozie
>>>
>>>
>>> We typically don't configure SQOOP_CONF_DIR for Oozie. Rather, we
>>> make sure the list of parameters we include in the Sqoop action
>>> includes all the relevant configurations.
>>>
>>> Gwen
>>>
>>> 2014-09-18 7:18 GMT-07:00 Felipe Naberezny Lopes
>>> <felipe.naberezny@b2wdigital.com<mailto:felipe.naberezny@b2wdigital.com>>:
>>>> Hi,
>>>>
>>>>
>>>>
>>>> How do I set SQOOP_CONF_DIR using Oozie?
>>>>
>>>>
>>>>
>>>> Any idea?
>>>>
>>>>
>>>>
>>>> Tks
>>>>
>>>>
>>>>
>>>> Felipe Naberezny
>>>>
>>>> B2W Digital / B2W Labs
>>>>
>>>> +55 11 3454-3021<tel:%2B55%2011%203454-3021> / +55 11 9 8561-8525<tel:%2B55%2011%209%208561-8525>
>>>>
>>>> felipe.naberezny@b2wdigital.com<mailto:felipe.naberezny@b2wdigital.com>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> ________________________________
>>>>
>>>> Esta mensagem pode conter informações confidenciais e somente o
>>>> indivíduo ou entidade a quem foi destinada pode utilizá-la. A
>>>> transmissão incorreta da mensagem não acarreta a perda de sua
>>>> confidencialidade. Caso esta mensagem tenha sido recebida por
>>>> engano, solicitamos que o fato seja comunicado ao remetente e que a
>>>> mensagem seja eliminada de seu sistema imediatamente. É vedado a
>>>> qualquer pessoa que não seja o destinatário usar, revelar,
>>>> distribuir ou copiar qualquer parte desta mensagem. Ambiente de
>>>> comunicação sujeito a monitoramento.
>>>>
>>>> This message may include confidential information and only the
>>>> intended addresses have the right to use it as is, or any part of it.
>>>> A wrong transmission does not break its confidentiality. If you've
>>>> received it because of a mistake or erroneous transmission, please
>>>> notify the sender and delete it from your system immediately. This
>>>> communication environment is controlled and monitored.
>>>>
>>>> B2W Digital
>>>>
>>>>
>>>
>>> ________________________________
>>> Esta mensagem pode conter informações confidenciais e somente o
>>> indivíduo ou entidade a quem foi destinada pode utilizá-la. A
>>> transmissão incorreta da mensagem não acarreta a perda de sua
>>> confidencialidade. Caso esta mensagem tenha sido recebida por
>>> engano, solicitamos que o fato seja comunicado ao remetente e que a
>>> mensagem seja eliminada de seu sistema imediatamente. É vedado a
>>> qualquer pessoa que não seja o destinatário usar, revelar,
>>> distribuir ou copiar qualquer parte desta mensagem. Ambiente de comunicação
sujeito a monitoramento.
>>>
>>> This message may include confidential information and only the
>>> intended addresses have the right to use it as is, or any part of it.
>>> A wrong transmission does not break its confidentiality. If you've
>>> received it because of a mistake or erroneous transmission, please
>>> notify the sender and delete it from your system immediately. This
>>> communication environment is controlled and monitored.
>>>
>>> B2W Digital
>>>
>>>
>>
>> ________________________________
>> Esta mensagem pode conter informações confidenciais e somente o indivíduo ou entidade
a quem foi destinada pode utilizá-la. A transmissão incorreta da mensagem não acarreta
a perda de sua confidencialidade. Caso esta mensagem tenha sido recebida por engano, solicitamos
que o fato seja comunicado ao remetente e que a mensagem seja eliminada de seu sistema imediatamente.
É vedado a qualquer pessoa que não seja o destinatário usar, revelar, distribuir ou copiar
qualquer parte desta mensagem. Ambiente de comunicação sujeito a monitoramento.
>>
>> This message may include confidential information and only the intended addresses
have the right to use it as is, or any part of it. A wrong transmission does not break its
confidentiality. If you've received it because of a mistake or erroneous transmission, please
notify the sender and delete it from your system immediately. This communication environment
is controlled and monitored.
>>
>> B2W Digital
>>

Mime
View raw message