sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gwen Shapira <gshap...@cloudera.com>
Subject Re: Sqoop with Oozie
Date Mon, 22 Sep 2014 17:00:44 GMT
Awesome! Thanks for updating us with the solution :)

On Mon, Sep 22, 2014 at 7:28 AM, Felipe Naberezny Lopes
<felipe.naberezny@b2wdigital.com> wrote:
> Gwen,
>
> I set this property "oraoop.oracle.rac.service.name" with my Oracle service name and
it works.
>
> Thanks for the help.
>
> -----Mensagem original-----
> De: Gwen Shapira [mailto:gshapira@cloudera.com]
> Enviada em: sexta-feira, 19 de setembro de 2014 17:30
> Para: user@sqoop.apache.org
> Assunto: Re: Sqoop with Oozie
>
> Its an info, not an error. All it means is that OraOop will not loadbalance sessions.
>
> Are you sure you need to fix it?
>
> On Fri, Sep 19, 2014 at 12:44 PM, Felipe Naberezny Lopes <felipe.naberezny@b2wdigital.com>
wrote:
>> David,
>>
>>
>>
>> I updated to the Sqoop 1.4.5.
>>
>>
>>
>> It works great. But I need to connect with a Oracle RAC.
>>
>>
>>
>> The process is throwing the given error:
>>
>>
>>
>> 18124 [main] INFO org.apache.sqoop.manager.oracle.OraOopManagerFactory
>> - Data Connector for Oracle and Hadoop will not attempt to
>> load-balance sessions across instances of an Oracle RAC - as multiple
>> JDBC URLs to the Oracle RAC could not be dynamically generated.
>>
>>
>>
>> Don’t have Idea how to fix it.
>>
>>
>>
>>
>>
>> De: David Robson [mailto:David.Robson@software.dell.com]
>> Enviada em: quinta-feira, 18 de setembro de 2014 20:50
>> Para: user@sqoop.apache.org
>> Assunto: RE: Sqoop with Oozie
>>
>>
>>
>> Are you able to upgrade to Sqoop 1.4.5? As Sqoop 1.4.5 now includes
>> OraOop – so you don’t need to manually install it – would make it a
>> lot easier I believe.
>>
>>
>>
>> The error you are getting is because you are trying to use the
>> connection factory as the manager factory – probably somewhere in the
>> config it has the wrong class name. Anyway if you can upgrade (or are
>> already using 1.4.5) then you can un-install OraOop and just add the
>> “—direct” option to your Sqoop job and it will use OraOop.
>>
>>
>>
>> From: Chalcy [mailto:chalcy@gmail.com]
>> Sent: Friday, 19 September 2014 7:30 AM
>> To: user@sqoop.apache.org
>> Subject: Re: Sqoop with Oozie
>>
>>
>>
>> The oraoop jar needs to be in hdfs for oozie to run I believe.  Please
>> copy the oraoop jar to hdfs and give that path to oozie conf.
>>
>>
>>
>>
>>
>> On Thu, Sep 18, 2014 at 2:32 PM, Felipe Naberezny Lopes
>> <felipe.naberezny@b2wdigital.com> wrote:
>>
>> Without Oozie the plugin works and the Oraoop jar is in the Oozie share lib.
>>
>> I'm not using a Cloudera distribution for Hadoop.
>>
>> Maybe that is the problem. I don't know anything else to check.
>>
>> -----Mensagem original-----
>> De: Gwen Shapira [mailto:gshapira@cloudera.com] Enviada em:
>> quinta-feira, 18 de setembro de 2014 15:25
>>
>> Para: user@sqoop.apache.org
>> Assunto: Re: Sqoop with Oozie
>>
>> Also, does everything work without Oozie?
>>
>> On Thu, Sep 18, 2014 at 11:24 AM, Gwen Shapira <gshapira@cloudera.com>
>> wrote:
>>> Do you have the Oraoop jar in Oozie's share lib path?
>>>
>>> On Thu, Sep 18, 2014 at 10:46 AM, Felipe Naberezny Lopes
>>> <felipe.naberezny@b2wdigital.com> wrote:
>>>> Gwen,
>>>>
>>>> It's throwing the same exception:
>>>>
>>>> 4569 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool  - Got error
>>>> creating database manager: java.lang.ClassCastException:
>>>> com.quest.oraoop.OracleConnectionFactory cannot be cast to
>>>> com.cloudera.sqoop.manager.ManagerFactory
>>>>
>>>> -----Mensagem original-----
>>>> De: Gwen Shapira [mailto:gshapira@cloudera.com] Enviada em:
>>>> quinta-feira, 18 de setembro de 2014 14:32
>>>> Para: user@sqoop.apache.org
>>>> Assunto: Re: Sqoop with Oozie
>>>>
>>>> I believe you need to add this:
>>>>
>>>>  import -D
>>>> sqoop.connection.factories=com.cloudera.sqoop.manager.DefaultManager
>>>> F actory,com.quest.oraoop.OraOopManagerFactory
>>>>
>>>> On Thu, Sep 18, 2014 at 10:30 AM, Felipe Naberezny Lopes
>>>> <felipe.naberezny@b2wdigital.com> wrote:
>>>>> The plugin is QuestDataConnector for Oracle and Hadoop. The plugin
>>>>> is a connection manager between Oracle and Hadoop.
>>>>>
>>>>>
>>>>>
>>>>> When I install it in my Sqoop Client Machine, it stays in
>>>>> SQOOP_HOME/conf/managers.d
>>>>>
>>>>>
>>>>>
>>>>> I’m trying to make Oozie use the plugin.
>>>>>
>>>>>
>>>>>
>>>>> The jars are in my Hadoop cluster.
>>>>>
>>>>>
>>>>>
>>>>> I set the property “sqoop.connection.factories “ with the path to
>>>>> the jars of plugin.
>>>>>
>>>>>
>>>>>
>>>>> Apparently, Oozie is using that configuration, but it’s throwing
>>>>> the given
>>>>> error:
>>>>>
>>>>>
>>>>>
>>>>> “ 5542 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool  - Got
>>>>> error creating database manager: java.lang.ClassCastException:
>>>>> com.quest.oraoop.OracleConnectionFactory cannot be cast to
>>>>> com.cloudera.sqoop.manager.ManagerFactory “
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> De: pratik khadloya [mailto:tispratik@gmail.com] Enviada em:
>>>>> quinta-feira, 18 de setembro de 2014 14:15
>>>>>
>>>>>
>>>>> Para: user@sqoop.apache.org
>>>>> Assunto: Re: Sqoop with Oozie
>>>>>
>>>>>
>>>>>
>>>>> Can you share the plugin name and what error (with stack trace) you
>>>>> are getting?
>>>>>
>>>>>
>>>>>
>>>>> ~Pratik
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Sep 18, 2014 at 10:04 AM, Felipe Naberezny Lopes
>>>>> <felipe.naberezny@b2wdigital.com> wrote:
>>>>>
>>>>> I'm connecting Sqoop with Oracle.
>>>>>
>>>>> There is a plugin made by Toad that improve the performance of data
>>>>> import.
>>>>>
>>>>> I'm trying to use it, but I don't know how to make Oozie use that
>>>>> plugin.
>>>>>
>>>>> You know something that solves this?
>>>>>
>>>>> -----Mensagem original-----
>>>>> De: Gwen Shapira [mailto:gshapira@cloudera.com] Enviada em:
>>>>> quinta-feira, 18 de setembro de 2014 13:55
>>>>> Para: user@sqoop.apache.org
>>>>> Assunto: Re: Sqoop with Oozie
>>>>>
>>>>>
>>>>> We typically don't configure SQOOP_CONF_DIR for Oozie. Rather, we
>>>>> make sure the list of parameters we include in the Sqoop action
>>>>> includes all the relevant configurations.
>>>>>
>>>>> Gwen
>>>>>
>>>>> 2014-09-18 7:18 GMT-07:00 Felipe Naberezny Lopes
>>>>> <felipe.naberezny@b2wdigital.com>:
>>>>>> Hi,
>>>>>>
>>>>>>
>>>>>>
>>>>>> How do I set SQOOP_CONF_DIR using Oozie?
>>>>>>
>>>>>>
>>>>>>
>>>>>> Any idea?
>>>>>>
>>>>>>
>>>>>>
>>>>>> Tks
>>>>>>
>>>>>>
>>>>>>
>>>>>> Felipe Naberezny
>>>>>>
>>>>>> B2W Digital / B2W Labs
>>>>>>
>>>>>> +55 11 3454-3021 / +55 11 9 8561-8525
>>>>>>
>>>>>> felipe.naberezny@b2wdigital.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ________________________________
>>>>>>
>>>>>> Esta mensagem pode conter informações confidenciais e somente o
>>>>>> indivíduo ou entidade a quem foi destinada pode utilizá-la. A
>>>>>> transmissão incorreta da mensagem não acarreta a perda de sua
>>>>>> confidencialidade. Caso esta mensagem tenha sido recebida por
>>>>>> engano, solicitamos que o fato seja comunicado ao remetente e que
>>>>>> a mensagem seja eliminada de seu sistema imediatamente. É vedado
a
>>>>>> qualquer pessoa que não seja o destinatário usar, revelar,
>>>>>> distribuir ou copiar qualquer parte desta mensagem. Ambiente de
>>>>>> comunicação sujeito a monitoramento.
>>>>>>
>>>>>> This message may include confidential information and only the
>>>>>> intended addresses have the right to use it as is, or any part of
it.
>>>>>> A wrong transmission does not break its confidentiality. If you've
>>>>>> received it because of a mistake or erroneous transmission, please
>>>>>> notify the sender and delete it from your system immediately. This
>>>>>> communication environment is controlled and monitored.
>>>>>>
>>>>>> B2W Digital
>>>>>>
>>>>>>
>>>>>
>>>>> ________________________________
>>>>> Esta mensagem pode conter informações confidenciais e somente o
>>>>> indivíduo ou entidade a quem foi destinada pode utilizá-la. A
>>>>> transmissão incorreta da mensagem não acarreta a perda de sua
>>>>> confidencialidade. Caso esta mensagem tenha sido recebida por
>>>>> engano, solicitamos que o fato seja comunicado ao remetente e que a
>>>>> mensagem seja eliminada de seu sistema imediatamente. É vedado a
>>>>> qualquer pessoa que não seja o destinatário usar, revelar,
>>>>> distribuir ou copiar qualquer parte desta mensagem. Ambiente de
>>>>> comunicação sujeito a monitoramento.
>>>>>
>>>>> This message may include confidential information and only the
>>>>> intended addresses have the right to use it as is, or any part of it.
>>>>> A wrong transmission does not break its confidentiality. If you've
>>>>> received it because of a mistake or erroneous transmission, please
>>>>> notify the sender and delete it from your system immediately. This
>>>>> communication environment is controlled and monitored.
>>>>>
>>>>> B2W Digital
>>>>>
>>>>>
>>>>
>>>> ________________________________
>>>> Esta mensagem pode conter informações confidenciais e somente o
>>>> indivíduo ou entidade a quem foi destinada pode utilizá-la. A
>>>> transmissão incorreta da mensagem não acarreta a perda de sua
>>>> confidencialidade. Caso esta mensagem tenha sido recebida por
>>>> engano, solicitamos que o fato seja comunicado ao remetente e que a
>>>> mensagem seja eliminada de seu sistema imediatamente. É vedado a
>>>> qualquer pessoa que não seja o destinatário usar, revelar,
>>>> distribuir ou copiar qualquer parte desta mensagem. Ambiente de comunicação
sujeito a monitoramento.
>>>>
>>>> This message may include confidential information and only the
>>>> intended addresses have the right to use it as is, or any part of
>>>> it. A wrong transmission does not break its confidentiality. If
>>>> you've received it because of a mistake or erroneous transmission,
>>>> please notify the sender and delete it from your system immediately.
>>>> This communication environment is controlled and monitored.
>>>>
>>>> B2W Digital
>>>>
>>
>>
>
> ________________________________
> Esta mensagem pode conter informações confidenciais e somente o indivíduo ou entidade
a quem foi destinada pode utilizá-la. A transmissão incorreta da mensagem não acarreta
a perda de sua confidencialidade. Caso esta mensagem tenha sido recebida por engano, solicitamos
que o fato seja comunicado ao remetente e que a mensagem seja eliminada de seu sistema imediatamente.
É vedado a qualquer pessoa que não seja o destinatário usar, revelar, distribuir ou copiar
qualquer parte desta mensagem. Ambiente de comunicação sujeito a monitoramento.
>
> This message may include confidential information and only the intended addresses have
the right to use it as is, or any part of it. A wrong transmission does not break its confidentiality.
If you've received it because of a mistake or erroneous transmission, please notify the sender
and delete it from your system immediately. This communication environment is controlled and
monitored.
>
> B2W Digital
>

Mime
View raw message