sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pratik khadloya <tispra...@gmail.com>
Subject Re: Sqoop with Oozie
Date Fri, 19 Sep 2014 19:57:29 GMT
You may need to set the following property:

  // The minimum number of active instances in an Oracle RAC required for
OraOop
  // to use dynamically generated JDBC URLs...
  public static final String ORAOOP_MIN_RAC_ACTIVE_INSTANCES =
"oraoop.min.rac.active.instances";
  public static final int
MIN_NUM_RAC_ACTIVE_INSTANCES_FOR_DYNAMIC_JDBC_URLS = 2;


    // If there are multiple JDBC URLs that work okay for the RAC, then
we'll
    // make use of them...
    if (jdbcOracleActiveThinConnections.size() < OraOopUtilities
        .getMinNumberOfOracleRacActiveInstancesForDynamicJdbcUrlUse(jobData
            .getSqoopOptions().getConf())) {
      LOG.info(String
          .format(
              "%s will not attempt to load-balance sessions across
instances "
            + "of an Oracle RAC - as multiple JDBC URLs to the "
            + "Oracle RAC could not be dynamically generated.",
              OraOopConstants.ORAOOP_PRODUCT_NAME));
      return;
    } else {
      StringBuilder msg = new StringBuilder();
      msg.append(String
          .format(
              "%s will load-balance sessions across the following instances
of"
            + "the Oracle RAC:\n",
              OraOopConstants.ORAOOP_PRODUCT_NAME));

      for (OraOopUtilities.JdbcOracleThinConnection thinConnection
               : jdbcOracleActiveThinConnections) {
        msg.append(String.format("\tInstance: %s \t URL: %s\n",
            thinConnection.getSid(), thinConnection.toString()));
      }
      LOG.info(msg.toString());
    }

On Fri, Sep 19, 2014 at 12:44 PM, Felipe Naberezny Lopes <
felipe.naberezny@b2wdigital.com> wrote:

>  David,
>
>
>
> I updated to the Sqoop 1.4.5.
>
>
>
> It works great. But I need to connect with a Oracle RAC.
>
>
>
> The process is throwing the given error:
>
>
>
> 18124 [main] INFO org.apache.sqoop.manager.oracle.OraOopManagerFactory - Data Connector
for Oracle and Hadoop will not attempt to load-balance sessions across instances of an Oracle
RAC - as multiple JDBC URLs to the Oracle RAC could not be dynamically generated.
>
>
>
> Don’t have Idea how to fix it.
>
>
>
>
>
> *De:* David Robson [mailto:David.Robson@software.dell.com]
> *Enviada em:* quinta-feira, 18 de setembro de 2014 20:50
> *Para:* user@sqoop.apache.org
> *Assunto:* RE: Sqoop with Oozie
>
>
>
> Are you able to upgrade to Sqoop 1.4.5? As Sqoop 1.4.5 now includes OraOop
> – so you don’t need to manually install it – would make it a lot easier I
> believe.
>
>
>
> The error you are getting is because you are trying to use the connection
> factory as the manager factory – probably somewhere in the config it has
> the wrong class name. Anyway if you can upgrade (or are already using
> 1.4.5) then you can un-install OraOop and just add the “—direct” option to
> your Sqoop job and it will use OraOop.
>
>
>
> *From:* Chalcy [mailto:chalcy@gmail.com <chalcy@gmail.com>]
> *Sent:* Friday, 19 September 2014 7:30 AM
> *To:* user@sqoop.apache.org
> *Subject:* Re: Sqoop with Oozie
>
>
>
> The oraoop jar needs to be in hdfs for oozie to run I believe.  Please
> copy the oraoop jar to hdfs and give that path to oozie conf.
>
>
>
>
>
> On Thu, Sep 18, 2014 at 2:32 PM, Felipe Naberezny Lopes <
> felipe.naberezny@b2wdigital.com> wrote:
>
> Without Oozie the plugin works and the Oraoop jar is in the Oozie share
> lib.
>
> I'm not using a Cloudera distribution for Hadoop.
>
> Maybe that is the problem. I don't know anything else to check.
>
> -----Mensagem original-----
> De: Gwen Shapira [mailto:gshapira@cloudera.com]
> Enviada em: quinta-feira, 18 de setembro de 2014 15:25
>
> Para: user@sqoop.apache.org
> Assunto: Re: Sqoop with Oozie
>
> Also, does everything work without Oozie?
>
> On Thu, Sep 18, 2014 at 11:24 AM, Gwen Shapira <gshapira@cloudera.com>
> wrote:
> > Do you have the Oraoop jar in Oozie's share lib path?
> >
> > On Thu, Sep 18, 2014 at 10:46 AM, Felipe Naberezny Lopes
> > <felipe.naberezny@b2wdigital.com> wrote:
> >> Gwen,
> >>
> >> It's throwing the same exception:
> >>
> >> 4569 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool  - Got error
> >> creating database manager: java.lang.ClassCastException:
> >> com.quest.oraoop.OracleConnectionFactory cannot be cast to
> >> com.cloudera.sqoop.manager.ManagerFactory
> >>
> >> -----Mensagem original-----
> >> De: Gwen Shapira [mailto:gshapira@cloudera.com] Enviada em:
> >> quinta-feira, 18 de setembro de 2014 14:32
> >> Para: user@sqoop.apache.org
> >> Assunto: Re: Sqoop with Oozie
> >>
> >> I believe you need to add this:
> >>
> >>  import -D
> >> sqoop.connection.factories=com.cloudera.sqoop.manager.DefaultManagerF
> >> actory,com.quest.oraoop.OraOopManagerFactory
> >>
> >> On Thu, Sep 18, 2014 at 10:30 AM, Felipe Naberezny Lopes <
> felipe.naberezny@b2wdigital.com> wrote:
> >>> The plugin is QuestDataConnector for Oracle and Hadoop. The plugin
> >>> is a connection manager between Oracle and Hadoop.
> >>>
> >>>
> >>>
> >>> When I install it in my Sqoop Client Machine, it stays in
> >>> SQOOP_HOME/conf/managers.d
> >>>
> >>>
> >>>
> >>> I’m trying to make Oozie use the plugin.
> >>>
> >>>
> >>>
> >>> The jars are in my Hadoop cluster.
> >>>
> >>>
> >>>
> >>> I set the property “sqoop.connection.factories “ with the path to
> >>> the jars of plugin.
> >>>
> >>>
> >>>
> >>> Apparently, Oozie is using that configuration, but it’s throwing the
> >>> given
> >>> error:
> >>>
> >>>
> >>>
> >>> “ 5542 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool  - Got error
> >>> creating database manager: java.lang.ClassCastException:
> >>> com.quest.oraoop.OracleConnectionFactory cannot be cast to
> >>> com.cloudera.sqoop.manager.ManagerFactory “
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> De: pratik khadloya [mailto:tispratik@gmail.com] Enviada em:
> >>> quinta-feira, 18 de setembro de 2014 14:15
> >>>
> >>>
> >>> Para: user@sqoop.apache.org
> >>> Assunto: Re: Sqoop with Oozie
> >>>
> >>>
> >>>
> >>> Can you share the plugin name and what error (with stack trace) you
> >>> are getting?
> >>>
> >>>
> >>>
> >>> ~Pratik
> >>>
> >>>
> >>>
> >>> On Thu, Sep 18, 2014 at 10:04 AM, Felipe Naberezny Lopes
> >>> <felipe.naberezny@b2wdigital.com> wrote:
> >>>
> >>> I'm connecting Sqoop with Oracle.
> >>>
> >>> There is a plugin made by Toad that improve the performance of data
> import.
> >>>
> >>> I'm trying to use it, but I don't know how to make Oozie use that
> plugin.
> >>>
> >>> You know something that solves this?
> >>>
> >>> -----Mensagem original-----
> >>> De: Gwen Shapira [mailto:gshapira@cloudera.com] Enviada em:
> >>> quinta-feira, 18 de setembro de 2014 13:55
> >>> Para: user@sqoop.apache.org
> >>> Assunto: Re: Sqoop with Oozie
> >>>
> >>>
> >>> We typically don't configure SQOOP_CONF_DIR for Oozie. Rather, we
> >>> make sure the list of parameters we include in the Sqoop action
> >>> includes all the relevant configurations.
> >>>
> >>> Gwen
> >>>
> >>> 2014-09-18 7:18 GMT-07:00 Felipe Naberezny Lopes
> >>> <felipe.naberezny@b2wdigital.com>:
> >>>> Hi,
> >>>>
> >>>>
> >>>>
> >>>> How do I set SQOOP_CONF_DIR using Oozie?
> >>>>
> >>>>
> >>>>
> >>>> Any idea?
> >>>>
> >>>>
> >>>>
> >>>> Tks
> >>>>
> >>>>
> >>>>
> >>>> Felipe Naberezny
> >>>>
> >>>> B2W Digital / B2W Labs
> >>>>
> >>>> +55 11 3454-3021 / +55 11 9 8561-8525
> >>>>
> >>>> felipe.naberezny@b2wdigital.com
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> ________________________________
> >>>>
> >>>> Esta mensagem pode conter informações confidenciais e somente o
> >>>> indivíduo ou entidade a quem foi destinada pode utilizá-la. A
> >>>> transmissão incorreta da mensagem não acarreta a perda de sua
> >>>> confidencialidade. Caso esta mensagem tenha sido recebida por
> >>>> engano, solicitamos que o fato seja comunicado ao remetente e que a
> >>>> mensagem seja eliminada de seu sistema imediatamente. É vedado a
> >>>> qualquer pessoa que não seja o destinatário usar, revelar,
> >>>> distribuir ou copiar qualquer parte desta mensagem. Ambiente de
> >>>> comunicação sujeito a monitoramento.
> >>>>
> >>>> This message may include confidential information and only the
> >>>> intended addresses have the right to use it as is, or any part of it.
> >>>> A wrong transmission does not break its confidentiality. If you've
> >>>> received it because of a mistake or erroneous transmission, please
> >>>> notify the sender and delete it from your system immediately. This
> >>>> communication environment is controlled and monitored.
> >>>>
> >>>> B2W Digital
> >>>>
> >>>>
> >>>
> >>> ________________________________
> >>> Esta mensagem pode conter informações confidenciais e somente o
> >>> indivíduo ou entidade a quem foi destinada pode utilizá-la. A
> >>> transmissão incorreta da mensagem não acarreta a perda de sua
> >>> confidencialidade. Caso esta mensagem tenha sido recebida por
> >>> engano, solicitamos que o fato seja comunicado ao remetente e que a
> >>> mensagem seja eliminada de seu sistema imediatamente. É vedado a
> >>> qualquer pessoa que não seja o destinatário usar, revelar,
> >>> distribuir ou copiar qualquer parte desta mensagem. Ambiente de
> comunicação sujeito a monitoramento.
> >>>
> >>> This message may include confidential information and only the
> >>> intended addresses have the right to use it as is, or any part of it.
> >>> A wrong transmission does not break its confidentiality. If you've
> >>> received it because of a mistake or erroneous transmission, please
> >>> notify the sender and delete it from your system immediately. This
> >>> communication environment is controlled and monitored.
> >>>
> >>> B2W Digital
> >>>
> >>>
> >>
> >> ________________________________
> >> Esta mensagem pode conter informações confidenciais e somente o
> indivíduo ou entidade a quem foi destinada pode utilizá-la. A transmissão
> incorreta da mensagem não acarreta a perda de sua confidencialidade. Caso
> esta mensagem tenha sido recebida por engano, solicitamos que o fato seja
> comunicado ao remetente e que a mensagem seja eliminada de seu sistema
> imediatamente. É vedado a qualquer pessoa que não seja o destinatário usar,
> revelar, distribuir ou copiar qualquer parte desta mensagem. Ambiente de
> comunicação sujeito a monitoramento.
> >>
> >> This message may include confidential information and only the intended
> addresses have the right to use it as is, or any part of it. A wrong
> transmission does not break its confidentiality. If you've received it
> because of a mistake or erroneous transmission, please notify the sender
> and delete it from your system immediately. This communication environment
> is controlled and monitored.
> >>
> >> B2W Digital
> >>
>
>
>

Mime
View raw message