sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gwen Shapira <gshap...@cloudera.com>
Subject Re: Sqoop documentation
Date Fri, 20 Jun 2014 15:27:11 GMT
+1 to --direct

Which leads right back to the documentation.

Currently the Netezza direct connector is documented in section 24 - Notes
for Specific Connectors.
Which seems like the right place, and I recommend adding Oraoop there too
(possibly with link to additional doc if this gets too long).

The documentation for MySQL and Postgres direct connectors is all over the
place.
I'll open a new Jira to refactor the docs and consolidate the "--direct"
documentation into section 24. This can be done in parallel to the Oraoop
work.

Does this make sense?

Gwen



On Thu, Jun 19, 2014 at 8:54 PM, Venkat Ranganathan <
vranganathan@hortonworks.com> wrote:

> Why don't we use the --direct mode to say OraOop to be the connector
> and the default to use the existing oracle connector?
>
> Also, as I initially mentioned, I had made changes to OraOop to
> support Oracle wallets.   I would like to get that in after your
> changes are in.
>
> Thanks
>
> Venkat
>
> On Thu, Jun 19, 2014 at 8:22 PM, David Robson
> <David.Robson@software.dell.com> wrote:
> > Yes it will be a bit confusing for users while there are two connectors
> - perhaps we should disable OraOop by default until they are merged into
> one connector? At least the user will only have to change a config
> parameter to use OraOop - where previously they had to install it.
> >
> > At the moment they handle things differently as you said - and depending
> on things like if there is only one mapper, or if they are importing a
> view, they will use the default connector. Could be quite confusing for a
> user who doesn't understand the reasons why one import has dates as longs
> and the other as strings. At least if they have manually installed OraOop
> in the past they have read the user's guide so know what to expect.
> >
> > As part of merging them into one connector we really need to make sure
> everything is consistent between the two - I'm thinking this will be a bit
> of work and I wanted to do it separately so at least we isolate the QA work
> to an initial regression test of OraOop. Then we can code / test the single
> connector at a later date.
> >
> > If we were to document the parameters on the wiki instead of in the
> documentation - how do you envisage this would be structured - I can't
> really see an example of something similar already on the wiki. It seems
> like other connectors have put the information directly in the
> documentation eg:
> http://sqoop.apache.org/docs/1.4.4/SqoopUserGuide.html#_microsoft_sql_connector
> >
> > But the sections are quite short whereas the OraOop one could end up
> being quite long. It looks like in Sqoop 2 the documentation will be split
> into multiple documents - perhaps a separate page dedicated to all Oracle /
> Sqoop topics could be included?
> >
> > -----Original Message-----
> > From: Gwen Shapira [mailto:gshapira@cloudera.com]
> > Sent: Wednesday, 18 June 2014 3:00 PM
> > To: dev@sqoop.apache.org
> > Subject: Re: Sqoop documentation
> >
> > And on similar/different topic:
> >
> > The biggest risk IMO is that once we add Oraoop as default Oracle
> manager, users without "select dictionary" privileges will get "table not
> found" on jobs that prior to the upgrade were successful. This breaks
> backward compatibility in a pretty serious way.
> >
> > IMO, this has to be fixed before we commit the merge (and yes, I need to
> add this comment to the review board)
> >
> > Gwen
> >
> >
> > On Tue, Jun 17, 2014 at 9:57 PM, Gwen Shapira <gshapira@cloudera.com>
> wrote:
> >
> >> Random thoughts:
> >>
> >> The good news is that the first 10 pages ("Installing Oraoop") are
> >> redundant.
> >>
> >> I'd like to look into cases where the default behavior will change
> >> (timestamp processing for instance) and in which case we must amend
> >> the current docs.
> >> In cases where Oraoop offers extra functionality that the users may or
> >> may not use (oraoop merge or partitions) we can refer users to a wiki
> >> or another external source.
> >>
> >> Gwen
> >>
> >>
> >> On Tue, Jun 17, 2014 at 9:26 PM, David Robson <
> >> David.Robson@software.dell.com> wrote:
> >>
> >>> Hi,
> >>>
> >>> I am having a look at how we could integrate the OraOop documentation
> >>> into Sqoop. Currently it is a Flare project which we generate a PDF
> >>> from - obviously this doesn't really fit in with the current Sqoop
> documentation.
> >>>
> >>> So currently it looks like it is written in asciidoc and converted
> >>> into docbook, then into html. We could create an asciidoc file
> >>> related to OraOop
> >>> - the problem is I'm not sure how well this would fit in to the
> >>> current documentation - it is already one long page and we would
> >>> probably be adding another 20-30 pages worth of content in.
> >>>
> >>> You also have a Confluence wiki - we could create a page (or pages)
> >>> there with information.
> >>>
> >>> What sort of requirements are there for documentation - has someone
> >>> got some ideas of how they would like to see the current OraOop
> >>> documentation integrated in?
> >>> https://github.com/QuestSoftwareTCD/OracleSQOOPconnector/blob/master/
> >>> docs/oraoopuserguide.pdf?raw=true
> >>>
> >>> Thanks,
> >>>
> >>> David
> >>>
> >>
> >>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message