sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abraham Elmahrek <...@cloudera.com>
Subject Re: Final Jiras and Release Date
Date Wed, 23 Jul 2014 21:12:23 GMT
Thanks Jarcec!


On Wed, Jul 23, 2014 at 2:03 PM, Jarek Jarcec Cecho <jarcec@apache.org>
wrote:

> Release branch is alive here:
>
> https://github.com/apache/sqoop/tree/branch-1.4.5
>
> Trunk has been switched to version 1.4.6-SNAPSHOT.
>
> Jarcec
>
> On Jul 23, 2014, at 2:00 PM, Jarek Jarcec Cecho <jarcec@apache.org> wrote:
>
> > Too late, I’m already in process of branching :-) But that’s fine - we
> can always cherry-pick whatever will be necessary.
> >
> > Jarcec
> >
> > On Jul 23, 2014, at 1:57 PM, Abraham Elmahrek <abe@cloudera.com> wrote:
> >
> >> Ah actually, it seems there is a broken OraOop test case. Let me see
> what's
> >> going on there before we branch. Jarcec, will you be free around 4PM to
> >> branch?
> >>
> >> -Abe
> >>
> >>
> >> On Wed, Jul 23, 2014 at 1:19 PM, Abraham Elmahrek <abe@cloudera.com>
> wrote:
> >>
> >>> Awesome! Thanks for your hard work guys!
> >>>
> >>> Jarcec, can we use
> >>>
> https://github.com/apache/sqoop/commit/81624ddf3c8ca5834ab015ebafc8b8649ac36ab7
> ?
> >>> In terms of when... let's shoot for 2PM PST?
> >>>
> >>> -Abe
> >>>
> >>>
> >>> On Wed, Jul 23, 2014 at 12:32 PM, Jarek Jarcec Cecho <
> jarcec@apache.org>
> >>> wrote:
> >>>
> >>>> I think that both are in. Do let me know when and from which commit
> you
> >>>> want me to branch Abe :-)
> >>>>
> >>>> Jarcec
> >>>>
> >>>> On Jul 23, 2014, at 10:38 AM, Abraham Elmahrek <abe@cloudera.com>
> wrote:
> >>>>
> >>>>> Hey guys,
> >>>>>
> >>>>> As soon as the Ivy and Avro version changes are in, let's start
the
> >>>>> branching process?
> >>>>>
> >>>>> -Abe
> >>>>>
> >>>>>
> >>>>> On Wed, Jul 23, 2014 at 1:02 AM, Venkat Ranganathan <
> >>>>> vranganathan@hortonworks.com> wrote:
> >>>>>
> >>>>>> Generally  the number of tasks is a hint to InputFormat.getSplits()
> >>>>>> based on number of splits wanted and it can potentially return
> >>>>>> different number of splits.
> >>>>>>
> >>>>>> If you see  ExportInputFormat, the split size is determined
by
> >>>>>> combined file sizes divided by requested num mappers.   This
is a
> >>>>>> integer division and  there is a potential for getting additional
> >>>>>> splits.
> >>>>>>
> >>>>>> Thanks
> >>>>>>
> >>>>>> Venkat
> >>>>>>
> >>>>>> On Tue, Jul 22, 2014 at 11:41 PM, David Robson
> >>>>>> <David.Robson@software.dell.com> wrote:
> >>>>>>> Yes OraOop is not enabled by default - and also this particular
> issue
> >>>> is
> >>>>>> only when using partitioning - so they would have turned on
some
> extra
> >>>>>> flags. Then they also have to be using 1 more mapper than requested
> as
> >>>> you
> >>>>>> said - so it's a pretty specific combination of events - so
I don't
> >>>> think
> >>>>>> we need to hold up the release for it.
> >>>>>>>
> >>>>>>> Do you recall what the behaviour should be in regards to
this - the
> >>>>>> documentation leads me to believe if we request 4 mappers we
should
> >>>> get 4
> >>>>>> mappers? If this is only a request and we may get more - perhaps
we
> >>>> should
> >>>>>> update the documentation about this. Otherwise we could change
the
> >>>> code to
> >>>>>> guarantee no more than 4 mappers?
> >>>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: Abraham Elmahrek [mailto:abe@cloudera.com]
> >>>>>>> Sent: Wednesday, 23 July 2014 3:16 PM
> >>>>>>> To: dev@sqoop.apache.org
> >>>>>>> Subject: Re: Final Jiras and Release Date
> >>>>>>>
> >>>>>>> Hey guys,
> >>>>>>>
> >>>>>>> AFAIK we can fall back onto the original connector? With
that being
> >>>>>> said, OraOop is an awesome connector... So i'm completely open
to
> >>>> waiting
> >>>>>> for a fix if it isn't too large.
> >>>>>>>
> >>>>>>> David, I'm assuming you're referring to the "-m" option?
I do
> recall
> >>>>>> some cases where there may be one more split than desired if
> splitting
> >>>> is
> >>>>>> not clean. Hopefully Venkat has more insight!
> >>>>>>>
> >>>>>>> -Abe
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, Jul 22, 2014 at 9:42 PM, David Robson <
> >>>>>> David.Robson@software.dell.com> wrote:
> >>>>>>>
> >>>>>>>> Hey Venkat,
> >>>>>>>>
> >>>>>>>> I had a look into that issue - it seems the problem
is even though
> >>>> you
> >>>>>>>> request 4 mappers - Sqoop runs 5 mappers. Is the number
of mappers
> >>>>>>>> meant to be guaranteed? For example if I say 4 mappers,
should I
> get
> >>>> 4
> >>>>>>>> mappers? I guess there could be potential to get less
mappers if
> the
> >>>>>>>> data was for example 1 row - then that would only be
processed by
> 1
> >>>>>>>> mapper. But should it be possible to get more mappers
than
> requested?
> >>>>>>>>
> >>>>>>>> OraOop is assuming there will be no more mappers than
requested -
> so
> >>>>>>>> if this is a valid scenario we would need to modify
OraOop. On the
> >>>>>>>> other hand if this is unexpected behaviour then we should
look at
> why
> >>>>>>>> the num mappers is not working?
> >>>>>>>>
> >>>>>>>> Either way I don't think we need to fix it for this
release - as
> it
> >>>>>>>> seems customers would be unlikely to hit this issue.
> >>>>>>>>
> >>>>>>>> David
> >>>>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: Venkat Ranganathan [mailto:vranganathan@hortonworks.com]
> >>>>>>>> Sent: Wednesday, 23 July 2014 1:04 PM
> >>>>>>>> To: dev@sqoop.apache.org
> >>>>>>>> Subject: Re: Final Jiras and Release Date
> >>>>>>>>
> >>>>>>>> Abe
> >>>>>>>>
> >>>>>>>> Do we want to target SQOOP-1388 also for 1.4.5 - the
one that
> Vidya
> >>>>>>>> has raised for one of the Oracle connector export failure?
> >>>>>>>>
> >>>>>>>> My avro patch is in RB.
> >>>>>>>>
> >>>>>>>> Thanks
> >>>>>>>>
> >>>>>>>> Venkat
> >>>>>>>>
> >>>>>>>> On Tue, Jul 22, 2014 at 6:12 PM, Abraham Elmahrek <
> abe@cloudera.com>
> >>>>>>>> wrote:
> >>>>>>>>> Hey guys,
> >>>>>>>>>
> >>>>>>>>> It looks like we still have a couple of Jiras still
open. Let's
> see
> >>>>>>>>> how things look tomorrow, but let's have these be
the last two
> Jiras
> >>>>>>>>> slotted for this release.
> >>>>>>>>>
> >>>>>>>>> -Abe
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Fri, Jul 18, 2014 at 1:20 PM, Venkat Ranganathan
<
> >>>>>>>>> vranganathan@hortonworks.com> wrote:
> >>>>>>>>>
> >>>>>>>>>> Sounds good.   I have already uploaded the patch
for SQOOP-1358.
> >>>> It
> >>>>>>>>>> needs to be reviewed by a committer and then
committed if no
> >>>>>>>>>> further changes are needed (or more work needed
otherwise).
> >>>>>>>>>>
> >>>>>>>>>> Thanks for driving the release!
> >>>>>>>>>>
> >>>>>>>>>> Venkat
> >>>>>>>>>>
> >>>>>>>>>> On Fri, Jul 18, 2014 at 10:58 AM, Jarek Jarcec
Cecho
> >>>>>>>>>> <jarcec@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>> Seems reasonable to me, thank you for driving
the release Abe!
> >>>>>>>>>>>
> >>>>>>>>>>> Jarcec
> >>>>>>>>>>>
> >>>>>>>>>>> On Jul 18, 2014, at 10:40 AM, Abraham Elmahrek
<
> abe@cloudera.com>
> >>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hey guys,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I haven't seen a -1 on this suggestion.
So let's create a
> >>>>>>>>>>>> release branch come Wednesday July 23rd.
> >>>>>>>>>>>>
> >>>>>>>>>>>> -Abe
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Wed, Jul 16, 2014 at 11:38 AM, Abraham
Elmahrek
> >>>>>>>>>>>> <abe@cloudera.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Are there any objections to this
branch date?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> -Abe
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Mon, Jul 14, 2014 at 1:22 PM,
Abraham Elmahrek
> >>>>>>>>>>>>> <abe@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hey guys,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> It looks like there are a couple
of Jiras that are in
> progress
> >>>>>>>>>>>>>> and slotted for the 1.4.5 release.
If we aim for July 23rd
> to
> >>>>>>>>>>>>>> create a
> >>>>>>>>>> release
> >>>>>>>>>>>>>> branch, does that give every
one enough time to finish up
> what
> >>>>>>>>>>>>>> they're working on?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> See https://issues.apache.org/jira/browse/SQOOP-1353
for
> more
> >>>>>>>>>>>>>> information.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> -Abe
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> CONFIDENTIALITY NOTICE
> >>>>>>>>>> NOTICE: This message is intended for the use
of the individual
> or
> >>>>>>>>>> entity to which it is addressed and may contain
information
> that is
> >>>>>>>>>> confidential, privileged and exempt from disclosure
under
> >>>>>>>>>> applicable law. If the reader of this message
is not the
> intended
> >>>>>>>>>> recipient, you are hereby notified that any
printing, copying,
> >>>>>>>>>> dissemination, distribution, disclosure or forwarding
of this
> >>>>>>>>>> communication is strictly prohibited. If you
have received this
> >>>>>>>>>> communication in error, please contact the sender
immediately
> and
> >>>>>>>>>> delete it from your
> >>>>>>>> system. Thank You.
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> CONFIDENTIALITY NOTICE
> >>>>>>>> NOTICE: This message is intended for the use of the
individual or
> >>>>>>>> entity to which it is addressed and may contain information
that
> is
> >>>>>>>> confidential, privileged and exempt from disclosure
under
> applicable
> >>>>>>>> law. If the reader of this message is not the intended
recipient,
> you
> >>>>>>>> are hereby notified that any printing, copying, dissemination,
> >>>>>>>> distribution, disclosure or forwarding of this communication
is
> >>>>>>>> strictly prohibited. If you have received this communication
in
> >>>> error,
> >>>>>>>> please contact the sender immediately and delete it
from your
> system.
> >>>>>> Thank You.
> >>>>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> CONFIDENTIALITY NOTICE
> >>>>>> NOTICE: This message is intended for the use of the individual
or
> >>>> entity to
> >>>>>> which it is addressed and may contain information that is
> confidential,
> >>>>>> privileged and exempt from disclosure under applicable law.
If the
> >>>> reader
> >>>>>> of this message is not the intended recipient, you are hereby
> notified
> >>>> that
> >>>>>> any printing, copying, dissemination, distribution, disclosure
or
> >>>>>> forwarding of this communication is strictly prohibited. If
you have
> >>>>>> received this communication in error, please contact the sender
> >>>> immediately
> >>>>>> and delete it from your system. Thank You.
> >>>>>>
> >>>>
> >>>>
> >>>
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message